2024-01-31T05:01:24.795 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-01-31T05:01:24.795 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495 --owner scheduled_vshankar@teuthology --name vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} -- /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495/orig.config.yaml 2024-01-31T05:01:24.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-31T05:01:24.954 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495 branch: wip-vshankar-testing-20240125.044502 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7539495' kernel: client: sha1: distro flavor: default kdb: true sha1: cd42140e628558a450ba1d39a9d0835a2fa62cda ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240125.044502 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 661946e7044723130c03627323f6a879a9848cb5 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 661946e7044723130c03627323f6a879a9848cb5 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240125.044502 sha1: 661946e7044723130c03627323f6a879a9848cb5 owner: scheduled_vshankar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9619 sha1: 661946e7044723130c03627323f6a879a9848cb5 sleep_before_teardown: 0 subset: 293/1024 suite: fs suite_branch: wip-vshankar-testing-20240125.044502 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 661946e7044723130c03627323f6a879a9848cb5 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNSxe+NPNLn2hNrNVy6jyPJzBOyr5pUwBsl+J1EMRy/k41qx1/zLWtQoT+TEgppHcHPgn2ctwSeUCDXAQJjpL0Q= smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMHl72PSsbKgQJaUREZyo9KxvIc+cVgWoxkXfYzE0Zxg5U7AvqHorr/G8Yk+4nL75wm6Pvb3oCKBkTNjb/O6drg= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC9j+A9OPXsHtLkVf29d1PbaVSgqkap1n/t4GAgqgP/OQcNQ8qF93XQFRqQO/n7pX/WRFarKi1NFu1L3JuBotS4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-01-31_04:15:12 tube: smithi use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-31T05:01:24.955 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa; will attempt to use it 2024-01-31T05:01:24.955 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks 2024-01-31T05:01:24.955 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-31T05:01:24.960 INFO:teuthology.task.internal:Checking packages... 2024-01-31T05:01:24.983 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '661946e7044723130c03627323f6a879a9848cb5' 2024-01-31T05:01:24.983 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-31T05:01:24.983 INFO:teuthology.packaging:ref: None 2024-01-31T05:01:24.983 INFO:teuthology.packaging:tag: None 2024-01-31T05:01:24.983 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240125.044502 2024-01-31T05:01:24.983 INFO:teuthology.packaging:sha1: 661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:01:24.983 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240125.044502 2024-01-31T05:01:25.131 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-885.g661946e7 2024-01-31T05:01:25.133 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-31T05:01:25.140 INFO:teuthology.task.internal:no buildpackages task found 2024-01-31T05:01:25.140 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-31T05:01:25.200 INFO:teuthology.task.internal:Saving configuration 2024-01-31T05:01:25.222 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-31T05:01:25.253 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-31T05:01:25.275 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-31 04:55:51.352273', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNSxe+NPNLn2hNrNVy6jyPJzBOyr5pUwBsl+J1EMRy/k41qx1/zLWtQoT+TEgppHcHPgn2ctwSeUCDXAQJjpL0Q='} 2024-01-31T05:01:25.297 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-31 04:55:51.355689', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMHl72PSsbKgQJaUREZyo9KxvIc+cVgWoxkXfYzE0Zxg5U7AvqHorr/G8Yk+4nL75wm6Pvb3oCKBkTNjb/O6drg='} 2024-01-31T05:01:25.317 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-31 04:55:51.354172', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC9j+A9OPXsHtLkVf29d1PbaVSgqkap1n/t4GAgqgP/OQcNQ8qF93XQFRqQO/n7pX/WRFarKi1NFu1L3JuBotS4='} 2024-01-31T05:01:25.317 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-31T05:01:25.323 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-01-31T05:01:25.323 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-01-31T05:01:25.323 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-01-31T05:01:25.323 INFO:teuthology.run_tasks:Running task console_log... 2024-01-31T05:01:25.447 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe83ded8160>, signals=[15]) 2024-01-31T05:01:25.447 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-31T05:01:25.453 INFO:teuthology.task.internal:Opening connections... 2024-01-31T05:01:25.453 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-01-31T05:01:25.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T05:01:25.562 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2024-01-31T05:01:25.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T05:01:25.651 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2024-01-31T05:01:25.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T05:01:25.740 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-31T05:01:25.747 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-01-31T05:01:25.766 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-01-31T05:01:25.766 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-01-31T05:01:25.823 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-01-31T05:01:25.823 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-01-31T05:01:25.823 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-01-31T05:01:25.823 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-01-31T05:01:25.824 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-01-31T05:01:25.824 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-01-31T05:01:25.824 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-31T05:01:25.824 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-01-31T05:01:25.824 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-01-31T05:01:25.824 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-31T05:01:25.824 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-01-31T05:01:25.824 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-31T05:01:25.824 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-31T05:01:25.824 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-31T05:01:25.825 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-01-31T05:01:25.863 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2024-01-31T05:01:25.881 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2024-01-31T05:01:25.882 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2024-01-31T05:01:25.939 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="CentOS Stream" 2024-01-31T05:01:25.939 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="9" 2024-01-31T05:01:25.939 INFO:teuthology.orchestra.run.smithi156.stdout:ID="centos" 2024-01-31T05:01:25.939 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="rhel fedora" 2024-01-31T05:01:25.939 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="9" 2024-01-31T05:01:25.939 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el9" 2024-01-31T05:01:25.939 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-31T05:01:25.940 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2024-01-31T05:01:25.940 INFO:teuthology.orchestra.run.smithi156.stdout:LOGO="fedora-logo-icon" 2024-01-31T05:01:25.940 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-31T05:01:25.940 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://centos.org/" 2024-01-31T05:01:25.940 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-31T05:01:25.940 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-31T05:01:25.940 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-31T05:01:25.940 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2024-01-31T05:01:25.963 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2024-01-31T05:01:25.981 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2024-01-31T05:01:25.981 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2024-01-31T05:01:26.038 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2024-01-31T05:01:26.038 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="9" 2024-01-31T05:01:26.038 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2024-01-31T05:01:26.038 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2024-01-31T05:01:26.038 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="9" 2024-01-31T05:01:26.039 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el9" 2024-01-31T05:01:26.039 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-31T05:01:26.039 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2024-01-31T05:01:26.039 INFO:teuthology.orchestra.run.smithi186.stdout:LOGO="fedora-logo-icon" 2024-01-31T05:01:26.039 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-31T05:01:26.039 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2024-01-31T05:01:26.039 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-31T05:01:26.039 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-31T05:01:26.039 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-31T05:01:26.040 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2024-01-31T05:01:26.071 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-31T05:01:26.082 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-31T05:01:26.100 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-31T05:01:26.100 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-01-31T05:01:26.103 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2024-01-31T05:01:26.105 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2024-01-31T05:01:26.121 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-31T05:01:26.127 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-31T05:01:26.127 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-01-31T05:01:26.160 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2024-01-31T05:01:26.162 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2024-01-31T05:01:26.247 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-31T05:01:26.319 INFO:teuthology.run_tasks:Running task kernel... 2024-01-31T05:01:26.333 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd42140e628558a450ba1d39a9d0835a2fa62cda'} 2024-01-31T05:01:26.333 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-01-31T05:01:26.333 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-01-31T05:01:26.333 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-31T05:01:26.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:01:26.351 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-01-31T05:01:26.409 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-361.el9.x86_64 2024-01-31T05:01:26.409 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-361.el9.x86_64 2024-01-31T05:01:26.409 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2024-01-31T05:01:28.791 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - AppStream 11 MB/s | 7.8 MB 00:00 2024-01-31T05:01:30.525 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.5 MB 00:00 2024-01-31T05:01:41.914 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 750 kB/s | 8.0 MB 00:10 2024-01-31T05:01:53.547 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 1.8 MB/s | 19 MB 00:10 2024-01-31T05:01:58.650 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 5.9 MB 00:01 2024-01-31T05:02:00.699 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 15 kB 00:00 2024-01-31T05:02:01.832 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 21 MB/s | 20 MB 00:00 2024-01-31T05:02:06.908 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-01-31T05:02:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-01-31T05:02:09.199 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-01-31T05:02:09.199 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-01-31T05:02:09.199 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240105104119:20125149.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.199 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-01-31T05:02:09.199 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.199 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-01-31T05:02:09.199 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.199 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-01-31T05:02:09.200 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.200 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-01-31T05:02:09.200 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.200 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-01-31T05:02:09.200 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.200 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-01-31T05:02:09.200 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.200 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-01-31T05:02:09.200 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.200 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-01-31T05:02:09.200 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.200 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240105093033:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.201 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820231201124232:182f7c73.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.202 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-01-31T05:02:09.203 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240117192000:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.204 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-01-31T05:02:09.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.206 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-01-31T05:02:09.581 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-01-31T05:02:09.581 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-01-31T05:02:09.686 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-01-31T05:02:09.704 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-01-31T05:02:09.704 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-412.el9 baseos 6.0 M 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 5.14.0-412.el9 baseos 21 M 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 5.14.0-412.el9 baseos 39 M 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core x86_64 5.14.0-412.el9 baseos 33 M 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:02:09.706 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 99 M 2024-01-31T05:02:09.706 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 122 M 2024-01-31T05:02:09.706 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-01-31T05:02:10.834 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): kernel-5.14.0-412.el9.x86_64.rpm 5.7 MB/s | 6.0 MB 00:01 2024-01-31T05:02:11.509 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): kernel-core-5.14.0-412.el9.x86_64.rpm 12 MB/s | 21 MB 00:01 2024-01-31T05:02:12.401 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): kernel-modules-5.14.0-412.el9.x86_64.rpm 15 MB/s | 39 MB 00:02 2024-01-31T05:02:12.717 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): kernel-modules-core-5.14.0-412.el9.x86_6 18 MB/s | 33 MB 00:01 2024-01-31T05:02:12.720 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-01-31T05:02:12.721 INFO:teuthology.orchestra.run.smithi045.stdout:Total 33 MB/s | 99 MB 00:03 2024-01-31T05:02:13.640 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-01-31T05:02:14.111 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-01-31T05:02:14.111 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-01-31T05:02:15.491 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-01-31T05:02:15.492 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-01-31T05:02:18.339 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-01-31T05:02:19.046 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-core-5.14.0-412.el9.x86_64 1/4 2024-01-31T05:02:19.186 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-5.14.0-412.el9.x86_64 2/4 2024-01-31T05:02:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-412.el9.x86_64 2/4 2024-01-31T05:02:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-5.14.0-412.el9.x86_64 3/4 2024-01-31T05:02:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-412.el9.x86_64 3/4 2024-01-31T05:02:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-5.14.0-412.el9.x86_64 4/4 2024-01-31T05:02:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-core-5.14.0-412.el9.x86_64 4/4 2024-01-31T05:03:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-412.el9.x86_64 4/4 2024-01-31T05:03:03.419 INFO:teuthology.orchestra.run.smithi045.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-31T05:03:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:03:03.507 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-412.el9.x86_64 4/4 2024-01-31T05:03:03.909 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-412.el9.x86_64 4/4 2024-01-31T05:03:03.909 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-412.el9.x86_64 1/4 2024-01-31T05:03:03.909 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-5.14.0-412.el9.x86_64 2/4 2024-01-31T05:03:03.909 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-5.14.0-412.el9.x86_64 3/4 2024-01-31T05:03:04.357 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-core-5.14.0-412.el9.x86_64 4/4 2024-01-31T05:03:04.357 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:03:04.357 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-01-31T05:03:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-412.el9.x86_64 2024-01-31T05:03:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core-5.14.0-412.el9.x86_64 2024-01-31T05:03:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-5.14.0-412.el9.x86_64 2024-01-31T05:03:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core-5.14.0-412.el9.x86_64 2024-01-31T05:03:04.358 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:03:04.358 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-01-31T05:03:05.012 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-01-31T05:03:05.164 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-412.el9.x86_64 2024-01-31T05:03:05.164 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-412.el9.x86_64 2024-01-31T05:03:05.165 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-412.el9.x86_64 2024-01-31T05:03:05.165 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-01-31T05:03:05.165 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-01-31T05:03:05.165 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-01-31T05:03:05.165 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2024-01-31T05:03:05.165 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-01-31T05:03:05.291 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-412.el9.x86_64 2024-01-31T05:03:05.292 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-412.el9.x86_64 2024-01-31T05:03:05.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-01-31T05:03:05.341 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 68.el9 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8264254 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-01-31T05:03:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-31T05:03:05.343 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-01-31T05:03:05.343 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-01-31T05:03:05.343 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:03:05.343 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-31T05:03:05.343 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-31T05:03:05.343 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-31T05:03:05.343 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-01-31T05:03:05.343 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:03:05.343 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-01-31T05:03:05.344 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-31T05:03:05.344 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 5.14.0-412.el9.x86_64 2024-01-31T05:03:05.344 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-31T05:03:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-01-31T05:03:07.589 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-31T05:03:07.606 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-01-31T05:03:07.608 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-01-31T05:03:07.623 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.0lOYtNBoIv 2024-01-31T05:03:07.623 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0lOYtNBoIv 2024-01-31T05:03:07.696 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.0lOYtNBoIv 2024-01-31T05:03:07.843 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.0lOYtNBoIv is 6KB 2024-01-31T05:03:07.892 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.0lOYtNBoIv 2024-01-31T05:03:07.907 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-01-31T05:03:07.975 INFO:teuthology.orchestra.run.smithi045.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-31T05:03:07.975 INFO:teuthology.orchestra.run.smithi045.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-31T05:03:07.975 INFO:teuthology.orchestra.run.smithi045.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-31T05:03:07.975 INFO:teuthology.orchestra.run.smithi045.stdout:15949e3c88704300882482eddfd7baae-5.14.0-412.el9.x86_64.conf 2024-01-31T05:03:07.976 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-412.el9.x86_64 2024-01-31T05:03:08.130 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-01-31T05:03:38.134 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-01-31T05:03:38.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-01-31T05:03:38.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T05:04:38.140 DEBUG:teuthology.orchestra.remote:timed out 2024-01-31T05:04:47.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-01-31T05:04:47.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T05:04:47.791 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-01-31T05:04:48.195 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-01-31T05:04:48.195 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-412.el9.x86_64"... 2024-01-31T05:04:48.196 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-01-31T05:04:48.209 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-412.el9.x86_64 2024-01-31T05:04:48.210 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-412.el9.x86_64 vs 5.14.0-412.el9.x86_64 2024-01-31T05:04:48.210 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-31T05:04:48.210 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-31T05:04:49.211 DEBUG:teuthology.parallel:result is None 2024-01-31T05:04:49.211 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-31T05:04:49.221 INFO:teuthology.task.internal:Creating test directory... 2024-01-31T05:04:49.221 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-31T05:04:49.224 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-31T05:04:49.226 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-31T05:04:49.244 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-31T05:04:49.264 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-31T05:04:49.297 INFO:teuthology.task.internal:Creating archive directory... 2024-01-31T05:04:49.297 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-31T05:04:49.299 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-31T05:04:49.302 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-31T05:04:49.385 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-31T05:04:49.391 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-31T05:04:49.392 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-31T05:04:49.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:49.411 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-31T05:04:49.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:49.428 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-31T05:04:49.430 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-31T05:04:49.453 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-31T05:04:49.497 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-31T05:04:49.506 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-31T05:04:49.516 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-31T05:04:49.532 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-31T05:04:49.533 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-31T05:04:49.600 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-31T05:04:49.601 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-31T05:04:49.608 INFO:teuthology.task.internal:Configuring sudo... 2024-01-31T05:04:49.608 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-31T05:04:49.611 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-31T05:04:49.613 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-31T05:04:49.642 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-31T05:04:49.651 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-31T05:04:49.652 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-31T05:04:49.676 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-31T05:04:49.683 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-31T05:04:49.698 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-31T05:04:49.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-31T05:04:49.811 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-31T05:04:49.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-31T05:04:49.928 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:04:49.928 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-31T05:04:49.990 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-31T05:04:50.031 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-31T05:04:50.098 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-31T05:04:50.136 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-31T05:04:50.186 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-01-31T05:04:50.186 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-31T05:04:50.255 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-31T05:04:50.299 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-31T05:04:50.365 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-31T05:04:50.402 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-31T05:04:50.453 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-01-31T05:04:50.453 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-31T05:04:50.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-01-31T05:04:50.524 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2024-01-31T05:04:50.527 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2024-01-31T05:04:50.567 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-31T05:04:50.600 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-31T05:04:50.717 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-31T05:04:51.041 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-31T05:04:51.049 INFO:teuthology.task.internal:Starting timer... 2024-01-31T05:04:51.050 INFO:teuthology.run_tasks:Running task pcp... 2024-01-31T05:04:51.093 INFO:teuthology.run_tasks:Running task selinux... 2024-01-31T05:04:51.124 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-01-31T05:04:51.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-01-31T05:04:51.215 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-01-31T05:04:51.216 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2024-01-31T05:04:51.259 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2024-01-31T05:04:51.261 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2024-01-31T05:04:51.307 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2024-01-31T05:04:51.309 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-31T05:04:51.309 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-01-31T05:04:51.327 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-01-31T05:04:51.327 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2024-01-31T05:04:51.352 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2024-01-31T05:04:51.352 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2024-01-31T05:04:51.376 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2024-01-31T05:04:51.376 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi156.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2024-01-31T05:04:51.376 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-31T05:04:51.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:51.430 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-31T05:04:51.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:51.459 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-31T05:04:51.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:51.488 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-31T05:04:51.488 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-01-31T05:04:51.570 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2024-01-31T05:04:51.605 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2024-01-31T05:04:51.637 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-31T05:04:51.657 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-31T05:04:51.676 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-31T05:04:51.677 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi156.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-31T05:09:40.891 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2024-01-31T05:09:40.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-01-31T05:09:40.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T05:09:40.991 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-01-31T05:09:41.076 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-01-31T05:09:41.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-01-31T05:09:41.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T05:09:41.167 DEBUG:teuthology.orchestra.run.smithi156:> true 2024-01-31T05:09:41.239 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-01-31T05:09:41.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-01-31T05:09:41.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T05:09:41.381 DEBUG:teuthology.orchestra.run.smithi186:> true 2024-01-31T05:09:41.451 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-01-31T05:09:41.451 INFO:teuthology.run_tasks:Running task clock... 2024-01-31T05:09:41.462 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-31T05:09:41.462 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-31T05:09:41.463 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-31T05:09:41.465 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-31T05:09:41.465 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-31T05:09:41.468 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-31T05:09:41.468 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-31T05:09:41.496 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-31T05:09:41.496 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-31T05:09:41.513 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-31T05:09:41.513 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-31T05:09:41.523 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-31T05:09:41.541 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-31T05:09:41.542 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2024-01-31T05:09:41.543 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-01-31T05:09:41.555 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2024-01-31T05:09:41.557 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-01-31T05:09:41.570 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2024-01-31T05:09:41.572 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-31T05:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-31T05:09:41.584 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2024-01-31T05:09:41.589 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-31T05:09:41.591 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-31T05:09:41.601 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-31T05:09:41.617 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-31T05:09:41.633 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2024-01-31T05:09:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-31T05:09:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2024-01-31T05:09:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:09:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:09:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:09:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:09:41.641 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-01-31T05:09:41.645 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-31T05:09:41.645 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-01-31T05:09:41.645 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:09:41.645 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:09:41.645 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:09:41.645 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:09:41.661 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-01-31T05:09:41.665 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-31T05:09:41.665 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-01-31T05:09:41.665 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:09:41.665 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:09:41.665 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:09:41.665 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:09:41.665 INFO:teuthology.run_tasks:Running task install... 2024-01-31T05:09:41.674 DEBUG:teuthology.task.install:project ceph 2024-01-31T05:09:41.674 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '661946e7044723130c03627323f6a879a9848cb5'}} 2024-01-31T05:09:41.674 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '661946e7044723130c03627323f6a879a9848cb5'} 2024-01-31T05:09:41.674 INFO:teuthology.task.install:Using flavor: default 2024-01-31T05:09:41.679 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-31T05:09:41.680 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-01-31T05:09:41.680 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'tag': None, 'wait_for_package': False} 2024-01-31T05:09:41.681 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:09:41.682 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'tag': None, 'wait_for_package': False} 2024-01-31T05:09:41.683 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:09:41.683 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'tag': None, 'wait_for_package': False} 2024-01-31T05:09:41.684 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:09:41.836 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/ 2024-01-31T05:09:41.836 INFO:teuthology.task.install.rpm:Package version is 19.0.0-885.g661946e7 2024-01-31T05:09:41.839 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/ 2024-01-31T05:09:41.839 INFO:teuthology.task.install.rpm:Package version is 19.0.0-885.g661946e7 2024-01-31T05:09:41.907 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/ 2024-01-31T05:09:41.908 INFO:teuthology.task.install.rpm:Package version is 19.0.0-885.g661946e7 2024-01-31T05:09:41.959 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-31T05:09:41.959 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-01-31T05:09:41.959 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-31T05:09:41.964 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-31T05:09:41.964 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-01-31T05:09:41.964 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-31T05:09:41.987 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-31T05:09:41.987 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/661946e7044723130c03627323f6a879a9848cb5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-31T05:09:41.992 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-31T05:09:41.992 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/661946e7044723130c03627323f6a879a9848cb5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-31T05:09:42.055 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-31T05:09:42.058 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-31T05:09:42.106 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-31T05:09:42.106 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:09:42.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-31T05:09:42.134 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-31T05:09:42.134 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/661946e7044723130c03627323f6a879a9848cb5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-31T05:09:42.135 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-31T05:09:42.138 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-31T05:09:42.201 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2024-01-31T05:09:42.202 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-31T05:09:42.203 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2024-01-31T05:09:42.204 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2024-01-31T05:09:42.205 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2024-01-31T05:09:42.282 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-31T05:09:42.348 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-01-31T05:09:42.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-01-31T05:09:42.438 INFO:teuthology.orchestra.run.smithi186.stdout:74 files removed 2024-01-31T05:09:42.440 INFO:teuthology.orchestra.run.smithi156.stdout:74 files removed 2024-01-31T05:09:42.465 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-31T05:09:42.467 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-31T05:09:42.607 INFO:teuthology.orchestra.run.smithi045.stdout:74 files removed 2024-01-31T05:09:42.635 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-31T05:09:43.238 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 166 kB/s | 80 kB 00:00 2024-01-31T05:09:43.239 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 166 kB/s | 80 kB 00:00 2024-01-31T05:09:43.385 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 187 kB/s | 80 kB 00:00 2024-01-31T05:09:43.598 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 30 kB/s | 9.5 kB 00:00 2024-01-31T05:09:43.697 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 23 kB/s | 9.5 kB 00:00 2024-01-31T05:09:43.852 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 23 kB/s | 9.5 kB 00:00 2024-01-31T05:09:44.049 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-01-31T05:09:44.106 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-01-31T05:09:44.278 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-01-31T05:09:44.493 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - AppStream 19 MB/s | 7.8 MB 00:00 2024-01-31T05:09:44.566 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS-9 - AppStream 18 MB/s | 7.8 MB 00:00 2024-01-31T05:09:44.797 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - AppStream 16 MB/s | 7.8 MB 00:00 2024-01-31T05:09:46.093 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.5 MB 00:00 2024-01-31T05:09:46.150 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS-9 - BaseOS 8.6 MB/s | 2.5 MB 00:00 2024-01-31T05:09:46.423 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - BaseOS 8.6 MB/s | 2.5 MB 00:00 2024-01-31T05:09:47.903 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.0 MB 00:01 2024-01-31T05:09:48.449 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-01-31T05:09:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.0 MB 00:02 2024-01-31T05:09:49.802 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-01-31T05:09:51.700 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - AppStream 8.8 MB/s | 19 MB 00:02 2024-01-31T05:09:57.134 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 5.9 MB 00:01 2024-01-31T05:09:58.719 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 15 kB 00:00 2024-01-31T05:10:00.343 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - CRB 900 kB/s | 5.9 MB 00:06 2024-01-31T05:10:00.645 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-01-31T05:10:01.894 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 15 kB 00:00 2024-01-31T05:10:03.906 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-01-31T05:10:05.537 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-01-31T05:10:05.879 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-01-31T05:10:08.174 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-01-31T05:10:08.174 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-01-31T05:10:08.174 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-01-31T05:10:08.174 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240105104119:20125149.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.174 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-01-31T05:10:08.175 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240105093033:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.176 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-01-31T05:10:08.177 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.177 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-01-31T05:10:08.177 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.177 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-01-31T05:10:08.177 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.177 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-01-31T05:10:08.177 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.177 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-01-31T05:10:08.177 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.177 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-01-31T05:10:08.177 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.177 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820231201124232:182f7c73.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-01-31T05:10:08.178 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.179 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-01-31T05:10:08.179 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.179 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-01-31T05:10:08.179 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.179 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-01-31T05:10:08.179 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.179 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-01-31T05:10:08.179 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.179 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-01-31T05:10:08.179 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.179 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-01-31T05:10:08.180 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.180 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-01-31T05:10:08.180 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.180 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-01-31T05:10:08.180 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.180 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-01-31T05:10:08.180 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240117192000:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.180 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-01-31T05:10:08.180 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.180 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.180 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.180 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-01-31T05:10:08.181 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.182 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.182 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.182 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.182 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-01-31T05:10:08.182 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.182 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.182 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.182 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.182 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-01-31T05:10:08.182 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.182 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.182 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-01-31T05:10:08.183 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.184 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.184 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.184 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-01-31T05:10:08.600 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-31T05:10:08.600 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-31T05:10:08.613 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-31T05:10:08.614 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-31T05:10:08.615 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-31T05:10:08.616 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-31T05:10:08.618 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-31T05:10:08.621 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-31T05:10:08.622 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-31T05:10:08.708 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-01-31T05:10:08.725 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================== 2024-01-31T05:10:08.725 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-01-31T05:10:08.725 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================== 2024-01-31T05:10:08.725 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-01-31T05:10:08.725 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-01-31T05:10:08.725 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:19.0.0-885.g661946e7.el9 ceph 6.5 k 2024-01-31T05:10:08.725 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:19.0.0-885.g661946e7.el9 ceph 5.1 M 2024-01-31T05:10:08.725 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 886 k 2024-01-31T05:10:08.725 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-885.g661946e7.el9 ceph 145 k 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:19.0.0-885.g661946e7.el9 ceph 1.5 M 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 133 k 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 1.7 M 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 7.4 M 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 51 k 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 8.7 M 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:19.0.0-885.g661946e7.el9 ceph 46 M 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 262 k 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 742 k 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 217 k 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell x86_64 2:19.0.0-885.g661946e7.el9 ceph 27 k 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 18 k 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-01-31T05:10:08.726 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 31 k 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 743 k 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 127 k 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:19.0.0-885.g661946e7.el9 ceph 162 k 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:19.0.0-885.g661946e7.el9 ceph 321 k 2024-01-31T05:10:08.727 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 298 k 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 99 k 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 85 k 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 170 k 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager x86_64 1:1.45.10-1.el9 baseos 2.3 M 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 baseos 1.8 M 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 baseos 43 k 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 baseos 247 k 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-01-31T05:10:08.728 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.4 M 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-01-31T05:10:08.729 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 19 M 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 28 k 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:19.0.0-885.g661946e7.el9 ceph 2.1 M 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 245 k 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.7 M 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:19.0.0-885.g661946e7.el9 ceph 17 M 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 15 k 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:19.0.0-885.g661946e7.el9 ceph 25 k 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-31T05:10:08.730 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:19.0.0-885.g661946e7.el9 ceph 163 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 499 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.8 M 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-01-31T05:10:08.731 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:19.0.0-885.g661946e7.el9 ceph 45 k 2024-01-31T05:10:08.732 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 130 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-31T05:10:08.733 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.11.3-4.el9 appstream 249 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-01-31T05:10:08.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-31T05:10:08.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-01-31T05:10:08.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-31T05:10:08.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-01-31T05:10:08.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-01-31T05:10:08.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-01-31T05:10:08.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-31T05:10:08.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-01-31T05:10:08.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-31T05:10:08.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-01-31T05:10:08.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-01-31T05:10:08.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-31T05:10:08.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-31T05:10:08.736 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-01-31T05:10:08.737 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-01-31T05:10:08.737 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-01-31T05:10:08.737 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:10:08.737 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-01-31T05:10:08.737 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================== 2024-01-31T05:10:08.737 INFO:teuthology.orchestra.run.smithi186.stdout:Install 139 Packages 2024-01-31T05:10:08.737 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 16 Packages 2024-01-31T05:10:08.737 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:10:08.738 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 212 M 2024-01-31T05:10:08.738 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-01-31T05:10:09.070 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 7.9 kB/s | 1.7 kB 00:00 2024-01-31T05:10:09.146 INFO:teuthology.orchestra.run.smithi186.stdout:(1/155): ceph-19.0.0-885.g661946e7.el9.x86_64.r 32 kB/s | 6.5 kB 00:00 2024-01-31T05:10:09.438 INFO:teuthology.orchestra.run.smithi186.stdout:(2/155): ceph-fuse-19.0.0-885.g661946e7.el9.x86 3.0 MB/s | 886 kB 00:00 2024-01-31T05:10:09.513 INFO:teuthology.orchestra.run.smithi186.stdout:(3/155): ceph-immutable-object-cache-19.0.0-885 1.9 MB/s | 145 kB 00:00 2024-01-31T05:10:09.972 INFO:teuthology.orchestra.run.smithi186.stdout:(4/155): ceph-mds-19.0.0-885.g661946e7.el9.x86_ 4.6 MB/s | 2.1 MB 00:00 2024-01-31T05:10:10.131 INFO:teuthology.orchestra.run.smithi186.stdout:(5/155): ceph-base-19.0.0-885.g661946e7.el9.x86 4.3 MB/s | 5.1 MB 00:01 2024-01-31T05:10:10.339 INFO:teuthology.orchestra.run.smithi186.stdout:(6/155): ceph-mgr-19.0.0-885.g661946e7.el9.x86_ 4.1 MB/s | 1.5 MB 00:00 2024-01-31T05:10:10.523 INFO:teuthology.orchestra.run.smithi186.stdout:(7/155): ceph-common-19.0.0-885.g661946e7.el9.x 12 MB/s | 19 MB 00:01 2024-01-31T05:10:10.798 INFO:teuthology.orchestra.run.smithi186.stdout:(8/155): ceph-mon-19.0.0-885.g661946e7.el9.x86_ 7.0 MB/s | 4.7 MB 00:00 2024-01-31T05:10:10.882 INFO:teuthology.orchestra.run.smithi186.stdout:(9/155): ceph-selinux-19.0.0-885.g661946e7.el9. 302 kB/s | 25 kB 00:00 2024-01-31T05:10:11.071 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 5.9 MB 00:01 2024-01-31T05:10:11.215 INFO:teuthology.orchestra.run.smithi186.stdout:(10/155): ceph-radosgw-19.0.0-885.g661946e7.el9 13 MB/s | 8.7 MB 00:00 2024-01-31T05:10:11.282 INFO:teuthology.orchestra.run.smithi186.stdout:(11/155): cephfs-mirror-19.0.0-885.g661946e7.el 3.2 MB/s | 217 kB 00:00 2024-01-31T05:10:11.349 INFO:teuthology.orchestra.run.smithi186.stdout:(12/155): cephfs-shell-19.0.0-885.g661946e7.el9 406 kB/s | 27 kB 00:00 2024-01-31T05:10:11.416 INFO:teuthology.orchestra.run.smithi186.stdout:(13/155): libcephfs-devel-19.0.0-885.g661946e7. 467 kB/s | 31 kB 00:00 2024-01-31T05:10:11.524 INFO:teuthology.orchestra.run.smithi186.stdout:(14/155): libcephfs2-19.0.0-885.g661946e7.el9.x 6.7 MB/s | 743 kB 00:00 2024-01-31T05:10:11.599 INFO:teuthology.orchestra.run.smithi156.stderr:Modular dependency problems: 2024-01-31T05:10:11.599 INFO:teuthology.orchestra.run.smithi156.stderr: 2024-01-31T05:10:11.599 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 1: conflicting requests 2024-01-31T05:10:11.599 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240105104119:20125149.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 2: conflicting requests 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 3: conflicting requests 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 4: conflicting requests 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 5: conflicting requests 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 6: conflicting requests 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 7: conflicting requests 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.600 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 8: conflicting requests 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 9: conflicting requests 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 10: conflicting requests 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240105093033:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 11: conflicting requests 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 12: conflicting requests 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 13: conflicting requests 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 14: conflicting requests 2024-01-31T05:10:11.601 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 15: conflicting requests 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 16: conflicting requests 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 17: conflicting requests 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 18: conflicting requests 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 19: conflicting requests 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 20: conflicting requests 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.602 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 21: conflicting requests 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 22: conflicting requests 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 23: conflicting requests 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820231201124232:182f7c73.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 24: conflicting requests 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 25: conflicting requests 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 26: conflicting requests 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 27: conflicting requests 2024-01-31T05:10:11.603 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.604 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 28: conflicting requests 2024-01-31T05:10:11.604 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.604 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 29: conflicting requests 2024-01-31T05:10:11.604 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.604 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 30: conflicting requests 2024-01-31T05:10:11.604 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.604 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 31: conflicting requests 2024-01-31T05:10:11.604 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.604 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 32: conflicting requests 2024-01-31T05:10:11.604 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.604 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 33: conflicting requests 2024-01-31T05:10:11.604 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 34: conflicting requests 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 35: conflicting requests 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 36: conflicting requests 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240117192000:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 37: conflicting requests 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 38: cannot install the best candidate for the job 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.605 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 39: conflicting requests 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 40: cannot install the best candidate for the job 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.606 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 41: cannot install the best candidate for the job 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 42: cannot install the best candidate for the job 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 43: cannot install the best candidate for the job 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.607 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.608 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.608 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 44: cannot install the best candidate for the job 2024-01-31T05:10:11.608 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.608 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.608 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.608 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.608 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 45: cannot install the best candidate for the job 2024-01-31T05:10:11.608 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.608 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.608 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.608 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-01-31T05:10:11.609 INFO:teuthology.orchestra.run.smithi186.stdout:(15/155): libcephsqlite-19.0.0-885.g661946e7.el 1.9 MB/s | 163 kB 00:00 2024-01-31T05:10:11.700 INFO:teuthology.orchestra.run.smithi186.stdout:(16/155): librados-devel-19.0.0-885.g661946e7.e 1.4 MB/s | 127 kB 00:00 2024-01-31T05:10:11.809 INFO:teuthology.orchestra.run.smithi186.stdout:(17/155): libradosstriper1-19.0.0-885.g661946e7 4.5 MB/s | 499 kB 00:00 2024-01-31T05:10:12.018 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-31T05:10:12.019 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-31T05:10:12.031 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-31T05:10:12.032 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-31T05:10:12.033 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-31T05:10:12.034 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-31T05:10:12.036 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-31T05:10:12.038 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-31T05:10:12.039 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-31T05:10:12.125 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-01-31T05:10:12.149 INFO:teuthology.orchestra.run.smithi156.stdout:======================================================================================== 2024-01-31T05:10:12.149 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-01-31T05:10:12.149 INFO:teuthology.orchestra.run.smithi156.stdout:======================================================================================== 2024-01-31T05:10:12.149 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-01-31T05:10:12.149 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-01-31T05:10:12.149 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:19.0.0-885.g661946e7.el9 ceph 6.5 k 2024-01-31T05:10:12.149 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:19.0.0-885.g661946e7.el9 ceph 5.1 M 2024-01-31T05:10:12.149 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 886 k 2024-01-31T05:10:12.149 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-885.g661946e7.el9 ceph 145 k 2024-01-31T05:10:12.149 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:19.0.0-885.g661946e7.el9 ceph 1.5 M 2024-01-31T05:10:12.150 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 133 k 2024-01-31T05:10:12.150 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 1.7 M 2024-01-31T05:10:12.150 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 7.4 M 2024-01-31T05:10:12.150 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 51 k 2024-01-31T05:10:12.150 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 8.7 M 2024-01-31T05:10:12.150 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:19.0.0-885.g661946e7.el9 ceph 46 M 2024-01-31T05:10:12.150 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 262 k 2024-01-31T05:10:12.150 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 742 k 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 217 k 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-shell x86_64 2:19.0.0-885.g661946e7.el9 ceph 27 k 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 18 k 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 31 k 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 743 k 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 127 k 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-01-31T05:10:12.151 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-01-31T05:10:12.152 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-01-31T05:10:12.152 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-01-31T05:10:12.152 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:19.0.0-885.g661946e7.el9 ceph 162 k 2024-01-31T05:10:12.152 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:19.0.0-885.g661946e7.el9 ceph 321 k 2024-01-31T05:10:12.152 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 298 k 2024-01-31T05:10:12.152 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 99 k 2024-01-31T05:10:12.152 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 85 k 2024-01-31T05:10:12.152 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-31T05:10:12.152 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 170 k 2024-01-31T05:10:12.152 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-01-31T05:10:12.153 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2024-01-31T05:10:12.153 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager x86_64 1:1.45.10-1.el9 baseos 2.3 M 2024-01-31T05:10:12.153 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 baseos 1.8 M 2024-01-31T05:10:12.153 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 baseos 43 k 2024-01-31T05:10:12.153 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 baseos 247 k 2024-01-31T05:10:12.153 INFO:teuthology.orchestra.run.smithi156.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-01-31T05:10:12.153 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-01-31T05:10:12.153 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-01-31T05:10:12.153 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.4 M 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 19 M 2024-01-31T05:10:12.154 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 28 k 2024-01-31T05:10:12.155 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:19.0.0-885.g661946e7.el9 ceph 2.1 M 2024-01-31T05:10:12.155 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 245 k 2024-01-31T05:10:12.155 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.7 M 2024-01-31T05:10:12.155 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:19.0.0-885.g661946e7.el9 ceph 17 M 2024-01-31T05:10:12.155 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 15 k 2024-01-31T05:10:12.155 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:19.0.0-885.g661946e7.el9 ceph 25 k 2024-01-31T05:10:12.155 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-01-31T05:10:12.155 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-01-31T05:10:12.155 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-01-31T05:10:12.156 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-01-31T05:10:12.156 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-31T05:10:12.156 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-01-31T05:10:12.156 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-31T05:10:12.156 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-31T05:10:12.156 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:19.0.0-885.g661946e7.el9 ceph 163 k 2024-01-31T05:10:12.156 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-01-31T05:10:12.156 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 499 k 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.8 M 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-31T05:10:12.157 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-01-31T05:10:12.158 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-01-31T05:10:12.158 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-01-31T05:10:12.158 INFO:teuthology.orchestra.run.smithi156.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-31T05:10:12.158 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-01-31T05:10:12.158 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-31T05:10:12.158 INFO:teuthology.orchestra.run.smithi156.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-01-31T05:10:12.158 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-01-31T05:10:12.158 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-31T05:10:12.158 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-31T05:10:12.158 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:19.0.0-885.g661946e7.el9 ceph 45 k 2024-01-31T05:10:12.158 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 130 k 2024-01-31T05:10:12.158 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-31T05:10:12.159 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-31T05:10:12.160 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.11.3-4.el9 appstream 249 k 2024-01-31T05:10:12.160 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-01-31T05:10:12.160 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-01-31T05:10:12.160 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-31T05:10:12.160 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-31T05:10:12.160 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-01-31T05:10:12.160 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-01-31T05:10:12.160 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-31T05:10:12.160 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-31T05:10:12.160 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-01-31T05:10:12.160 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-01-31T05:10:12.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-01-31T05:10:12.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-31T05:10:12.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-01-31T05:10:12.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-31T05:10:12.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-01-31T05:10:12.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-31T05:10:12.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-01-31T05:10:12.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-01-31T05:10:12.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-01-31T05:10:12.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-31T05:10:12.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-31T05:10:12.162 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-31T05:10:12.163 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-31T05:10:12.163 INFO:teuthology.orchestra.run.smithi156.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-31T05:10:12.163 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-01-31T05:10:12.163 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-31T05:10:12.163 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-01-31T05:10:12.163 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-01-31T05:10:12.163 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-01-31T05:10:12.163 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:10:12.163 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-01-31T05:10:12.163 INFO:teuthology.orchestra.run.smithi156.stdout:======================================================================================== 2024-01-31T05:10:12.163 INFO:teuthology.orchestra.run.smithi156.stdout:Install 139 Packages 2024-01-31T05:10:12.163 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 16 Packages 2024-01-31T05:10:12.164 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:10:12.164 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 212 M 2024-01-31T05:10:12.164 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-01-31T05:10:12.184 INFO:teuthology.orchestra.run.smithi186.stdout:(18/155): ceph-osd-19.0.0-885.g661946e7.el9.x86 9.1 MB/s | 17 MB 00:01 2024-01-31T05:10:12.276 INFO:teuthology.orchestra.run.smithi186.stdout:(19/155): python3-ceph-argparse-19.0.0-885.g661 493 kB/s | 45 kB 00:00 2024-01-31T05:10:12.343 INFO:teuthology.orchestra.run.smithi186.stdout:(20/155): librgw2-19.0.0-885.g661946e7.el9.x86_ 9.0 MB/s | 4.8 MB 00:00 2024-01-31T05:10:12.368 INFO:teuthology.orchestra.run.smithi186.stdout:(21/155): python3-ceph-common-19.0.0-885.g66194 1.4 MB/s | 130 kB 00:00 2024-01-31T05:10:12.402 INFO:teuthology.orchestra.run.smithi186.stdout:(22/155): python3-cephfs-19.0.0-885.g661946e7.e 2.7 MB/s | 162 kB 00:00 2024-01-31T05:10:12.444 INFO:teuthology.orchestra.run.smithi186.stdout:(23/155): python3-rados-19.0.0-885.g661946e7.el 4.2 MB/s | 321 kB 00:00 2024-01-31T05:10:12.469 INFO:teuthology.orchestra.run.smithi186.stdout:(24/155): python3-rbd-19.0.0-885.g661946e7.el9. 4.4 MB/s | 298 kB 00:00 2024-01-31T05:10:12.511 INFO:teuthology.orchestra.run.smithi186.stdout:(25/155): rbd-fuse-19.0.0-885.g661946e7.el9.x86 2.0 MB/s | 85 kB 00:00 2024-01-31T05:10:12.536 INFO:teuthology.orchestra.run.smithi186.stdout:(26/155): python3-rgw-19.0.0-885.g661946e7.el9. 1.1 MB/s | 99 kB 00:00 2024-01-31T05:10:12.637 INFO:teuthology.orchestra.run.smithi186.stdout:(27/155): rbd-nbd-19.0.0-885.g661946e7.el9.x86_ 1.7 MB/s | 170 kB 00:00 2024-01-31T05:10:12.697 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 15 kB 00:00 2024-01-31T05:10:12.720 INFO:teuthology.orchestra.run.smithi186.stdout:(28/155): ceph-grafana-dashboards-19.0.0-885.g6 340 kB/s | 28 kB 00:00 2024-01-31T05:10:12.796 INFO:teuthology.orchestra.run.smithi186.stdout:(29/155): rbd-mirror-19.0.0-885.g661946e7.el9.x 11 MB/s | 3.1 MB 00:00 2024-01-31T05:10:12.812 INFO:teuthology.orchestra.run.smithi186.stdout:(30/155): ceph-mgr-cephadm-19.0.0-885.g661946e7 1.4 MB/s | 133 kB 00:00 2024-01-31T05:10:13.046 INFO:teuthology.orchestra.run.smithi186.stdout:(31/155): ceph-mgr-dashboard-19.0.0-885.g661946 6.9 MB/s | 1.7 MB 00:00 2024-01-31T05:10:13.122 INFO:teuthology.orchestra.run.smithi186.stdout:(32/155): ceph-mgr-modules-core-19.0.0-885.g661 3.2 MB/s | 245 kB 00:00 2024-01-31T05:10:13.205 INFO:teuthology.orchestra.run.smithi186.stdout:(33/155): ceph-mgr-rook-19.0.0-885.g661946e7.el 608 kB/s | 51 kB 00:00 2024-01-31T05:10:13.264 INFO:teuthology.orchestra.run.smithi186.stdout:(34/155): ceph-prometheus-alerts-19.0.0-885.g66 259 kB/s | 15 kB 00:00 2024-01-31T05:10:13.373 INFO:teuthology.orchestra.run.smithi186.stdout:(35/155): ceph-volume-19.0.0-885.g661946e7.el9. 2.4 MB/s | 262 kB 00:00 2024-01-31T05:10:13.638 INFO:teuthology.orchestra.run.smithi156.stdout:(1/155): ceph-19.0.0-885.g661946e7.el9.x86_64.r 7.3 kB/s | 6.5 kB 00:00 2024-01-31T05:10:13.673 INFO:teuthology.orchestra.run.smithi186.stdout:(36/155): cephadm-19.0.0-885.g661946e7.el9.noar 2.4 MB/s | 742 kB 00:00 2024-01-31T05:10:13.732 INFO:teuthology.orchestra.run.smithi186.stdout:(37/155): cephfs-top-19.0.0-885.g661946e7.el9.n 313 kB/s | 18 kB 00:00 2024-01-31T05:10:13.865 INFO:teuthology.orchestra.run.smithi186.stdout:(38/155): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-01-31T05:10:13.907 INFO:teuthology.orchestra.run.smithi186.stdout:(39/155): libaio-devel-0.3.112-1.el8.x86_64.rpm 100 kB/s | 18 kB 00:00 2024-01-31T05:10:13.991 INFO:teuthology.orchestra.run.smithi186.stdout:(40/155): fuse-2.9.9-15.el9.x86_64.rpm 648 kB/s | 80 kB 00:00 2024-01-31T05:10:14.049 INFO:teuthology.orchestra.run.smithi186.stdout:(41/155): jq-1.6-16.el9.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2024-01-31T05:10:14.100 INFO:teuthology.orchestra.run.smithi186.stdout:(42/155): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-01-31T05:10:14.125 INFO:teuthology.orchestra.run.smithi186.stdout:(43/155): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-01-31T05:10:14.150 INFO:teuthology.orchestra.run.smithi186.stdout:(44/155): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 218 kB 00:00 2024-01-31T05:10:14.176 INFO:teuthology.orchestra.run.smithi186.stdout:(45/155): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-01-31T05:10:14.234 INFO:teuthology.orchestra.run.smithi186.stdout:(46/155): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-01-31T05:10:14.259 INFO:teuthology.orchestra.run.smithi186.stdout:(47/155): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-01-31T05:10:14.285 INFO:teuthology.orchestra.run.smithi186.stdout:(48/155): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-01-31T05:10:14.315 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 14 MB/s | 20 MB 00:01 2024-01-31T05:10:14.318 INFO:teuthology.orchestra.run.smithi186.stdout:(49/155): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-01-31T05:10:14.344 INFO:teuthology.orchestra.run.smithi186.stdout:(50/155): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-01-31T05:10:14.880 INFO:teuthology.orchestra.run.smithi156.stdout:(2/155): ceph-base-19.0.0-885.g661946e7.el9.x86 2.4 MB/s | 5.1 MB 00:02 2024-01-31T05:10:14.946 INFO:teuthology.orchestra.run.smithi156.stdout:(3/155): ceph-immutable-object-cache-19.0.0-885 2.1 MB/s | 145 kB 00:00 2024-01-31T05:10:15.047 INFO:teuthology.orchestra.run.smithi156.stdout:(4/155): ceph-fuse-19.0.0-885.g661946e7.el9.x86 629 kB/s | 886 kB 00:01 2024-01-31T05:10:15.119 INFO:teuthology.orchestra.run.smithi186.stdout:(51/155): ceph-test-19.0.0-885.g661946e7.el9.x8 11 MB/s | 46 MB 00:04 2024-01-31T05:10:15.152 INFO:teuthology.orchestra.run.smithi186.stdout:(52/155): automake-1.16.2-8.el9.noarch.rpm 826 kB/s | 689 kB 00:00 2024-01-31T05:10:15.186 INFO:teuthology.orchestra.run.smithi186.stdout:(53/155): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-01-31T05:10:15.245 INFO:teuthology.orchestra.run.smithi186.stdout:(54/155): elfutils-libelf-devel-0.190-2.el9.x86 234 kB/s | 21 kB 00:00 2024-01-31T05:10:15.295 INFO:teuthology.orchestra.run.smithi186.stdout:(55/155): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-01-31T05:10:15.337 INFO:teuthology.orchestra.run.smithi186.stdout:(56/155): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-01-31T05:10:15.430 INFO:teuthology.orchestra.run.smithi156.stdout:(5/155): ceph-mds-19.0.0-885.g661946e7.el9.x86_ 4.4 MB/s | 2.1 MB 00:00 2024-01-31T05:10:15.445 INFO:teuthology.orchestra.run.smithi186.stdout:(57/155): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-01-31T05:10:15.562 INFO:teuthology.orchestra.run.smithi186.stdout:(58/155): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-01-31T05:10:15.588 INFO:teuthology.orchestra.run.smithi186.stdout:(59/155): boost-program-options-1.75.0-8.el9.x8 228 kB/s | 107 kB 00:00 2024-01-31T05:10:15.663 INFO:teuthology.orchestra.run.smithi186.stdout:(60/155): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-01-31T05:10:15.756 INFO:teuthology.orchestra.run.smithi156.stdout:(6/155): ceph-mgr-19.0.0-885.g661946e7.el9.x86_ 2.1 MB/s | 1.5 MB 00:00 2024-01-31T05:10:15.772 INFO:teuthology.orchestra.run.smithi186.stdout:(61/155): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-01-31T05:10:15.872 INFO:teuthology.orchestra.run.smithi186.stdout:(62/155): libuuid-devel-2.37.4-15.el9.x86_64.rp 255 kB/s | 26 kB 00:00 2024-01-31T05:10:15.972 INFO:teuthology.orchestra.run.smithi186.stdout:(63/155): libzstd-devel-1.5.1-2.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-01-31T05:10:16.023 INFO:teuthology.orchestra.run.smithi186.stdout:(64/155): librdkafka-1.6.1-102.el9.x86_64.rpm 1.5 MB/s | 662 kB 00:00 2024-01-31T05:10:16.081 INFO:teuthology.orchestra.run.smithi186.stdout:(65/155): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-01-31T05:10:16.131 INFO:teuthology.orchestra.run.smithi186.stdout:(66/155): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-01-31T05:10:16.173 INFO:teuthology.orchestra.run.smithi186.stdout:(67/155): openblas-0.3.21-2.el9.x86_64.rpm 383 kB/s | 35 kB 00:00 2024-01-31T05:10:16.281 INFO:teuthology.orchestra.run.smithi156.stdout:(7/155): ceph-mon-19.0.0-885.g661946e7.el9.x86_ 5.5 MB/s | 4.7 MB 00:00 2024-01-31T05:10:16.424 INFO:teuthology.orchestra.run.smithi186.stdout:(68/155): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-01-31T05:10:16.907 INFO:teuthology.orchestra.run.smithi186.stdout:(69/155): openblas-openmp-0.3.21-2.el9.x86_64.r 6.1 MB/s | 4.7 MB 00:00 2024-01-31T05:10:17.523 INFO:teuthology.orchestra.run.smithi156.stdout:(8/155): ceph-osd-19.0.0-885.g661946e7.el9.x86_ 9.5 MB/s | 17 MB 00:01 2024-01-31T05:10:17.581 INFO:teuthology.orchestra.run.smithi156.stdout:(9/155): ceph-selinux-19.0.0-885.g661946e7.el9. 434 kB/s | 25 kB 00:00 2024-01-31T05:10:17.673 INFO:teuthology.orchestra.run.smithi156.stdout:(10/155): ceph-radosgw-19.0.0-885.g661946e7.el9 6.2 MB/s | 8.7 MB 00:01 2024-01-31T05:10:17.765 INFO:teuthology.orchestra.run.smithi156.stdout:(11/155): cephfs-mirror-19.0.0-885.g661946e7.el 2.3 MB/s | 217 kB 00:00 2024-01-31T05:10:17.824 INFO:teuthology.orchestra.run.smithi156.stdout:(12/155): cephfs-shell-19.0.0-885.g661946e7.el9 465 kB/s | 27 kB 00:00 2024-01-31T05:10:17.874 INFO:teuthology.orchestra.run.smithi156.stdout:(13/155): libcephfs-devel-19.0.0-885.g661946e7. 625 kB/s | 31 kB 00:00 2024-01-31T05:10:18.066 INFO:teuthology.orchestra.run.smithi186.stdout:(70/155): postgresql-contrib-13.11-1.el9.x86_64 782 kB/s | 905 kB 00:01 2024-01-31T05:10:18.108 INFO:teuthology.orchestra.run.smithi156.stdout:(14/155): libcephfs2-19.0.0-885.g661946e7.el9.x 3.1 MB/s | 743 kB 00:00 2024-01-31T05:10:18.183 INFO:teuthology.orchestra.run.smithi156.stdout:(15/155): libcephsqlite-19.0.0-885.g661946e7.el 2.1 MB/s | 163 kB 00:00 2024-01-31T05:10:18.249 INFO:teuthology.orchestra.run.smithi186.stdout:(71/155): postgresql-private-libs-13.11-1.el9.x 736 kB/s | 135 kB 00:00 2024-01-31T05:10:18.258 INFO:teuthology.orchestra.run.smithi156.stdout:(16/155): librados-devel-19.0.0-885.g661946e7.e 1.7 MB/s | 127 kB 00:00 2024-01-31T05:10:18.316 INFO:teuthology.orchestra.run.smithi186.stdout:(72/155): openssl-devel-3.0.7-25.el9.x86_64.rpm 1.9 MB/s | 4.1 MB 00:02 2024-01-31T05:10:18.408 INFO:teuthology.orchestra.run.smithi186.stdout:(73/155): postgresql-13.11-1.el9.x86_64.rpm 817 kB/s | 1.6 MB 00:01 2024-01-31T05:10:18.433 INFO:teuthology.orchestra.run.smithi186.stdout:(74/155): python3-attrs-20.3.0-7.el9.noarch.rpm 750 kB/s | 87 kB 00:00 2024-01-31T05:10:18.466 INFO:teuthology.orchestra.run.smithi156.stdout:(17/155): libradosstriper1-19.0.0-885.g661946e7 2.3 MB/s | 499 kB 00:00 2024-01-31T05:10:18.700 INFO:teuthology.orchestra.run.smithi186.stdout:(75/155): python3-devel-3.9.17-2.el9.x86_64.rpm 919 kB/s | 245 kB 00:00 2024-01-31T05:10:18.967 INFO:teuthology.orchestra.run.smithi186.stdout:(76/155): python3-jinja2-2.11.3-4.el9.noarch.rp 936 kB/s | 249 kB 00:00 2024-01-31T05:10:19.084 INFO:teuthology.orchestra.run.smithi186.stdout:(77/155): python3-jsonpatch-1.21-16.el9.noarch. 220 kB/s | 26 kB 00:00 2024-01-31T05:10:19.193 INFO:teuthology.orchestra.run.smithi186.stdout:(78/155): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-01-31T05:10:19.340 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-01-31T05:10:19.376 INFO:teuthology.orchestra.run.smithi186.stdout:(79/155): python3-mako-1.1.4-6.el9.noarch.rpm 939 kB/s | 172 kB 00:00 2024-01-31T05:10:19.485 INFO:teuthology.orchestra.run.smithi186.stdout:(80/155): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-01-31T05:10:20.109 INFO:teuthology.orchestra.run.smithi156.stdout:(18/155): ceph-test-19.0.0-885.g661946e7.el9.x8 18 MB/s | 46 MB 00:02 2024-01-31T05:10:20.209 INFO:teuthology.orchestra.run.smithi156.stdout:(19/155): librgw2-19.0.0-885.g661946e7.el9.x86_ 2.7 MB/s | 4.8 MB 00:01 2024-01-31T05:10:20.225 INFO:teuthology.orchestra.run.smithi156.stdout:(20/155): python3-ceph-argparse-19.0.0-885.g661 387 kB/s | 45 kB 00:00 2024-01-31T05:10:20.259 INFO:teuthology.orchestra.run.smithi156.stdout:(21/155): python3-ceph-common-19.0.0-885.g66194 2.5 MB/s | 130 kB 00:00 2024-01-31T05:10:20.293 INFO:teuthology.orchestra.run.smithi156.stdout:(22/155): python3-cephfs-19.0.0-885.g661946e7.e 2.4 MB/s | 162 kB 00:00 2024-01-31T05:10:20.360 INFO:teuthology.orchestra.run.smithi156.stdout:(23/155): python3-rados-19.0.0-885.g661946e7.el 3.1 MB/s | 321 kB 00:00 2024-01-31T05:10:20.385 INFO:teuthology.orchestra.run.smithi156.stdout:(24/155): python3-rbd-19.0.0-885.g661946e7.el9. 3.2 MB/s | 298 kB 00:00 2024-01-31T05:10:20.419 INFO:teuthology.orchestra.run.smithi156.stdout:(25/155): python3-rgw-19.0.0-885.g661946e7.el9. 1.7 MB/s | 99 kB 00:00 2024-01-31T05:10:20.444 INFO:teuthology.orchestra.run.smithi156.stdout:(26/155): rbd-fuse-19.0.0-885.g661946e7.el9.x86 1.4 MB/s | 85 kB 00:00 2024-01-31T05:10:20.544 INFO:teuthology.orchestra.run.smithi156.stdout:(27/155): rbd-nbd-19.0.0-885.g661946e7.el9.x86_ 1.7 MB/s | 170 kB 00:00 2024-01-31T05:10:20.594 INFO:teuthology.orchestra.run.smithi156.stdout:(28/155): ceph-grafana-dashboards-19.0.0-885.g6 569 kB/s | 28 kB 00:00 2024-01-31T05:10:20.662 INFO:teuthology.orchestra.run.smithi156.stdout:(29/155): rbd-mirror-19.0.0-885.g661946e7.el9.x 13 MB/s | 3.1 MB 00:00 2024-01-31T05:10:20.687 INFO:teuthology.orchestra.run.smithi156.stdout:(30/155): ceph-mgr-cephadm-19.0.0-885.g661946e7 1.4 MB/s | 133 kB 00:00 2024-01-31T05:10:20.804 INFO:teuthology.orchestra.run.smithi156.stdout:(31/155): ceph-mgr-dashboard-19.0.0-885.g661946 12 MB/s | 1.7 MB 00:00 2024-01-31T05:10:21.004 INFO:teuthology.orchestra.run.smithi156.stdout:(32/155): ceph-common-19.0.0-885.g661946e7.el9. 2.3 MB/s | 19 MB 00:08 2024-01-31T05:10:21.030 INFO:teuthology.orchestra.run.smithi156.stdout:(33/155): ceph-mgr-modules-core-19.0.0-885.g661 1.1 MB/s | 245 kB 00:00 2024-01-31T05:10:21.071 INFO:teuthology.orchestra.run.smithi156.stdout:(34/155): ceph-mgr-rook-19.0.0-885.g661946e7.el 763 kB/s | 51 kB 00:00 2024-01-31T05:10:21.097 INFO:teuthology.orchestra.run.smithi156.stdout:(35/155): ceph-prometheus-alerts-19.0.0-885.g66 226 kB/s | 15 kB 00:00 2024-01-31T05:10:21.139 INFO:teuthology.orchestra.run.smithi156.stdout:(36/155): ceph-volume-19.0.0-885.g661946e7.el9. 3.8 MB/s | 262 kB 00:00 2024-01-31T05:10:21.197 INFO:teuthology.orchestra.run.smithi156.stdout:(37/155): cephfs-top-19.0.0-885.g661946e7.el9.n 314 kB/s | 18 kB 00:00 2024-01-31T05:10:21.281 INFO:teuthology.orchestra.run.smithi156.stdout:(38/155): libaio-devel-0.3.112-1.el8.x86_64.rpm 211 kB/s | 18 kB 00:00 2024-01-31T05:10:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:(39/155): cephadm-19.0.0-885.g661946e7.el9.noar 3.0 MB/s | 742 kB 00:00 2024-01-31T05:10:21.431 INFO:teuthology.orchestra.run.smithi156.stdout:(40/155): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-01-31T05:10:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:(41/155): jq-1.6-16.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-01-31T05:10:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:(42/155): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-01-31T05:10:21.540 INFO:teuthology.orchestra.run.smithi156.stdout:(43/155): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-01-31T05:10:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:(44/155): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 218 kB 00:00 2024-01-31T05:10:21.591 INFO:teuthology.orchestra.run.smithi156.stdout:(45/155): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-01-31T05:10:21.641 INFO:teuthology.orchestra.run.smithi156.stdout:(46/155): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-01-31T05:10:21.666 INFO:teuthology.orchestra.run.smithi156.stdout:(47/155): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-01-31T05:10:21.692 INFO:teuthology.orchestra.run.smithi156.stdout:(48/155): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-01-31T05:10:21.717 INFO:teuthology.orchestra.run.smithi156.stdout:(49/155): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-01-31T05:10:21.742 INFO:teuthology.orchestra.run.smithi156.stdout:(50/155): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-01-31T05:10:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-01-31T05:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-01-31T05:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-01-31T05:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240105104119:20125149.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-01-31T05:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-01-31T05:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-01-31T05:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-01-31T05:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.971 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-01-31T05:10:21.972 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.972 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-01-31T05:10:21.972 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.972 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-01-31T05:10:21.972 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.972 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-01-31T05:10:21.972 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.972 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-01-31T05:10:21.972 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240105093033:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.972 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-01-31T05:10:21.972 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-01-31T05:10:21.973 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.974 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-01-31T05:10:21.974 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.974 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-01-31T05:10:21.974 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.974 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-01-31T05:10:21.974 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.974 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-01-31T05:10:21.974 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.974 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-01-31T05:10:21.975 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820231201124232:182f7c73.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.975 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-01-31T05:10:21.975 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.975 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-01-31T05:10:21.975 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.975 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-01-31T05:10:21.975 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.975 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-01-31T05:10:21.975 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.975 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-01-31T05:10:21.975 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.975 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-01-31T05:10:21.976 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.976 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-01-31T05:10:21.976 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.976 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-01-31T05:10:21.976 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.976 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-01-31T05:10:21.976 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.976 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-01-31T05:10:21.976 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.976 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240117192000:9edba152.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.978 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-01-31T05:10:21.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.978 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-01-31T05:10:21.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.979 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-01-31T05:10:21.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.980 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-01-31T05:10:21.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.980 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-01-31T05:10:21.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.981 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-01-31T05:10:21.981 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-01-31T05:10:22.134 INFO:teuthology.orchestra.run.smithi156.stdout:(51/155): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-01-31T05:10:22.451 INFO:teuthology.orchestra.run.smithi156.stdout:(52/155): boost-program-options-1.75.0-8.el9.x8 338 kB/s | 107 kB 00:00 2024-01-31T05:10:22.493 INFO:teuthology.orchestra.run.smithi156.stdout:(53/155): automake-1.16.2-8.el9.noarch.rpm 888 kB/s | 689 kB 00:00 2024-01-31T05:10:22.527 INFO:teuthology.orchestra.run.smithi156.stdout:(54/155): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-01-31T05:10:22.543 INFO:teuthology.orchestra.run.smithi156.stdout:(55/155): elfutils-libelf-devel-0.190-2.el9.x86 235 kB/s | 21 kB 00:00 2024-01-31T05:10:22.555 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-31T05:10:22.556 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-31T05:10:22.570 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-31T05:10:22.570 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-31T05:10:22.571 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-31T05:10:22.572 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-31T05:10:22.574 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-31T05:10:22.577 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-31T05:10:22.578 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-31T05:10:22.619 INFO:teuthology.orchestra.run.smithi156.stdout:(56/155): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-01-31T05:10:22.635 INFO:teuthology.orchestra.run.smithi156.stdout:(57/155): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-01-31T05:10:22.668 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-01-31T05:10:22.693 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================== 2024-01-31T05:10:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-01-31T05:10:22.693 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================== 2024-01-31T05:10:22.693 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-01-31T05:10:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-01-31T05:10:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.0.0-885.g661946e7.el9 ceph 6.5 k 2024-01-31T05:10:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.0.0-885.g661946e7.el9 ceph 5.1 M 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 886 k 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-885.g661946e7.el9 ceph 145 k 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.0.0-885.g661946e7.el9 ceph 1.5 M 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 133 k 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 1.7 M 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 7.4 M 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 51 k 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 8.7 M 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.0.0-885.g661946e7.el9 ceph 46 M 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 262 k 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 742 k 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 217 k 2024-01-31T05:10:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell x86_64 2:19.0.0-885.g661946e7.el9 ceph 27 k 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 18 k 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 31 k 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 743 k 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 127 k 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-01-31T05:10:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.0.0-885.g661946e7.el9 ceph 162 k 2024-01-31T05:10:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.0.0-885.g661946e7.el9 ceph 321 k 2024-01-31T05:10:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 298 k 2024-01-31T05:10:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 99 k 2024-01-31T05:10:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 85 k 2024-01-31T05:10:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-31T05:10:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 170 k 2024-01-31T05:10:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-01-31T05:10:22.696 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-01-31T05:10:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager x86_64 1:1.45.10-1.el9 baseos 2.3 M 2024-01-31T05:10:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 baseos 1.8 M 2024-01-31T05:10:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 baseos 43 k 2024-01-31T05:10:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 baseos 247 k 2024-01-31T05:10:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-01-31T05:10:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-01-31T05:10:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-01-31T05:10:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-01-31T05:10:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-01-31T05:10:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-01-31T05:10:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.4 M 2024-01-31T05:10:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-31T05:10:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-01-31T05:10:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-01-31T05:10:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-01-31T05:10:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-01-31T05:10:22.698 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-01-31T05:10:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-01-31T05:10:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-01-31T05:10:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 19 M 2024-01-31T05:10:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 28 k 2024-01-31T05:10:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.0.0-885.g661946e7.el9 ceph 2.1 M 2024-01-31T05:10:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 245 k 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.7 M 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.0.0-885.g661946e7.el9 ceph 17 M 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 15 k 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.0.0-885.g661946e7.el9 ceph 25 k 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.0.0-885.g661946e7.el9 ceph 163 k 2024-01-31T05:10:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-01-31T05:10:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-31T05:10:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-01-31T05:10:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-01-31T05:10:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-01-31T05:10:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 499 k 2024-01-31T05:10:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-01-31T05:10:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.8 M 2024-01-31T05:10:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-31T05:10:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-01-31T05:10:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-01-31T05:10:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-31T05:10:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.0.0-885.g661946e7.el9 ceph 45 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 130 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-31T05:10:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-31T05:10:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-31T05:10:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-4.el9 appstream 249 k 2024-01-31T05:10:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-01-31T05:10:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-01-31T05:10:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-31T05:10:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-31T05:10:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-01-31T05:10:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-01-31T05:10:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-31T05:10:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-31T05:10:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-01-31T05:10:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-01-31T05:10:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-01-31T05:10:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-31T05:10:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-01-31T05:10:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-31T05:10:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-01-31T05:10:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-31T05:10:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-01-31T05:10:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-01-31T05:10:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-01-31T05:10:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-01-31T05:10:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================== 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout:Install 139 Packages 2024-01-31T05:10:22.706 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 16 Packages 2024-01-31T05:10:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:10:22.707 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 212 M 2024-01-31T05:10:22.707 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-01-31T05:10:22.827 INFO:teuthology.orchestra.run.smithi156.stdout:(58/155): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-01-31T05:10:22.845 INFO:teuthology.orchestra.run.smithi156.stdout:(59/155): flexiblas-openblas-openmp-3.0.4-8.el9 81 kB/s | 18 kB 00:00 2024-01-31T05:10:22.870 INFO:teuthology.orchestra.run.smithi156.stdout:(60/155): libpmemobj-1.12.1-1.el9.x86_64.rpm 686 kB/s | 160 kB 00:00 2024-01-31T05:10:22.923 INFO:teuthology.orchestra.run.smithi156.stdout:(61/155): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-01-31T05:10:22.995 INFO:teuthology.orchestra.run.smithi156.stdout:(62/155): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-01-31T05:10:23.029 INFO:teuthology.orchestra.run.smithi156.stdout:(63/155): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-01-31T05:10:23.054 INFO:teuthology.orchestra.run.smithi156.stdout:(64/155): libuuid-devel-2.37.4-15.el9.x86_64.rp 191 kB/s | 26 kB 00:00 2024-01-31T05:10:23.096 INFO:teuthology.orchestra.run.smithi156.stdout:(65/155): libzstd-devel-1.5.1-2.el9.x86_64.rpm 477 kB/s | 47 kB 00:00 2024-01-31T05:10:23.305 INFO:teuthology.orchestra.run.smithi156.stdout:(66/155): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.0 MB/s | 292 kB 00:00 2024-01-31T05:10:23.331 INFO:teuthology.orchestra.run.smithi156.stdout:(67/155): lua-5.4.4-4.el9.x86_64.rpm 684 kB/s | 188 kB 00:00 2024-01-31T05:10:23.347 INFO:teuthology.orchestra.run.smithi156.stdout:(68/155): openblas-0.3.21-2.el9.x86_64.rpm 141 kB/s | 35 kB 00:00 2024-01-31T05:10:23.572 INFO:teuthology.orchestra.run.smithi156.stdout:(69/155): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-01-31T05:10:23.583 INFO:teuthology.orchestra.run.smithi045.stdout:(1/155): ceph-19.0.0-885.g661946e7.el9.x86_64.r 23 kB/s | 6.5 kB 00:00 2024-01-31T05:10:23.714 INFO:teuthology.orchestra.run.smithi156.stdout:(70/155): openssl-devel-3.0.7-25.el9.x86_64.rpm 11 MB/s | 4.1 MB 00:00 2024-01-31T05:10:23.748 INFO:teuthology.orchestra.run.smithi156.stdout:(71/155): postgresql-13.11-1.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-01-31T05:10:23.773 INFO:teuthology.orchestra.run.smithi156.stdout:(72/155): postgresql-contrib-13.11-1.el9.x86_64 4.4 MB/s | 905 kB 00:00 2024-01-31T05:10:23.815 INFO:teuthology.orchestra.run.smithi156.stdout:(73/155): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 135 kB 00:00 2024-01-31T05:10:23.957 INFO:teuthology.orchestra.run.smithi156.stdout:(74/155): python3-attrs-20.3.0-7.el9.noarch.rpm 476 kB/s | 87 kB 00:00 2024-01-31T05:10:24.040 INFO:teuthology.orchestra.run.smithi156.stdout:(75/155): postgresql-server-13.11-1.el9.x86_64. 20 MB/s | 5.7 MB 00:00 2024-01-31T05:10:24.216 INFO:teuthology.orchestra.run.smithi156.stdout:(76/155): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-01-31T05:10:24.241 INFO:teuthology.orchestra.run.smithi156.stdout:(77/155): python3-devel-3.9.17-2.el9.x86_64.rpm 865 kB/s | 245 kB 00:00 2024-01-31T05:10:24.266 INFO:teuthology.orchestra.run.smithi156.stdout:(78/155): python3-jinja2-2.11.3-4.el9.noarch.rp 1.1 MB/s | 249 kB 00:00 2024-01-31T05:10:24.316 INFO:teuthology.orchestra.run.smithi156.stdout:(79/155): python3-jsonpatch-1.21-16.el9.noarch. 257 kB/s | 26 kB 00:00 2024-01-31T05:10:24.367 INFO:teuthology.orchestra.run.smithi156.stdout:(80/155): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-01-31T05:10:24.392 INFO:teuthology.orchestra.run.smithi156.stdout:(81/155): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-01-31T05:10:24.417 INFO:teuthology.orchestra.run.smithi156.stdout:(82/155): python3-markupsafe-1.1.1-12.el9.x86_6 345 kB/s | 35 kB 00:00 2024-01-31T05:10:24.642 INFO:teuthology.orchestra.run.smithi156.stdout:(83/155): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-01-31T05:10:24.643 INFO:teuthology.orchestra.run.smithi186.stdout:(81/155): python3-babel-2.9.1-2.el9.noarch.rpm 979 kB/s | 6.0 MB 00:06 2024-01-31T05:10:24.666 INFO:teuthology.orchestra.run.smithi045.stdout:(2/155): ceph-fuse-19.0.0-885.g661946e7.el9.x86 818 kB/s | 886 kB 00:01 2024-01-31T05:10:24.667 INFO:teuthology.orchestra.run.smithi156.stdout:(84/155): python3-numpy-f2py-1.20.1-5.el9.x86_6 909 kB/s | 250 kB 00:00 2024-01-31T05:10:24.693 INFO:teuthology.orchestra.run.smithi156.stdout:(85/155): python3-oauthlib-3.1.1-5.el9.noarch.r 807 kB/s | 222 kB 00:00 2024-01-31T05:10:24.743 INFO:teuthology.orchestra.run.smithi156.stdout:(86/155): python3-prettytable-0.7.2-27.el9.noar 424 kB/s | 42 kB 00:00 2024-01-31T05:10:24.793 INFO:teuthology.orchestra.run.smithi156.stdout:(87/155): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-01-31T05:10:24.818 INFO:teuthology.orchestra.run.smithi156.stdout:(88/155): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-01-31T05:10:24.833 INFO:teuthology.orchestra.run.smithi045.stdout:(3/155): ceph-immutable-object-cache-19.0.0-885 874 kB/s | 145 kB 00:00 2024-01-31T05:10:24.844 INFO:teuthology.orchestra.run.smithi156.stdout:(89/155): python3-pytz-2021.1-5.el9.noarch.rpm 507 kB/s | 51 kB 00:00 2024-01-31T05:10:24.894 INFO:teuthology.orchestra.run.smithi156.stdout:(90/155): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-01-31T05:10:24.910 INFO:teuthology.orchestra.run.smithi186.stdout:(82/155): python3-numpy-f2py-1.20.1-5.el9.x86_6 941 kB/s | 250 kB 00:00 2024-01-31T05:10:25.036 INFO:teuthology.orchestra.run.smithi156.stdout:(91/155): python3-toml-0.10.2-6.el9.noarch.rpm 217 kB/s | 42 kB 00:00 2024-01-31T05:10:25.061 INFO:teuthology.orchestra.run.smithi156.stdout:(92/155): python3-wcwidth-0.2.5-8.el9.noarch.rp 264 kB/s | 44 kB 00:00 2024-01-31T05:10:25.177 INFO:teuthology.orchestra.run.smithi186.stdout:(83/155): python3-oauthlib-3.1.1-5.el9.noarch.r 833 kB/s | 222 kB 00:00 2024-01-31T05:10:25.262 INFO:teuthology.orchestra.run.smithi156.stdout:(93/155): uuid-1.6.2-55.el9.x86_64.rpm 293 kB/s | 58 kB 00:00 2024-01-31T05:10:25.286 INFO:teuthology.orchestra.run.smithi186.stdout:(84/155): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-01-31T05:10:25.361 INFO:teuthology.orchestra.run.smithi186.stdout:(85/155): postgresql-server-13.11-1.el9.x86_64. 827 kB/s | 5.7 MB 00:07 2024-01-31T05:10:25.412 INFO:teuthology.orchestra.run.smithi156.stdout:(94/155): socat-1.7.4.1-5.el9.x86_64.rpm 811 kB/s | 305 kB 00:00 2024-01-31T05:10:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:(4/155): ceph-mds-19.0.0-885.g661946e7.el9.x86_ 3.4 MB/s | 2.1 MB 00:00 2024-01-31T05:10:25.469 INFO:teuthology.orchestra.run.smithi186.stdout:(86/155): python3-pyasn1-0.4.8-6.el9.noarch.rpm 865 kB/s | 159 kB 00:00 2024-01-31T05:10:25.561 INFO:teuthology.orchestra.run.smithi186.stdout:(87/155): python3-pytz-2021.1-5.el9.noarch.rpm 556 kB/s | 51 kB 00:00 2024-01-31T05:10:25.571 INFO:teuthology.orchestra.run.smithi156.stdout:(95/155): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-01-31T05:10:25.596 INFO:teuthology.orchestra.run.smithi156.stdout:(96/155): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 283 kB/s | 94 kB 00:00 2024-01-31T05:10:25.621 INFO:teuthology.orchestra.run.smithi156.stdout:(97/155): xmlstarlet-1.6.1-20.el9.x86_64.rpm 305 kB/s | 64 kB 00:00 2024-01-31T05:10:25.625 INFO:teuthology.orchestra.run.smithi045.stdout:(5/155): ceph-base-19.0.0-885.g661946e7.el9.x86 2.2 MB/s | 5.1 MB 00:02 2024-01-31T05:10:25.653 INFO:teuthology.orchestra.run.smithi186.stdout:(88/155): python3-requests-oauthlib-1.3.0-12.el 586 kB/s | 54 kB 00:00 2024-01-31T05:10:25.671 INFO:teuthology.orchestra.run.smithi156.stdout:(98/155): zlib-devel-1.2.11-40.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-01-31T05:10:25.712 INFO:teuthology.orchestra.run.smithi186.stdout:(89/155): python3-pyasn1-modules-0.4.8-6.el9.no 795 kB/s | 279 kB 00:00 2024-01-31T05:10:25.721 INFO:teuthology.orchestra.run.smithi156.stdout:(99/155): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-01-31T05:10:25.776 INFO:teuthology.orchestra.run.smithi045.stdout:(6/155): ceph-mgr-19.0.0-885.g661946e7.el9.x86_ 4.6 MB/s | 1.5 MB 00:00 2024-01-31T05:10:25.821 INFO:teuthology.orchestra.run.smithi186.stdout:(90/155): python3-numpy-1.20.1-5.el9.x86_64.rpm 824 kB/s | 5.1 MB 00:06 2024-01-31T05:10:25.838 INFO:teuthology.orchestra.run.smithi186.stdout:(91/155): python3-toml-0.10.2-6.el9.noarch.rpm 332 kB/s | 42 kB 00:00 2024-01-31T05:10:25.921 INFO:teuthology.orchestra.run.smithi186.stdout:(92/155): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-01-31T05:10:25.955 INFO:teuthology.orchestra.run.smithi156.stdout:(100/155): libarrow-doc-9.0.0-10.el9.noarch.rpm 143 kB/s | 26 kB 00:00 2024-01-31T05:10:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:(101/155): gperftools-libs-2.9.1-2.el9.x86_64.r 841 kB/s | 309 kB 00:00 2024-01-31T05:10:26.013 INFO:teuthology.orchestra.run.smithi186.stdout:(93/155): uuid-1.6.2-55.el9.x86_64.rpm 632 kB/s | 58 kB 00:00 2024-01-31T05:10:26.039 INFO:teuthology.orchestra.run.smithi156.stdout:(102/155): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-01-31T05:10:26.064 INFO:teuthology.orchestra.run.smithi156.stdout:(103/155): libunwind-1.6.2-1.el9.x86_64.rpm 896 kB/s | 67 kB 00:00 2024-01-31T05:10:26.122 INFO:teuthology.orchestra.run.smithi186.stdout:(94/155): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 867 kB/s | 94 kB 00:00 2024-01-31T05:10:26.123 INFO:teuthology.orchestra.run.smithi156.stdout:(104/155): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-01-31T05:10:26.147 INFO:teuthology.orchestra.run.smithi186.stdout:(95/155): socat-1.7.4.1-5.el9.x86_64.rpm 986 kB/s | 305 kB 00:00 2024-01-31T05:10:26.173 INFO:teuthology.orchestra.run.smithi156.stdout:(105/155): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.5 MB/s | 838 kB 00:00 2024-01-31T05:10:26.222 INFO:teuthology.orchestra.run.smithi186.stdout:(96/155): xmlstarlet-1.6.1-20.el9.x86_64.rpm 635 kB/s | 64 kB 00:00 2024-01-31T05:10:26.231 INFO:teuthology.orchestra.run.smithi156.stdout:(106/155): python3-bcrypt-3.2.2-1.el9.x86_64.rp 748 kB/s | 43 kB 00:00 2024-01-31T05:10:26.248 INFO:teuthology.orchestra.run.smithi186.stdout:(97/155): zlib-devel-1.2.11-40.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-01-31T05:10:26.273 INFO:teuthology.orchestra.run.smithi156.stdout:(107/155): python3-asyncssh-2.13.2-3.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-01-31T05:10:26.299 INFO:teuthology.orchestra.run.smithi156.stdout:(108/155): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-01-31T05:10:26.314 INFO:teuthology.orchestra.run.smithi186.stdout:(98/155): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-01-31T05:10:26.332 INFO:teuthology.orchestra.run.smithi156.stdout:(109/155): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2024-01-31T05:10:26.391 INFO:teuthology.orchestra.run.smithi156.stdout:(110/155): libarrow-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-01-31T05:10:26.416 INFO:teuthology.orchestra.run.smithi156.stdout:(111/155): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-01-31T05:10:26.441 INFO:teuthology.orchestra.run.smithi156.stdout:(112/155): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-01-31T05:10:26.466 INFO:teuthology.orchestra.run.smithi156.stdout:(113/155): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-01-31T05:10:26.492 INFO:teuthology.orchestra.run.smithi156.stdout:(114/155): python3-colorama-0.4.4-12.el9.noarch 468 kB/s | 35 kB 00:00 2024-01-31T05:10:26.515 INFO:teuthology.orchestra.run.smithi186.stdout:(99/155): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.1 MB/s | 309 kB 00:00 2024-01-31T05:10:26.517 INFO:teuthology.orchestra.run.smithi156.stdout:(115/155): python3-google-auth-2.26.2-1.el9.noa 2.6 MB/s | 202 kB 00:00 2024-01-31T05:10:26.517 INFO:teuthology.orchestra.run.smithi045.stdout:(7/155): ceph-mon-19.0.0-885.g661946e7.el9.x86_ 5.3 MB/s | 4.7 MB 00:00 2024-01-31T05:10:26.534 INFO:teuthology.orchestra.run.smithi156.stdout:(116/155): python3-jaraco-8.2.1-3.el9.noarch.rp 159 kB/s | 11 kB 00:00 2024-01-31T05:10:26.548 INFO:teuthology.orchestra.run.smithi186.stdout:(100/155): libarrow-doc-9.0.0-10.el9.noarch.rpm 784 kB/s | 26 kB 00:00 2024-01-31T05:10:26.551 INFO:teuthology.orchestra.run.smithi156.stdout:(117/155): python3-jaraco-classes-3.2.1-5.el9.n 300 kB/s | 18 kB 00:00 2024-01-31T05:10:26.568 INFO:teuthology.orchestra.run.smithi156.stdout:(118/155): python3-jaraco-collections-3.0.0-8.e 458 kB/s | 23 kB 00:00 2024-01-31T05:10:26.582 INFO:teuthology.orchestra.run.smithi186.stdout:(101/155): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-01-31T05:10:26.585 INFO:teuthology.orchestra.run.smithi156.stdout:(119/155): python3-jaraco-functools-3.5.0-2.el9 385 kB/s | 19 kB 00:00 2024-01-31T05:10:26.602 INFO:teuthology.orchestra.run.smithi156.stdout:(120/155): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-01-31T05:10:26.624 INFO:teuthology.orchestra.run.smithi186.stdout:(102/155): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-01-31T05:10:26.635 INFO:teuthology.orchestra.run.smithi156.stdout:(121/155): python3-logutils-0.3.5-21.el9.noarch 925 kB/s | 46 kB 00:00 2024-01-31T05:10:26.669 INFO:teuthology.orchestra.run.smithi156.stdout:(122/155): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-01-31T05:10:26.683 INFO:teuthology.orchestra.run.smithi186.stdout:(103/155): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-01-31T05:10:26.694 INFO:teuthology.orchestra.run.smithi156.stdout:(123/155): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-01-31T05:10:26.728 INFO:teuthology.orchestra.run.smithi156.stdout:(124/155): python3-more-itertools-8.12.0-2.el9. 625 kB/s | 79 kB 00:00 2024-01-31T05:10:26.753 INFO:teuthology.orchestra.run.smithi156.stdout:(125/155): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-01-31T05:10:26.779 INFO:teuthology.orchestra.run.smithi156.stdout:(126/155): python3-portend-3.1.0-2.el9.noarch.r 195 kB/s | 16 kB 00:00 2024-01-31T05:10:26.804 INFO:teuthology.orchestra.run.smithi156.stdout:(127/155): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-01-31T05:10:26.829 INFO:teuthology.orchestra.run.smithi156.stdout:(128/155): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-01-31T05:10:26.854 INFO:teuthology.orchestra.run.smithi156.stdout:(129/155): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-01-31T05:10:26.879 INFO:teuthology.orchestra.run.smithi156.stdout:(130/155): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-01-31T05:10:26.905 INFO:teuthology.orchestra.run.smithi156.stdout:(131/155): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-01-31T05:10:26.930 INFO:teuthology.orchestra.run.smithi156.stdout:(132/155): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-01-31T05:10:26.941 INFO:teuthology.orchestra.run.smithi186.stdout:(104/155): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 838 kB 00:00 2024-01-31T05:10:26.955 INFO:teuthology.orchestra.run.smithi156.stdout:(133/155): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-01-31T05:10:26.981 INFO:teuthology.orchestra.run.smithi156.stdout:(134/155): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-01-31T05:10:27.007 INFO:teuthology.orchestra.run.smithi156.stdout:(135/155): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-01-31T05:10:27.039 INFO:teuthology.orchestra.run.smithi156.stdout:(136/155): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-01-31T05:10:27.065 INFO:teuthology.orchestra.run.smithi156.stdout:(137/155): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-01-31T05:10:27.091 INFO:teuthology.orchestra.run.smithi156.stdout:(138/155): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-01-31T05:10:27.101 INFO:teuthology.orchestra.run.smithi045.stdout:(8/155): ceph-osd-19.0.0-885.g661946e7.el9.x86_ 13 MB/s | 17 MB 00:01 2024-01-31T05:10:27.116 INFO:teuthology.orchestra.run.smithi186.stdout:(105/155): python3-asyncssh-2.13.2-3.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-01-31T05:10:27.150 INFO:teuthology.orchestra.run.smithi186.stdout:(106/155): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-01-31T05:10:27.168 INFO:teuthology.orchestra.run.smithi045.stdout:(9/155): ceph-selinux-19.0.0-885.g661946e7.el9. 378 kB/s | 25 kB 00:00 2024-01-31T05:10:27.184 INFO:teuthology.orchestra.run.smithi186.stdout:(107/155): python3-cachetools-4.2.4-1.el9.noarc 972 kB/s | 32 kB 00:00 2024-01-31T05:10:27.198 INFO:teuthology.orchestra.run.smithi156.stdout:(139/155): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-01-31T05:10:27.217 INFO:teuthology.orchestra.run.smithi186.stdout:(108/155): python3-certifi-2021.10.8-2.el9.noar 451 kB/s | 15 kB 00:00 2024-01-31T05:10:27.293 INFO:teuthology.orchestra.run.smithi186.stdout:(109/155): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-01-31T05:10:27.399 INFO:teuthology.orchestra.run.smithi156.stdout:(140/155): NetworkManager-1.45.10-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-01-31T05:10:27.435 INFO:teuthology.orchestra.run.smithi186.stdout:(110/155): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-01-31T05:10:27.474 INFO:teuthology.orchestra.run.smithi156.stdout:(141/155): NetworkManager-libnm-1.45.10-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-01-31T05:10:27.516 INFO:teuthology.orchestra.run.smithi156.stdout:(142/155): NetworkManager-team-1.45.10-1.el9.x8 1.0 MB/s | 43 kB 00:00 2024-01-31T05:10:27.543 INFO:teuthology.orchestra.run.smithi186.stdout:(111/155): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-01-31T05:10:27.558 INFO:teuthology.orchestra.run.smithi156.stdout:(143/155): NetworkManager-tui-1.45.10-1.el9.x86 5.8 MB/s | 247 kB 00:00 2024-01-31T05:10:27.585 INFO:teuthology.orchestra.run.smithi186.stdout:(112/155): python3-colorama-0.4.4-12.el9.noarch 849 kB/s | 35 kB 00:00 2024-01-31T05:10:27.600 INFO:teuthology.orchestra.run.smithi156.stdout:(144/155): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-01-31T05:10:27.642 INFO:teuthology.orchestra.run.smithi156.stdout:(145/155): elfutils-debuginfod-client-0.190-2.e 874 kB/s | 36 kB 00:00 2024-01-31T05:10:27.661 INFO:teuthology.orchestra.run.smithi186.stdout:(113/155): python3-google-auth-2.26.2-1.el9.noa 2.6 MB/s | 202 kB 00:00 2024-01-31T05:10:27.684 INFO:teuthology.orchestra.run.smithi156.stdout:(146/155): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-01-31T05:10:27.702 INFO:teuthology.orchestra.run.smithi186.stdout:(114/155): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-01-31T05:10:27.725 INFO:teuthology.orchestra.run.smithi156.stdout:(147/155): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-01-31T05:10:27.744 INFO:teuthology.orchestra.run.smithi186.stdout:(115/155): python3-jaraco-classes-3.2.1-5.el9.n 425 kB/s | 18 kB 00:00 2024-01-31T05:10:27.768 INFO:teuthology.orchestra.run.smithi156.stdout:(148/155): libacl-2.3.1-4.el9.x86_64.rpm 553 kB/s | 23 kB 00:00 2024-01-31T05:10:27.786 INFO:teuthology.orchestra.run.smithi186.stdout:(116/155): python3-jaraco-collections-3.0.0-8.e 557 kB/s | 23 kB 00:00 2024-01-31T05:10:27.828 INFO:teuthology.orchestra.run.smithi186.stdout:(117/155): python3-jaraco-functools-3.5.0-2.el9 468 kB/s | 19 kB 00:00 2024-01-31T05:10:27.834 INFO:teuthology.orchestra.run.smithi156.stdout:(149/155): librados2-19.0.0-885.g661946e7.el9.x 4.4 MB/s | 3.4 MB 00:00 2024-01-31T05:10:27.868 INFO:teuthology.orchestra.run.smithi156.stdout:(150/155): openssl-3.0.7-25.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-01-31T05:10:27.870 INFO:teuthology.orchestra.run.smithi186.stdout:(118/155): python3-jaraco-text-3.2.0-6.el9.noar 471 kB/s | 20 kB 00:00 2024-01-31T05:10:27.910 INFO:teuthology.orchestra.run.smithi156.stdout:(151/155): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2024-01-31T05:10:27.960 INFO:teuthology.orchestra.run.smithi156.stdout:(152/155): xfsprogs-6.3.0-1.el9.x86_64.rpm 21 MB/s | 1.1 MB 00:00 2024-01-31T05:10:28.021 INFO:teuthology.orchestra.run.smithi186.stdout:(119/155): libarrow-9.0.0-10.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-01-31T05:10:28.036 INFO:teuthology.orchestra.run.smithi156.stdout:(153/155): openssl-libs-3.0.7-25.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-01-31T05:10:28.061 INFO:teuthology.orchestra.run.smithi156.stdout:(154/155): libacl-devel-2.3.1-4.el9.x86_64.rpm 743 kB/s | 74 kB 00:00 2024-01-31T05:10:28.079 INFO:teuthology.orchestra.run.smithi186.stdout:(120/155): python3-logutils-0.3.5-21.el9.noarch 795 kB/s | 46 kB 00:00 2024-01-31T05:10:28.138 INFO:teuthology.orchestra.run.smithi186.stdout:(121/155): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-01-31T05:10:28.197 INFO:teuthology.orchestra.run.smithi186.stdout:(122/155): python3-natsort-7.1.1-5.el9.noarch.r 993 kB/s | 58 kB 00:00 2024-01-31T05:10:28.227 INFO:teuthology.orchestra.run.smithi156.stdout:(155/155): librbd1-19.0.0-885.g661946e7.el9.x86 2.8 MB/s | 3.1 MB 00:01 2024-01-31T05:10:28.236 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-01-31T05:10:28.236 INFO:teuthology.orchestra.run.smithi156.stdout:Total 13 MB/s | 212 MB 00:16 2024-01-31T05:10:28.238 INFO:teuthology.orchestra.run.smithi186.stdout:(123/155): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-01-31T05:10:28.280 INFO:teuthology.orchestra.run.smithi186.stdout:(124/155): python3-portend-3.1.0-2.el9.noarch.r 394 kB/s | 16 kB 00:00 2024-01-31T05:10:28.339 INFO:teuthology.orchestra.run.smithi186.stdout:(125/155): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-01-31T05:10:28.364 INFO:teuthology.orchestra.run.smithi186.stdout:(126/155): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-01-31T05:10:28.381 INFO:teuthology.orchestra.run.smithi186.stdout:(127/155): python3-pyperclip-1.8.2-2.el9.noarch 674 kB/s | 28 kB 00:00 2024-01-31T05:10:28.398 INFO:teuthology.orchestra.run.smithi186.stdout:(128/155): python3-repoze-lru-0.7-16.el9.noarch 914 kB/s | 31 kB 00:00 2024-01-31T05:10:28.457 INFO:teuthology.orchestra.run.smithi186.stdout:(129/155): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-01-31T05:10:28.474 INFO:teuthology.orchestra.run.smithi186.stdout:(130/155): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-01-31T05:10:28.491 INFO:teuthology.orchestra.run.smithi186.stdout:(131/155): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-01-31T05:10:28.525 INFO:teuthology.orchestra.run.smithi186.stdout:(132/155): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-01-31T05:10:28.575 INFO:teuthology.orchestra.run.smithi186.stdout:(133/155): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-01-31T05:10:28.600 INFO:teuthology.orchestra.run.smithi186.stdout:(134/155): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-01-31T05:10:28.633 INFO:teuthology.orchestra.run.smithi186.stdout:(135/155): python3-zc-lockfile-2.0-10.el9.noarc 605 kB/s | 20 kB 00:00 2024-01-31T05:10:28.709 INFO:teuthology.orchestra.run.smithi186.stdout:(136/155): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-01-31T05:10:28.751 INFO:teuthology.orchestra.run.smithi186.stdout:(137/155): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-01-31T05:10:28.793 INFO:teuthology.orchestra.run.smithi045.stdout:(10/155): ceph-radosgw-19.0.0-885.g661946e7.el9 3.8 MB/s | 8.7 MB 00:02 2024-01-31T05:10:28.893 INFO:teuthology.orchestra.run.smithi045.stdout:(11/155): cephfs-mirror-19.0.0-885.g661946e7.el 2.1 MB/s | 217 kB 00:00 2024-01-31T05:10:28.952 INFO:teuthology.orchestra.run.smithi045.stdout:(12/155): cephfs-shell-19.0.0-885.g661946e7.el9 463 kB/s | 27 kB 00:00 2024-01-31T05:10:29.019 INFO:teuthology.orchestra.run.smithi045.stdout:(13/155): libcephfs-devel-19.0.0-885.g661946e7. 469 kB/s | 31 kB 00:00 2024-01-31T05:10:29.251 INFO:teuthology.orchestra.run.smithi186.stdout:(138/155): thrift-0.15.0-2.el9.x86_64.rpm 2.9 MB/s | 1.6 MB 00:00 2024-01-31T05:10:29.353 INFO:teuthology.orchestra.run.smithi045.stdout:(14/155): libcephfs2-19.0.0-885.g661946e7.el9.x 2.2 MB/s | 743 kB 00:00 2024-01-31T05:10:29.427 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-01-31T05:10:29.435 INFO:teuthology.orchestra.run.smithi186.stdout:(139/155): librados2-19.0.0-885.g661946e7.el9.x 4.9 MB/s | 3.4 MB 00:00 2024-01-31T05:10:29.474 INFO:teuthology.orchestra.run.smithi045.stdout:(15/155): libcephsqlite-19.0.0-885.g661946e7.el 1.3 MB/s | 163 kB 00:00 2024-01-31T05:10:29.545 INFO:teuthology.orchestra.run.smithi045.stdout:(16/155): librados-devel-19.0.0-885.g661946e7.e 1.8 MB/s | 127 kB 00:00 2024-01-31T05:10:29.627 INFO:teuthology.orchestra.run.smithi186.stdout:(140/155): NetworkManager-1.45.10-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-01-31T05:10:29.682 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-01-31T05:10:29.682 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-01-31T05:10:29.694 INFO:teuthology.orchestra.run.smithi186.stdout:(141/155): NetworkManager-libnm-1.45.10-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-01-31T05:10:29.727 INFO:teuthology.orchestra.run.smithi186.stdout:(142/155): NetworkManager-team-1.45.10-1.el9.x8 1.3 MB/s | 43 kB 00:00 2024-01-31T05:10:29.769 INFO:teuthology.orchestra.run.smithi186.stdout:(143/155): NetworkManager-tui-1.45.10-1.el9.x86 5.8 MB/s | 247 kB 00:00 2024-01-31T05:10:29.803 INFO:teuthology.orchestra.run.smithi186.stdout:(144/155): acl-2.3.1-4.el9.x86_64.rpm 2.1 MB/s | 71 kB 00:00 2024-01-31T05:10:29.837 INFO:teuthology.orchestra.run.smithi186.stdout:(145/155): elfutils-debuginfod-client-0.190-2.e 1.1 MB/s | 36 kB 00:00 2024-01-31T05:10:29.879 INFO:teuthology.orchestra.run.smithi186.stdout:(146/155): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-01-31T05:10:29.921 INFO:teuthology.orchestra.run.smithi186.stdout:(147/155): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-01-31T05:10:29.963 INFO:teuthology.orchestra.run.smithi186.stdout:(148/155): libacl-2.3.1-4.el9.x86_64.rpm 554 kB/s | 23 kB 00:00 2024-01-31T05:10:29.995 INFO:teuthology.orchestra.run.smithi045.stdout:(17/155): ceph-test-19.0.0-885.g661946e7.el9.x8 16 MB/s | 46 MB 00:02 2024-01-31T05:10:30.013 INFO:teuthology.orchestra.run.smithi186.stdout:(149/155): openssl-3.0.7-25.el9.x86_64.rpm 24 MB/s | 1.2 MB 00:00 2024-01-31T05:10:30.029 INFO:teuthology.orchestra.run.smithi045.stdout:(18/155): libradosstriper1-19.0.0-885.g661946e7 1.0 MB/s | 499 kB 00:00 2024-01-31T05:10:30.088 INFO:teuthology.orchestra.run.smithi186.stdout:(150/155): openssl-libs-3.0.7-25.el9.x86_64.rpm 29 MB/s | 2.2 MB 00:00 2024-01-31T05:10:30.112 INFO:teuthology.orchestra.run.smithi045.stdout:(19/155): python3-ceph-argparse-19.0.0-885.g661 543 kB/s | 45 kB 00:00 2024-01-31T05:10:30.139 INFO:teuthology.orchestra.run.smithi186.stdout:(151/155): librbd1-19.0.0-885.g661946e7.el9.x86 3.5 MB/s | 3.1 MB 00:00 2024-01-31T05:10:30.164 INFO:teuthology.orchestra.run.smithi186.stdout:(152/155): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2024-01-31T05:10:30.171 INFO:teuthology.orchestra.run.smithi045.stdout:(20/155): python3-ceph-common-19.0.0-885.g66194 2.2 MB/s | 130 kB 00:00 2024-01-31T05:10:30.306 INFO:teuthology.orchestra.run.smithi186.stdout:(153/155): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.4 MB/s | 1.1 MB 00:00 2024-01-31T05:10:30.338 INFO:teuthology.orchestra.run.smithi045.stdout:(21/155): python3-cephfs-19.0.0-885.g661946e7.e 972 kB/s | 162 kB 00:00 2024-01-31T05:10:30.422 INFO:teuthology.orchestra.run.smithi186.stdout:(154/155): libacl-devel-2.3.1-4.el9.x86_64.rpm 287 kB/s | 74 kB 00:00 2024-01-31T05:10:30.505 INFO:teuthology.orchestra.run.smithi045.stdout:(22/155): python3-rados-19.0.0-885.g661946e7.el 1.9 MB/s | 321 kB 00:00 2024-01-31T05:10:30.647 INFO:teuthology.orchestra.run.smithi045.stdout:(23/155): librgw2-19.0.0-885.g661946e7.el9.x86_ 7.4 MB/s | 4.8 MB 00:00 2024-01-31T05:10:30.689 INFO:teuthology.orchestra.run.smithi045.stdout:(24/155): python3-rbd-19.0.0-885.g661946e7.el9. 1.6 MB/s | 298 kB 00:00 2024-01-31T05:10:30.747 INFO:teuthology.orchestra.run.smithi045.stdout:(25/155): python3-rgw-19.0.0-885.g661946e7.el9. 993 kB/s | 99 kB 00:00 2024-01-31T05:10:30.772 INFO:teuthology.orchestra.run.smithi045.stdout:(26/155): rbd-fuse-19.0.0-885.g661946e7.el9.x86 1.0 MB/s | 85 kB 00:00 2024-01-31T05:10:30.906 INFO:teuthology.orchestra.run.smithi045.stdout:(27/155): rbd-nbd-19.0.0-885.g661946e7.el9.x86_ 1.2 MB/s | 170 kB 00:00 2024-01-31T05:10:30.981 INFO:teuthology.orchestra.run.smithi045.stdout:(28/155): ceph-grafana-dashboards-19.0.0-885.g6 379 kB/s | 28 kB 00:00 2024-01-31T05:10:31.082 INFO:teuthology.orchestra.run.smithi045.stdout:(29/155): ceph-mgr-cephadm-19.0.0-885.g661946e7 1.3 MB/s | 133 kB 00:00 2024-01-31T05:10:31.249 INFO:teuthology.orchestra.run.smithi045.stdout:(30/155): rbd-mirror-19.0.0-885.g661946e7.el9.x 6.2 MB/s | 3.1 MB 00:00 2024-01-31T05:10:31.685 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-01-31T05:10:31.685 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-01-31T05:10:31.907 INFO:teuthology.orchestra.run.smithi045.stdout:(31/155): ceph-mgr-dashboard-19.0.0-885.g661946 2.1 MB/s | 1.7 MB 00:00 2024-01-31T05:10:32.082 INFO:teuthology.orchestra.run.smithi045.stdout:(32/155): ceph-mgr-modules-core-19.0.0-885.g661 1.4 MB/s | 245 kB 00:00 2024-01-31T05:10:32.174 INFO:teuthology.orchestra.run.smithi045.stdout:(33/155): ceph-mgr-rook-19.0.0-885.g661946e7.el 554 kB/s | 51 kB 00:00 2024-01-31T05:10:32.258 INFO:teuthology.orchestra.run.smithi045.stdout:(34/155): ceph-prometheus-alerts-19.0.0-885.g66 181 kB/s | 15 kB 00:00 2024-01-31T05:10:32.392 INFO:teuthology.orchestra.run.smithi045.stdout:(35/155): ceph-volume-19.0.0-885.g661946e7.el9. 1.9 MB/s | 262 kB 00:00 2024-01-31T05:10:32.550 INFO:teuthology.orchestra.run.smithi045.stdout:(36/155): ceph-mgr-diskprediction-local-19.0.0- 5.7 MB/s | 7.4 MB 00:01 2024-01-31T05:10:32.651 INFO:teuthology.orchestra.run.smithi045.stdout:(37/155): cephfs-top-19.0.0-885.g661946e7.el9.n 183 kB/s | 18 kB 00:00 2024-01-31T05:10:32.742 INFO:teuthology.orchestra.run.smithi045.stdout:(38/155): libaio-devel-0.3.112-1.el8.x86_64.rpm 192 kB/s | 18 kB 00:00 2024-01-31T05:10:32.785 INFO:teuthology.orchestra.run.smithi045.stdout:(39/155): cephadm-19.0.0-885.g661946e7.el9.noar 1.8 MB/s | 742 kB 00:00 2024-01-31T05:10:32.876 INFO:teuthology.orchestra.run.smithi045.stdout:(40/155): fuse-2.9.9-15.el9.x86_64.rpm 601 kB/s | 80 kB 00:00 2024-01-31T05:10:32.935 INFO:teuthology.orchestra.run.smithi045.stdout:(41/155): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-01-31T05:10:33.010 INFO:teuthology.orchestra.run.smithi045.stdout:(42/155): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-01-31T05:10:33.052 INFO:teuthology.orchestra.run.smithi045.stdout:(43/155): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-01-31T05:10:33.077 INFO:teuthology.orchestra.run.smithi045.stdout:(44/155): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 218 kB 00:00 2024-01-31T05:10:33.120 INFO:teuthology.orchestra.run.smithi045.stdout:(45/155): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-01-31T05:10:33.161 INFO:teuthology.orchestra.run.smithi045.stdout:(46/155): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-01-31T05:10:33.212 INFO:teuthology.orchestra.run.smithi045.stdout:(47/155): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-01-31T05:10:33.254 INFO:teuthology.orchestra.run.smithi045.stdout:(48/155): python3-cryptography-36.0.1-4.el9.x86 7.0 MB/s | 1.2 MB 00:00 2024-01-31T05:10:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:(49/155): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-01-31T05:10:33.321 INFO:teuthology.orchestra.run.smithi045.stdout:(50/155): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-01-31T05:10:36.579 INFO:teuthology.orchestra.run.smithi045.stdout:(51/155): bison-3.7.4-5.el9.x86_64.rpm 289 kB/s | 943 kB 00:03 2024-01-31T05:10:36.613 INFO:teuthology.orchestra.run.smithi045.stdout:(52/155): boost-program-options-1.75.0-8.el9.x8 3.2 MB/s | 107 kB 00:00 2024-01-31T05:10:36.646 INFO:teuthology.orchestra.run.smithi045.stdout:(53/155): elfutils-libelf-devel-0.190-2.el9.x86 644 kB/s | 21 kB 00:00 2024-01-31T05:10:36.697 INFO:teuthology.orchestra.run.smithi045.stdout:(54/155): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-01-31T05:10:36.730 INFO:teuthology.orchestra.run.smithi045.stdout:(55/155): flexiblas-3.0.4-8.el9.x86_64.rpm 983 kB/s | 33 kB 00:00 2024-01-31T05:10:36.847 INFO:teuthology.orchestra.run.smithi045.stdout:(56/155): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-01-31T05:10:36.881 INFO:teuthology.orchestra.run.smithi045.stdout:(57/155): flexiblas-openblas-openmp-3.0.4-8.el9 549 kB/s | 18 kB 00:00 2024-01-31T05:10:36.923 INFO:teuthology.orchestra.run.smithi045.stdout:(58/155): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-01-31T05:10:36.956 INFO:teuthology.orchestra.run.smithi045.stdout:(59/155): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-01-31T05:10:37.006 INFO:teuthology.orchestra.run.smithi045.stdout:(60/155): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-01-31T05:10:37.048 INFO:teuthology.orchestra.run.smithi045.stdout:(61/155): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 582 kB 00:00 2024-01-31T05:10:37.090 INFO:teuthology.orchestra.run.smithi045.stdout:(62/155): libuuid-devel-2.37.4-15.el9.x86_64.rp 615 kB/s | 26 kB 00:00 2024-01-31T05:10:37.133 INFO:teuthology.orchestra.run.smithi045.stdout:(63/155): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-01-31T05:10:37.174 INFO:teuthology.orchestra.run.smithi045.stdout:(64/155): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-01-31T05:10:37.216 INFO:teuthology.orchestra.run.smithi045.stdout:(65/155): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-01-31T05:10:37.258 INFO:teuthology.orchestra.run.smithi045.stdout:(66/155): openblas-0.3.21-2.el9.x86_64.rpm 846 kB/s | 35 kB 00:00 2024-01-31T05:10:37.359 INFO:teuthology.orchestra.run.smithi045.stdout:(67/155): openblas-openmp-0.3.21-2.el9.x86_64.r 47 MB/s | 4.7 MB 00:00 2024-01-31T05:10:37.551 INFO:teuthology.orchestra.run.smithi045.stdout:(68/155): ceph-common-19.0.0-885.g661946e7.el9. 1.4 MB/s | 19 MB 00:14 2024-01-31T05:10:37.626 INFO:teuthology.orchestra.run.smithi045.stdout:(69/155): openssl-devel-3.0.7-25.el9.x86_64.rpm 15 MB/s | 4.1 MB 00:00 2024-01-31T05:10:37.684 INFO:teuthology.orchestra.run.smithi045.stdout:(70/155): postgresql-contrib-13.11-1.el9.x86_64 15 MB/s | 905 kB 00:00 2024-01-31T05:10:37.726 INFO:teuthology.orchestra.run.smithi045.stdout:(71/155): postgresql-private-libs-13.11-1.el9.x 3.2 MB/s | 135 kB 00:00 2024-01-31T05:10:37.844 INFO:teuthology.orchestra.run.smithi045.stdout:(72/155): postgresql-server-13.11-1.el9.x86_64. 49 MB/s | 5.7 MB 00:00 2024-01-31T05:10:37.886 INFO:teuthology.orchestra.run.smithi045.stdout:(73/155): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-01-31T05:10:38.002 INFO:teuthology.orchestra.run.smithi045.stdout:(74/155): python3-babel-2.9.1-2.el9.noarch.rpm 51 MB/s | 6.0 MB 00:00 2024-01-31T05:10:38.044 INFO:teuthology.orchestra.run.smithi045.stdout:(75/155): postgresql-13.11-1.el9.x86_64.rpm 3.2 MB/s | 1.6 MB 00:00 2024-01-31T05:10:38.070 INFO:teuthology.orchestra.run.smithi045.stdout:(76/155): python3-devel-3.9.17-2.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-01-31T05:10:38.103 INFO:teuthology.orchestra.run.smithi045.stdout:(77/155): python3-jinja2-2.11.3-4.el9.noarch.rp 4.2 MB/s | 249 kB 00:00 2024-01-31T05:10:38.125 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-01-31T05:10:38.128 INFO:teuthology.orchestra.run.smithi045.stdout:(78/155): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-01-31T05:10:38.154 INFO:teuthology.orchestra.run.smithi045.stdout:(79/155): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-01-31T05:10:38.179 INFO:teuthology.orchestra.run.smithi045.stdout:(80/155): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-01-31T05:10:38.204 INFO:teuthology.orchestra.run.smithi045.stdout:(81/155): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-01-31T05:10:38.239 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/171 2024-01-31T05:10:38.278 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/171 2024-01-31T05:10:38.296 INFO:teuthology.orchestra.run.smithi045.stdout:(82/155): python3-numpy-1.20.1-5.el9.x86_64.rpm 44 MB/s | 5.1 MB 00:00 2024-01-31T05:10:38.321 INFO:teuthology.orchestra.run.smithi045.stdout:(83/155): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-01-31T05:10:38.339 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/171 2024-01-31T05:10:38.347 INFO:teuthology.orchestra.run.smithi045.stdout:(84/155): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-01-31T05:10:38.372 INFO:teuthology.orchestra.run.smithi045.stdout:(85/155): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-01-31T05:10:38.373 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/171 2024-01-31T05:10:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:(86/155): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-01-31T05:10:38.399 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/171 2024-01-31T05:10:38.431 INFO:teuthology.orchestra.run.smithi045.stdout:(87/155): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-01-31T05:10:38.441 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/171 2024-01-31T05:10:38.456 INFO:teuthology.orchestra.run.smithi045.stdout:(88/155): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-01-31T05:10:38.473 INFO:teuthology.orchestra.run.smithi045.stdout:(89/155): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-01-31T05:10:38.507 INFO:teuthology.orchestra.run.smithi045.stdout:(90/155): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-01-31T05:10:38.574 INFO:teuthology.orchestra.run.smithi045.stdout:(91/155): python3-wcwidth-0.2.5-8.el9.noarch.rp 662 kB/s | 44 kB 00:00 2024-01-31T05:10:38.615 INFO:teuthology.orchestra.run.smithi045.stdout:(92/155): socat-1.7.4.1-5.el9.x86_64.rpm 7.2 MB/s | 305 kB 00:00 2024-01-31T05:10:38.649 INFO:teuthology.orchestra.run.smithi045.stdout:(93/155): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2024-01-31T05:10:38.691 INFO:teuthology.orchestra.run.smithi045.stdout:(94/155): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-01-31T05:10:38.725 INFO:teuthology.orchestra.run.smithi045.stdout:(95/155): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-01-31T05:10:38.758 INFO:teuthology.orchestra.run.smithi045.stdout:(96/155): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-01-31T05:10:38.765 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/171 2024-01-31T05:10:38.824 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T05:10:38.825 INFO:teuthology.orchestra.run.smithi045.stdout:(97/155): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-01-31T05:10:38.929 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T05:10:38.959 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T05:10:39.042 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T05:10:39.042 INFO:teuthology.orchestra.run.smithi045.stdout:(98/155): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.4 MB/s | 309 kB 00:00 2024-01-31T05:10:39.072 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-31T05:10:39.149 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/171 2024-01-31T05:10:39.225 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/171 2024-01-31T05:10:39.411 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/171 2024-01-31T05:10:39.465 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/171 2024-01-31T05:10:39.493 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-31T05:10:39.596 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-31T05:10:39.619 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-31T05:10:39.681 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-31T05:10:39.775 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/171 2024-01-31T05:10:39.874 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-31T05:10:39.917 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-31T05:10:39.918 INFO:teuthology.orchestra.run.smithi045.stdout:(99/155): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-01-31T05:10:39.976 INFO:teuthology.orchestra.run.smithi045.stdout:(100/155): libarrow-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-01-31T05:10:39.986 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T05:10:40.035 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T05:10:40.043 INFO:teuthology.orchestra.run.smithi045.stdout:(101/155): liboath-2.6.7-2.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2024-01-31T05:10:40.085 INFO:teuthology.orchestra.run.smithi045.stdout:(102/155): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-01-31T05:10:40.103 INFO:teuthology.orchestra.run.smithi045.stdout:(103/155): libarrow-doc-9.0.0-10.el9.noarch.rpm 141 kB/s | 26 kB 00:00 2024-01-31T05:10:40.144 INFO:teuthology.orchestra.run.smithi045.stdout:(104/155): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-01-31T05:10:40.253 INFO:teuthology.orchestra.run.smithi045.stdout:(105/155): python3-asyncssh-2.13.2-3.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-01-31T05:10:40.286 INFO:teuthology.orchestra.run.smithi045.stdout:(106/155): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-01-31T05:10:40.306 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T05:10:40.328 INFO:teuthology.orchestra.run.smithi045.stdout:(107/155): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-01-31T05:10:40.338 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/171 2024-01-31T05:10:40.345 INFO:teuthology.orchestra.run.smithi045.stdout:(108/155): python3-cachetools-4.2.4-1.el9.noarc 551 kB/s | 32 kB 00:00 2024-01-31T05:10:40.362 INFO:teuthology.orchestra.run.smithi045.stdout:(109/155): python3-certifi-2021.10.8-2.el9.noar 447 kB/s | 15 kB 00:00 2024-01-31T05:10:40.373 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/171 2024-01-31T05:10:40.404 INFO:teuthology.orchestra.run.smithi045.stdout:(110/155): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-01-31T05:10:40.447 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/171 2024-01-31T05:10:40.481 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/171 2024-01-31T05:10:40.496 INFO:teuthology.orchestra.run.smithi045.stdout:(111/155): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-01-31T05:10:40.514 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/171 2024-01-31T05:10:40.521 INFO:teuthology.orchestra.run.smithi045.stdout:(112/155): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-01-31T05:10:40.543 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/171 2024-01-31T05:10:40.611 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/171 2024-01-31T05:10:40.663 INFO:teuthology.orchestra.run.smithi045.stdout:(113/155): python3-colorama-0.4.4-12.el9.noarch 211 kB/s | 35 kB 00:00 2024-01-31T05:10:40.666 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/171 2024-01-31T05:10:40.688 INFO:teuthology.orchestra.run.smithi045.stdout:(114/155): python3-google-auth-2.26.2-1.el9.noa 1.2 MB/s | 202 kB 00:00 2024-01-31T05:10:40.695 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/171 2024-01-31T05:10:40.705 INFO:teuthology.orchestra.run.smithi045.stdout:(115/155): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-01-31T05:10:40.722 INFO:teuthology.orchestra.run.smithi045.stdout:(116/155): python3-jaraco-classes-3.2.1-5.el9.n 526 kB/s | 18 kB 00:00 2024-01-31T05:10:40.731 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/171 2024-01-31T05:10:40.739 INFO:teuthology.orchestra.run.smithi045.stdout:(117/155): python3-jaraco-collections-3.0.0-8.e 692 kB/s | 23 kB 00:00 2024-01-31T05:10:40.756 INFO:teuthology.orchestra.run.smithi045.stdout:(118/155): python3-jaraco-functools-3.5.0-2.el9 584 kB/s | 19 kB 00:00 2024-01-31T05:10:40.773 INFO:teuthology.orchestra.run.smithi045.stdout:(119/155): python3-jaraco-text-3.2.0-6.el9.noar 586 kB/s | 20 kB 00:00 2024-01-31T05:10:40.798 INFO:teuthology.orchestra.run.smithi045.stdout:(120/155): automake-1.16.2-8.el9.noarch.rpm 92 kB/s | 689 kB 00:07 2024-01-31T05:10:40.805 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/171 2024-01-31T05:10:40.824 INFO:teuthology.orchestra.run.smithi045.stdout:(121/155): python3-logutils-0.3.5-21.el9.noarch 919 kB/s | 46 kB 00:00 2024-01-31T05:10:40.841 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 31/171 2024-01-31T05:10:40.882 INFO:teuthology.orchestra.run.smithi045.stdout:(122/155): python3-natsort-7.1.1-5.el9.noarch.r 1.9 MB/s | 58 kB 00:00 2024-01-31T05:10:40.884 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 32/171 2024-01-31T05:10:40.913 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 33/171 2024-01-31T05:10:40.958 INFO:teuthology.orchestra.run.smithi045.stdout:(123/155): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-01-31T05:10:40.983 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/171 2024-01-31T05:10:40.991 INFO:teuthology.orchestra.run.smithi045.stdout:(124/155): python3-more-itertools-8.12.0-2.el9. 410 kB/s | 79 kB 00:00 2024-01-31T05:10:41.008 INFO:teuthology.orchestra.run.smithi045.stdout:(125/155): python3-portend-3.1.0-2.el9.noarch.r 329 kB/s | 16 kB 00:00 2024-01-31T05:10:41.042 INFO:teuthology.orchestra.run.smithi045.stdout:(126/155): python3-pyperclip-1.8.2-2.el9.noarch 850 kB/s | 28 kB 00:00 2024-01-31T05:10:41.067 INFO:teuthology.orchestra.run.smithi045.stdout:(127/155): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-01-31T05:10:41.069 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/171 2024-01-31T05:10:41.092 INFO:teuthology.orchestra.run.smithi045.stdout:(128/155): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-01-31T05:10:41.127 INFO:teuthology.orchestra.run.smithi045.stdout:(129/155): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-01-31T05:10:41.151 INFO:teuthology.orchestra.run.smithi045.stdout:(130/155): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-01-31T05:10:41.177 INFO:teuthology.orchestra.run.smithi045.stdout:(131/155): python3-tempora-5.0.0-2.el9.noarch.r 707 kB/s | 36 kB 00:00 2024-01-31T05:10:41.202 INFO:teuthology.orchestra.run.smithi045.stdout:(132/155): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-01-31T05:10:41.204 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/171 2024-01-31T05:10:41.236 INFO:teuthology.orchestra.run.smithi045.stdout:(133/155): python3-kubernetes-26.1.0-2.el9.noar 2.1 MB/s | 1.0 MB 00:00 2024-01-31T05:10:41.261 INFO:teuthology.orchestra.run.smithi045.stdout:(134/155): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-01-31T05:10:41.271 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/171 2024-01-31T05:10:41.286 INFO:teuthology.orchestra.run.smithi045.stdout:(135/155): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-01-31T05:10:41.311 INFO:teuthology.orchestra.run.smithi045.stdout:(136/155): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-01-31T05:10:41.345 INFO:teuthology.orchestra.run.smithi045.stdout:(137/155): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-01-31T05:10:41.446 INFO:teuthology.orchestra.run.smithi045.stdout:(138/155): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2024-01-31T05:10:41.688 INFO:teuthology.orchestra.run.smithi045.stdout:(139/155): thrift-0.15.0-2.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-01-31T05:10:41.802 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/171 2024-01-31T05:10:41.896 INFO:teuthology.orchestra.run.smithi045.stdout:(140/155): NetworkManager-1.45.10-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-01-31T05:10:41.967 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/171 2024-01-31T05:10:41.988 INFO:teuthology.orchestra.run.smithi045.stdout:(141/155): NetworkManager-libnm-1.45.10-1.el9.x 20 MB/s | 1.8 MB 00:00 2024-01-31T05:10:42.007 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/171 2024-01-31T05:10:42.030 INFO:teuthology.orchestra.run.smithi045.stdout:(142/155): NetworkManager-team-1.45.10-1.el9.x8 1.0 MB/s | 43 kB 00:00 2024-01-31T05:10:42.047 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/171 2024-01-31T05:10:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:(143/155): NetworkManager-tui-1.45.10-1.el9.x86 4.8 MB/s | 247 kB 00:00 2024-01-31T05:10:42.085 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 42/171 2024-01-31T05:10:42.127 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/171 2024-01-31T05:10:42.130 INFO:teuthology.orchestra.run.smithi045.stdout:(144/155): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-01-31T05:10:42.172 INFO:teuthology.orchestra.run.smithi045.stdout:(145/155): elfutils-debuginfod-client-0.190-2.e 879 kB/s | 36 kB 00:00 2024-01-31T05:10:42.222 INFO:teuthology.orchestra.run.smithi045.stdout:(146/155): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2024-01-31T05:10:42.264 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/171 2024-01-31T05:10:42.273 INFO:teuthology.orchestra.run.smithi045.stdout:(147/155): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 256 kB 00:00 2024-01-31T05:10:42.310 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/171 2024-01-31T05:10:42.315 INFO:teuthology.orchestra.run.smithi045.stdout:(148/155): libacl-2.3.1-4.el9.x86_64.rpm 553 kB/s | 23 kB 00:00 2024-01-31T05:10:42.390 INFO:teuthology.orchestra.run.smithi045.stdout:(149/155): openssl-3.0.7-25.el9.x86_64.rpm 16 MB/s | 1.2 MB 00:00 2024-01-31T05:10:42.418 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/171 2024-01-31T05:10:42.447 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/171 2024-01-31T05:10:42.482 INFO:teuthology.orchestra.run.smithi045.stdout:(150/155): openssl-libs-3.0.7-25.el9.x86_64.rpm 24 MB/s | 2.2 MB 00:00 2024-01-31T05:10:42.502 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/171 2024-01-31T05:10:42.532 INFO:teuthology.orchestra.run.smithi045.stdout:(151/155): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2024-01-31T05:10:42.577 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/171 2024-01-31T05:10:42.616 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/171 2024-01-31T05:10:42.641 INFO:teuthology.orchestra.run.smithi045.stdout:(152/155): librados2-19.0.0-885.g661946e7.el9.x 2.6 MB/s | 3.4 MB 00:01 2024-01-31T05:10:42.652 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/171 2024-01-31T05:10:42.674 INFO:teuthology.orchestra.run.smithi045.stdout:(153/155): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2024-01-31T05:10:42.700 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/171 2024-01-31T05:10:42.724 INFO:teuthology.orchestra.run.smithi045.stdout:(154/155): libacl-devel-2.3.1-4.el9.x86_64.rpm 892 kB/s | 74 kB 00:00 2024-01-31T05:10:42.735 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/171 2024-01-31T05:10:42.779 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 54/171 2024-01-31T05:10:42.821 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 55/171 2024-01-31T05:10:42.957 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 56/171 2024-01-31T05:10:43.001 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 57/171 2024-01-31T05:10:43.031 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 58/171 2024-01-31T05:10:43.073 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 59/171 2024-01-31T05:10:43.185 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 60/171 2024-01-31T05:10:43.223 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 61/171 2024-01-31T05:10:43.263 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 62/171 2024-01-31T05:10:43.289 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 63/171 2024-01-31T05:10:43.315 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 64/171 2024-01-31T05:10:43.343 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 65/171 2024-01-31T05:10:43.379 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 66/171 2024-01-31T05:10:43.417 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-55.el9.x86_64 67/171 2024-01-31T05:10:43.449 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/171 2024-01-31T05:10:43.479 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/171 2024-01-31T05:10:43.513 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/171 2024-01-31T05:10:43.546 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/171 2024-01-31T05:10:43.587 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/171 2024-01-31T05:10:43.799 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/171 2024-01-31T05:10:43.825 INFO:teuthology.orchestra.run.smithi045.stdout:(155/155): librbd1-19.0.0-885.g661946e7.el9.x86 1.3 MB/s | 3.1 MB 00:02 2024-01-31T05:10:43.831 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-01-31T05:10:43.831 INFO:teuthology.orchestra.run.smithi045.stdout:Total 10 MB/s | 212 MB 00:21 2024-01-31T05:10:43.833 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/171 2024-01-31T05:10:43.950 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 75/171 2024-01-31T05:10:44.009 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 76/171 2024-01-31T05:10:44.739 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/171 2024-01-31T05:10:44.789 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 78/171 2024-01-31T05:10:44.823 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 79/171 2024-01-31T05:10:44.923 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 80/171 2024-01-31T05:10:44.975 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-01-31T05:10:44.981 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/171 2024-01-31T05:10:45.069 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 82/171 2024-01-31T05:10:45.100 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 83/171 2024-01-31T05:10:45.238 INFO:teuthology.orchestra.run.smithi186.stdout:(155/155): python3-scipy-1.6.2-8.el9.x86_64.rpm 850 kB/s | 16 MB 00:19 2024-01-31T05:10:45.244 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-01-31T05:10:45.244 INFO:teuthology.orchestra.run.smithi186.stdout:Total 5.8 MB/s | 212 MB 00:36 2024-01-31T05:10:45.277 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.21-2.el9.x86_64 84/171 2024-01-31T05:10:45.289 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-01-31T05:10:45.289 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-01-31T05:10:45.315 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 85/171 2024-01-31T05:10:45.392 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 86/171 2024-01-31T05:10:46.038 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 87/171 2024-01-31T05:10:46.155 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 88/171 2024-01-31T05:10:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-01-31T05:10:46.680 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-01-31T05:10:46.681 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-01-31T05:10:47.405 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 89/171 2024-01-31T05:10:47.412 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-01-31T05:10:47.412 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-01-31T05:10:47.503 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 90/171 2024-01-31T05:10:47.531 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 91/171 2024-01-31T05:10:47.564 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 92/171 2024-01-31T05:10:47.874 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 93/171 2024-01-31T05:10:47.911 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 94/171 2024-01-31T05:10:48.000 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 94/171 2024-01-31T05:10:48.032 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 95/171 2024-01-31T05:10:48.068 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 96/171 2024-01-31T05:10:48.394 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-01-31T05:10:48.395 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-01-31T05:10:48.498 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 97/171 2024-01-31T05:10:48.522 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 98/171 2024-01-31T05:10:48.593 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 98/171 2024-01-31T05:10:48.623 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 99/171 2024-01-31T05:10:50.735 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T05:10:50.771 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T05:10:51.283 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T05:10:51.316 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 101/171 2024-01-31T05:10:51.380 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 101/171 2024-01-31T05:10:51.380 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-31T05:10:51.380 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-31T05:10:51.380 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:10:51.409 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 102/171 2024-01-31T05:10:54.115 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-01-31T05:10:54.204 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/171 2024-01-31T05:10:54.249 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/171 2024-01-31T05:10:54.306 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/171 2024-01-31T05:10:54.346 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/171 2024-01-31T05:10:54.371 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/171 2024-01-31T05:10:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/171 2024-01-31T05:10:54.606 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-01-31T05:10:54.725 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/171 2024-01-31T05:10:54.763 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/171 2024-01-31T05:10:54.764 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/171 2024-01-31T05:10:54.814 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T05:10:54.816 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/171 2024-01-31T05:10:54.859 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/171 2024-01-31T05:10:54.884 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/171 2024-01-31T05:10:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T05:10:54.926 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/171 2024-01-31T05:10:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T05:10:55.023 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T05:10:55.048 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-31T05:10:55.133 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/171 2024-01-31T05:10:55.216 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/171 2024-01-31T05:10:55.243 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/171 2024-01-31T05:10:55.302 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T05:10:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/171 2024-01-31T05:10:55.424 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T05:10:55.445 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T05:10:55.445 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/171 2024-01-31T05:10:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-31T05:10:55.537 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T05:10:55.566 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-31T05:10:55.568 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-31T05:10:55.598 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-31T05:10:55.642 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/171 2024-01-31T05:10:55.642 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-31T05:10:55.715 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/171 2024-01-31T05:10:55.823 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/171 2024-01-31T05:10:55.889 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/171 2024-01-31T05:10:55.939 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/171 2024-01-31T05:10:55.944 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-31T05:10:55.971 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-31T05:10:56.004 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-31T05:10:56.079 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-31T05:10:56.101 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T05:10:56.101 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-31T05:10:56.138 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T05:10:56.162 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-31T05:10:56.256 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/171 2024-01-31T05:10:56.356 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-31T05:10:56.399 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T05:10:56.399 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-31T05:10:56.435 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/171 2024-01-31T05:10:56.469 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/171 2024-01-31T05:10:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T05:10:56.525 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T05:10:56.534 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/171 2024-01-31T05:10:56.576 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/171 2024-01-31T05:10:56.610 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/171 2024-01-31T05:10:56.638 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/171 2024-01-31T05:10:56.708 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/171 2024-01-31T05:10:56.769 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/171 2024-01-31T05:10:56.800 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/171 2024-01-31T05:10:56.802 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T05:10:56.829 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/171 2024-01-31T05:10:56.843 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/171 2024-01-31T05:10:56.874 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/171 2024-01-31T05:10:56.920 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/171 2024-01-31T05:10:56.939 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/171 2024-01-31T05:10:56.954 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 31/171 2024-01-31T05:10:56.974 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/171 2024-01-31T05:10:56.998 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 32/171 2024-01-31T05:10:57.026 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 33/171 2024-01-31T05:10:57.032 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/171 2024-01-31T05:10:57.060 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/171 2024-01-31T05:10:57.096 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/171 2024-01-31T05:10:57.129 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/171 2024-01-31T05:10:57.182 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/171 2024-01-31T05:10:57.182 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/171 2024-01-31T05:10:57.213 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/171 2024-01-31T05:10:57.249 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/171 2024-01-31T05:10:57.328 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/171 2024-01-31T05:10:57.332 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/171 2024-01-31T05:10:57.368 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 31/171 2024-01-31T05:10:57.385 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/171 2024-01-31T05:10:57.411 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 32/171 2024-01-31T05:10:57.440 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 33/171 2024-01-31T05:10:57.509 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/171 2024-01-31T05:10:57.585 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/171 2024-01-31T05:10:57.721 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/171 2024-01-31T05:10:57.781 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/171 2024-01-31T05:10:57.937 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/171 2024-01-31T05:10:58.031 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/171 2024-01-31T05:10:58.061 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/171 2024-01-31T05:10:58.102 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/171 2024-01-31T05:10:58.140 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 42/171 2024-01-31T05:10:58.182 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/171 2024-01-31T05:10:58.300 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/171 2024-01-31T05:10:58.328 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/171 2024-01-31T05:10:58.374 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/171 2024-01-31T05:10:58.377 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/171 2024-01-31T05:10:58.417 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/171 2024-01-31T05:10:58.466 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/171 2024-01-31T05:10:58.481 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/171 2024-01-31T05:10:58.504 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 42/171 2024-01-31T05:10:58.516 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/171 2024-01-31T05:10:58.545 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/171 2024-01-31T05:10:58.572 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/171 2024-01-31T05:10:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/171 2024-01-31T05:10:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/171 2024-01-31T05:10:58.681 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/171 2024-01-31T05:10:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/171 2024-01-31T05:10:58.721 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/171 2024-01-31T05:10:58.759 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/171 2024-01-31T05:10:58.795 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/171 2024-01-31T05:10:58.828 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/171 2024-01-31T05:10:58.846 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 54/171 2024-01-31T05:10:58.866 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/171 2024-01-31T05:10:58.889 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 55/171 2024-01-31T05:10:58.921 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/171 2024-01-31T05:10:58.985 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/171 2024-01-31T05:10:59.017 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/171 2024-01-31T05:10:59.026 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 56/171 2024-01-31T05:10:59.053 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/171 2024-01-31T05:10:59.069 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 57/171 2024-01-31T05:10:59.099 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 58/171 2024-01-31T05:10:59.100 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/171 2024-01-31T05:10:59.136 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/171 2024-01-31T05:10:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 59/171 2024-01-31T05:10:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 54/171 2024-01-31T05:10:59.222 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 55/171 2024-01-31T05:10:59.237 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 60/171 2024-01-31T05:10:59.274 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 61/171 2024-01-31T05:10:59.315 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 62/171 2024-01-31T05:10:59.347 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 63/171 2024-01-31T05:10:59.357 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 56/171 2024-01-31T05:10:59.374 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 64/171 2024-01-31T05:10:59.402 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 65/171 2024-01-31T05:10:59.412 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 57/171 2024-01-31T05:10:59.442 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 66/171 2024-01-31T05:10:59.442 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 58/171 2024-01-31T05:10:59.481 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-55.el9.x86_64 67/171 2024-01-31T05:10:59.484 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 59/171 2024-01-31T05:10:59.512 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/171 2024-01-31T05:10:59.542 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/171 2024-01-31T05:10:59.576 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/171 2024-01-31T05:10:59.580 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 60/171 2024-01-31T05:10:59.610 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/171 2024-01-31T05:10:59.617 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 61/171 2024-01-31T05:10:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/171 2024-01-31T05:10:59.657 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 62/171 2024-01-31T05:10:59.678 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/171 2024-01-31T05:10:59.683 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 63/171 2024-01-31T05:10:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/171 2024-01-31T05:10:59.710 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 64/171 2024-01-31T05:10:59.738 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 65/171 2024-01-31T05:10:59.773 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 66/171 2024-01-31T05:10:59.812 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-55.el9.x86_64 67/171 2024-01-31T05:10:59.818 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 75/171 2024-01-31T05:10:59.844 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/171 2024-01-31T05:10:59.873 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 76/171 2024-01-31T05:10:59.873 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/171 2024-01-31T05:10:59.908 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/171 2024-01-31T05:10:59.941 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/171 2024-01-31T05:10:59.983 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/171 2024-01-31T05:11:00.017 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/171 2024-01-31T05:11:00.052 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/171 2024-01-31T05:11:00.159 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 75/171 2024-01-31T05:11:00.212 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 76/171 2024-01-31T05:11:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/171 2024-01-31T05:11:00.686 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 78/171 2024-01-31T05:11:00.720 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 79/171 2024-01-31T05:11:00.822 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 80/171 2024-01-31T05:11:00.880 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/171 2024-01-31T05:11:00.942 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/171 2024-01-31T05:11:00.969 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 82/171 2024-01-31T05:11:00.992 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 78/171 2024-01-31T05:11:01.006 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 83/171 2024-01-31T05:11:01.026 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 79/171 2024-01-31T05:11:01.128 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 80/171 2024-01-31T05:11:01.187 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/171 2024-01-31T05:11:01.190 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.21-2.el9.x86_64 84/171 2024-01-31T05:11:01.220 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 85/171 2024-01-31T05:11:01.273 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 82/171 2024-01-31T05:11:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 86/171 2024-01-31T05:11:01.311 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 83/171 2024-01-31T05:11:01.489 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.21-2.el9.x86_64 84/171 2024-01-31T05:11:01.517 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 85/171 2024-01-31T05:11:01.596 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 86/171 2024-01-31T05:11:01.978 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 87/171 2024-01-31T05:11:02.089 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 88/171 2024-01-31T05:11:02.246 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 87/171 2024-01-31T05:11:02.357 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 88/171 2024-01-31T05:11:02.843 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 102/171 2024-01-31T05:11:02.843 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2024-01-31T05:11:02.844 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2024-01-31T05:11:02.844 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2024-01-31T05:11:02.844 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2024-01-31T05:11:02.844 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2024-01-31T05:11:02.844 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /root 2024-01-31T05:11:02.844 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2024-01-31T05:11:02.844 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:11:03.010 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 103/171 2024-01-31T05:11:03.027 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 103/171 2024-01-31T05:11:03.027 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:03.028 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-31T05:11:03.028 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:11:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 89/171 2024-01-31T05:11:03.518 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 90/171 2024-01-31T05:11:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 91/171 2024-01-31T05:11:03.578 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 89/171 2024-01-31T05:11:03.579 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 92/171 2024-01-31T05:11:03.647 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 90/171 2024-01-31T05:11:03.675 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 91/171 2024-01-31T05:11:03.700 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 92/171 2024-01-31T05:11:03.924 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 93/171 2024-01-31T05:11:04.007 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 93/171 2024-01-31T05:11:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 94/171 2024-01-31T05:11:04.038 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 94/171 2024-01-31T05:11:04.128 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 94/171 2024-01-31T05:11:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 94/171 2024-01-31T05:11:04.159 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 95/171 2024-01-31T05:11:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 95/171 2024-01-31T05:11:04.195 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 96/171 2024-01-31T05:11:04.217 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 96/171 2024-01-31T05:11:04.464 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 104/171 2024-01-31T05:11:04.489 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 104/171 2024-01-31T05:11:04.489 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:04.489 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-31T05:11:04.489 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-31T05:11:04.490 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-31T05:11:04.490 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:11:04.620 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 97/171 2024-01-31T05:11:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 98/171 2024-01-31T05:11:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 97/171 2024-01-31T05:11:04.689 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 105/171 2024-01-31T05:11:04.695 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 98/171 2024-01-31T05:11:04.704 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 98/171 2024-01-31T05:11:04.712 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 105/171 2024-01-31T05:11:04.712 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:04.712 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-31T05:11:04.712 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-31T05:11:04.712 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-31T05:11:04.713 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:11:04.734 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 99/171 2024-01-31T05:11:04.766 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 98/171 2024-01-31T05:11:04.788 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 99/171 2024-01-31T05:11:05.157 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-31T05:11:05.180 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-31T05:11:05.180 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:05.180 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-31T05:11:05.180 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-31T05:11:05.180 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-31T05:11:05.180 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:11:05.278 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.2-8.el9.noarch 107/171 2024-01-31T05:11:05.336 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ply-3.11-14.el9.noarch 108/171 2024-01-31T05:11:05.391 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 109/171 2024-01-31T05:11:05.577 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 110/171 2024-01-31T05:11:05.641 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 111/171 2024-01-31T05:11:05.728 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 112/171 2024-01-31T05:11:05.826 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 113/171 2024-01-31T05:11:05.966 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 114/171 2024-01-31T05:11:06.000 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 115/171 2024-01-31T05:11:06.041 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 116/171 2024-01-31T05:11:06.072 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 117/171 2024-01-31T05:11:06.107 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-16.el9.x86_64 118/171 2024-01-31T05:11:06.135 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.9-15.el9.x86_64 119/171 2024-01-31T05:11:06.525 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-31T05:11:06.551 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-31T05:11:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 121/171 2024-01-31T05:11:06.641 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 122/171 2024-01-31T05:11:06.666 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-31T05:11:06.835 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T05:11:06.873 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T05:11:06.890 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-31T05:11:06.917 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-31T05:11:06.999 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T05:11:07.036 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T05:11:07.386 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T05:11:07.418 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 101/171 2024-01-31T05:11:07.503 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 101/171 2024-01-31T05:11:07.503 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-31T05:11:07.503 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-31T05:11:07.503 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:07.539 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 102/171 2024-01-31T05:11:07.618 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T05:11:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 101/171 2024-01-31T05:11:07.747 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 101/171 2024-01-31T05:11:07.748 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-31T05:11:07.748 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-31T05:11:07.748 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:07.791 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 102/171 2024-01-31T05:11:07.876 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-31T05:11:07.900 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-31T05:11:08.001 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-31T05:11:08.163 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 126/171 2024-01-31T05:11:08.186 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-31T05:11:08.209 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-31T05:11:08.209 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:08.209 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-31T05:11:08.210 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-31T05:11:08.210 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-31T05:11:08.210 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:11:08.253 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-31T05:11:08.265 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-31T05:11:08.354 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:19.0.0-885.g661946e7.el9.x86_64 129/171 2024-01-31T05:11:12.607 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 130/171 2024-01-31T05:11:12.672 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 131/171 2024-01-31T05:11:13.448 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/171 2024-01-31T05:11:13.484 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-31T05:11:13.508 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-31T05:11:13.508 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:13.508 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-31T05:11:13.508 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-31T05:11:13.508 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-31T05:11:13.508 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:11:13.560 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-885.g661946 134/171 2024-01-31T05:11:13.583 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-885.g661946 134/171 2024-01-31T05:11:13.583 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:13.583 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-31T05:11:13.583 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:11:13.628 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-31T05:11:13.650 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-31T05:11:13.650 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:13.651 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-31T05:11:13.651 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-31T05:11:13.651 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-31T05:11:13.651 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:11:13.970 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-31T05:11:13.993 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-31T05:11:13.993 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:13.993 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-31T05:11:13.993 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-31T05:11:13.993 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-31T05:11:13.994 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:11:14.045 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 137/171 2024-01-31T05:11:14.086 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 138/171 2024-01-31T05:11:14.118 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 139/171 2024-01-31T05:11:14.145 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 140/171 2024-01-31T05:11:14.331 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 141/171 2024-01-31T05:11:14.376 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 142/171 2024-01-31T05:11:14.408 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 143/171 2024-01-31T05:11:14.448 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 144/171 2024-01-31T05:11:14.478 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 145/171 2024-01-31T05:11:14.541 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 146/171 2024-01-31T05:11:14.602 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 147/171 2024-01-31T05:11:14.636 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 148/171 2024-01-31T05:11:15.193 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T05:11:15.226 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T05:11:15.252 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T05:11:15.307 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 150/171 2024-01-31T05:11:16.150 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 151/171 2024-01-31T05:11:16.247 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 152/171 2024-01-31T05:11:16.337 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/171 2024-01-31T05:11:16.384 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/171 2024-01-31T05:11:16.409 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 155/171 2024-01-31T05:11:16.435 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 156/171 2024-01-31T05:11:16.460 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-31T05:11:16.487 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 158/171 2024-01-31T05:11:16.487 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-31T05:11:16.558 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-31T05:11:16.588 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 160/171 2024-01-31T05:11:16.613 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 161/171 2024-01-31T05:11:16.639 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 162/171 2024-01-31T05:11:16.664 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 163/171 2024-01-31T05:11:16.664 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-31T05:11:16.740 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-31T05:11:16.765 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 165/171 2024-01-31T05:11:16.771 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T05:11:16.771 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T05:11:16.951 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T05:11:16.951 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-31T05:11:17.034 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-31T05:11:17.059 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 168/171 2024-01-31T05:11:17.086 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 169/171 2024-01-31T05:11:17.127 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 170/171 2024-01-31T05:11:17.191 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-31T05:11:18.738 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 102/171 2024-01-31T05:11:18.738 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2024-01-31T05:11:18.738 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2024-01-31T05:11:18.739 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2024-01-31T05:11:18.739 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2024-01-31T05:11:18.739 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2024-01-31T05:11:18.739 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /root 2024-01-31T05:11:18.739 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2024-01-31T05:11:18.739 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:18.903 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 103/171 2024-01-31T05:11:18.918 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 103/171 2024-01-31T05:11:18.918 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:18.918 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-31T05:11:18.919 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:19.702 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 102/171 2024-01-31T05:11:19.703 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-01-31T05:11:19.703 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-01-31T05:11:19.703 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-01-31T05:11:19.703 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-01-31T05:11:19.703 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-01-31T05:11:19.703 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-01-31T05:11:19.703 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-01-31T05:11:19.703 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:19.871 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 103/171 2024-01-31T05:11:19.886 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 103/171 2024-01-31T05:11:19.886 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:19.886 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-31T05:11:19.886 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:20.347 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 104/171 2024-01-31T05:11:20.373 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 104/171 2024-01-31T05:11:20.373 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:20.373 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-31T05:11:20.373 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-31T05:11:20.373 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-31T05:11:20.373 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:20.573 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 105/171 2024-01-31T05:11:20.595 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 105/171 2024-01-31T05:11:20.595 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:20.595 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-31T05:11:20.595 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-31T05:11:20.596 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-31T05:11:20.596 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:21.040 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-31T05:11:21.062 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-31T05:11:21.062 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:21.062 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-31T05:11:21.062 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-31T05:11:21.062 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-31T05:11:21.062 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:21.170 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.2-8.el9.noarch 107/171 2024-01-31T05:11:21.228 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ply-3.11-14.el9.noarch 108/171 2024-01-31T05:11:21.282 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 109/171 2024-01-31T05:11:21.425 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 104/171 2024-01-31T05:11:21.451 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 104/171 2024-01-31T05:11:21.451 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:21.451 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-31T05:11:21.451 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-31T05:11:21.451 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-31T05:11:21.451 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:21.477 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 110/171 2024-01-31T05:11:21.524 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 111/171 2024-01-31T05:11:21.603 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 112/171 2024-01-31T05:11:21.675 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 105/171 2024-01-31T05:11:21.692 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 113/171 2024-01-31T05:11:21.698 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 105/171 2024-01-31T05:11:21.698 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:21.698 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-31T05:11:21.698 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-31T05:11:21.698 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-31T05:11:21.698 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:21.834 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 114/171 2024-01-31T05:11:21.882 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 115/171 2024-01-31T05:11:21.915 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 116/171 2024-01-31T05:11:21.947 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 117/171 2024-01-31T05:11:21.982 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : jq-1.6-16.el9.x86_64 118/171 2024-01-31T05:11:22.010 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.9-15.el9.x86_64 119/171 2024-01-31T05:11:22.168 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-31T05:11:22.191 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-31T05:11:22.191 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:22.191 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-31T05:11:22.191 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-31T05:11:22.191 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-31T05:11:22.191 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.2-8.el9.noarch 107/171 2024-01-31T05:11:22.315 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-31T05:11:22.316 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:19.0.0-885.g661946e7.el9.x86_64 1/171 2024-01-31T05:11:22.316 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2/171 2024-01-31T05:11:22.316 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 3/171 2024-01-31T05:11:22.316 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 4/171 2024-01-31T05:11:22.316 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-885.g661946 5/171 2024-01-31T05:11:22.316 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 6/171 2024-01-31T05:11:22.316 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 7/171 2024-01-31T05:11:22.316 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T05:11:22.316 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T05:11:22.316 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-31T05:11:22.316 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 11/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 12/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 13/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 14/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 15/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 16/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 17/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 18/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 19/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 20/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 21/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 22/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 23/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 24/171 2024-01-31T05:11:22.317 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 25/171 2024-01-31T05:11:22.331 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 26/171 2024-01-31T05:11:22.331 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 27/171 2024-01-31T05:11:22.331 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 28/171 2024-01-31T05:11:22.331 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 29/171 2024-01-31T05:11:22.331 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 30/171 2024-01-31T05:11:22.331 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 31/171 2024-01-31T05:11:22.331 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 32/171 2024-01-31T05:11:22.331 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 33/171 2024-01-31T05:11:22.331 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 34/171 2024-01-31T05:11:22.331 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 35/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 36/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 37/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:19.0.0-885.g661946e7.el9.noarch 38/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 39/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-16.el9.x86_64 42/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/171 2024-01-31T05:11:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/171 2024-01-31T05:11:22.333 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/171 2024-01-31T05:11:22.334 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 77/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 78/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 87/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 88/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 89/171 2024-01-31T05:11:22.335 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 90/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 91/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 92/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 93/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 94/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 95/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 96/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 97/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 98/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 99/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 100/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 101/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 102/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 103/171 2024-01-31T05:11:22.336 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 104/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 105/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 106/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 107/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 108/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 109/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 110/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 111/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 112/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 113/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 114/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 115/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 116/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 117/171 2024-01-31T05:11:22.337 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 118/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 119/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 120/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 121/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 122/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 123/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 124/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 125/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 126/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 127/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 128/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 129/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 130/171 2024-01-31T05:11:22.338 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 131/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 132/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 133/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 134/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 135/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 136/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 137/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : re2-1:20211101-3.el9.x86_64 138/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 139/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:19.0.0-885.g661946e7.el9.x86_64 140/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 141/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 142/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 143/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 144/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 145/171 2024-01-31T05:11:22.339 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 146/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 147/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 148/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 149/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 150/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 151/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : acl-2.3.1-4.el9.x86_64 152/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : acl-2.3.1-3.el9.x86_64 153/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 154/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 155/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 156/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 157/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 158/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 159/171 2024-01-31T05:11:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 160/171 2024-01-31T05:11:22.341 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 161/171 2024-01-31T05:11:22.341 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 162/171 2024-01-31T05:11:22.341 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 163/171 2024-01-31T05:11:22.341 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 164/171 2024-01-31T05:11:22.341 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 165/171 2024-01-31T05:11:22.341 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 166/171 2024-01-31T05:11:22.341 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 167/171 2024-01-31T05:11:22.341 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 168/171 2024-01-31T05:11:22.341 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 169/171 2024-01-31T05:11:22.341 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 170/171 2024-01-31T05:11:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 108/171 2024-01-31T05:11:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 109/171 2024-01-31T05:11:22.448 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-31T05:11:22.476 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-31T05:11:22.506 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 121/171 2024-01-31T05:11:22.567 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 122/171 2024-01-31T05:11:22.590 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-31T05:11:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 110/171 2024-01-31T05:11:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 111/171 2024-01-31T05:11:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 112/171 2024-01-31T05:11:22.813 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-31T05:11:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 113/171 2024-01-31T05:11:22.842 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-31T05:11:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 114/171 2024-01-31T05:11:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 115/171 2024-01-31T05:11:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 116/171 2024-01-31T05:11:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 117/171 2024-01-31T05:11:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : jq-1.6-16.el9.x86_64 118/171 2024-01-31T05:11:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-15.el9.x86_64 119/171 2024-01-31T05:11:23.502 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-31T05:11:23.529 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-31T05:11:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 121/171 2024-01-31T05:11:23.630 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 122/171 2024-01-31T05:11:23.661 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-31T05:11:23.799 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-31T05:11:23.825 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-31T05:11:23.893 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-31T05:11:23.925 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-31T05:11:23.928 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-31T05:11:24.087 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 126/171 2024-01-31T05:11:24.119 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-31T05:11:24.142 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-31T05:11:24.142 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:24.142 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-31T05:11:24.142 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-31T05:11:24.142 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-31T05:11:24.142 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:24.186 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-31T05:11:24.198 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-31T05:11:24.294 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:19.0.0-885.g661946e7.el9.x86_64 129/171 2024-01-31T05:11:24.941 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-31T05:11:24.962 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-31T05:11:25.066 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-31T05:11:25.177 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 171/171 2024-01-31T05:11:25.177 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:11:25.177 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2024-01-31T05:11:25.177 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: acl-2.3.1-4.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-31T05:11:25.178 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.2-8.el9.noarch 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.7.4-5.el9.x86_64 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:25.179 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.4-9.el9.x86_64 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-16.el9.x86_64 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.181 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-31T05:11:25.182 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: lua-5.4.4-4.el9.x86_64 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-13.11-1.el9.x86_64 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-31T05:11:25.183 INFO:teuthology.orchestra.run.smithi156.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-31T05:11:25.184 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-31T05:11:25.185 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-31T05:11:25.185 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-01-31T05:11:25.185 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-31T05:11:25.185 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-31T05:11:25.185 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-31T05:11:25.185 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-31T05:11:25.185 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-31T05:11:25.185 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-31T05:11:25.185 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-31T05:11:25.185 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-31T05:11:25.185 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-31T05:11:25.185 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply-3.11-14.el9.noarch 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-31T05:11:25.186 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-2.25.1-8.el9.noarch 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-31T05:11:25.187 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: re2-1:20211101-3.el9.x86_64 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-31T05:11:25.188 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-31T05:11:25.189 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-31T05:11:25.189 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-01-31T05:11:25.189 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-01-31T05:11:25.231 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 126/171 2024-01-31T05:11:25.264 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-31T05:11:25.287 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-31T05:11:25.287 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:25.287 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-31T05:11:25.288 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-31T05:11:25.288 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-31T05:11:25.288 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-31T05:11:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-31T05:11:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.0.0-885.g661946e7.el9.x86_64 129/171 2024-01-31T05:11:25.647 DEBUG:teuthology.parallel:result is None 2024-01-31T05:11:28.554 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 130/171 2024-01-31T05:11:28.613 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 131/171 2024-01-31T05:11:29.390 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/171 2024-01-31T05:11:29.434 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-31T05:11:29.457 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-31T05:11:29.457 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:29.457 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-31T05:11:29.457 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-31T05:11:29.457 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-31T05:11:29.457 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:29.502 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-885.g661946 134/171 2024-01-31T05:11:29.523 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-885.g661946 134/171 2024-01-31T05:11:29.524 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:29.524 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-31T05:11:29.524 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:29.569 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-31T05:11:29.592 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-31T05:11:29.592 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:29.592 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-31T05:11:29.592 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-31T05:11:29.592 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-31T05:11:29.592 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:29.895 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-31T05:11:29.917 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-31T05:11:29.917 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:29.917 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-31T05:11:29.917 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-31T05:11:29.917 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-31T05:11:29.918 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:29.969 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 137/171 2024-01-31T05:11:29.984 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 130/171 2024-01-31T05:11:30.011 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 138/171 2024-01-31T05:11:30.043 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 139/171 2024-01-31T05:11:30.048 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 131/171 2024-01-31T05:11:30.070 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 140/171 2024-01-31T05:11:30.250 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 141/171 2024-01-31T05:11:30.300 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 142/171 2024-01-31T05:11:30.324 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 143/171 2024-01-31T05:11:30.365 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 144/171 2024-01-31T05:11:30.395 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 145/171 2024-01-31T05:11:30.449 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 146/171 2024-01-31T05:11:30.502 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 147/171 2024-01-31T05:11:30.526 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 148/171 2024-01-31T05:11:30.875 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/171 2024-01-31T05:11:30.906 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-31T05:11:30.929 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-31T05:11:30.929 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:30.930 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-31T05:11:30.930 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-31T05:11:30.930 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-31T05:11:30.930 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-885.g661946 134/171 2024-01-31T05:11:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-885.g661946 134/171 2024-01-31T05:11:30.995 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:30.995 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-31T05:11:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-31T05:11:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-31T05:11:31.063 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:31.063 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-31T05:11:31.063 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-31T05:11:31.063 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-31T05:11:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:31.145 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T05:11:31.192 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T05:11:31.218 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T05:11:31.273 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 150/171 2024-01-31T05:11:31.374 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-31T05:11:31.397 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-31T05:11:31.397 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T05:11:31.397 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-31T05:11:31.397 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-31T05:11:31.397 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-31T05:11:31.397 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:31.450 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 137/171 2024-01-31T05:11:31.491 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 138/171 2024-01-31T05:11:31.523 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 139/171 2024-01-31T05:11:31.550 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 140/171 2024-01-31T05:11:31.749 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 141/171 2024-01-31T05:11:31.814 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 142/171 2024-01-31T05:11:31.846 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 143/171 2024-01-31T05:11:31.887 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 144/171 2024-01-31T05:11:31.916 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 145/171 2024-01-31T05:11:31.984 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 146/171 2024-01-31T05:11:32.066 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 147/171 2024-01-31T05:11:32.105 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 151/171 2024-01-31T05:11:32.107 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 148/171 2024-01-31T05:11:32.205 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 152/171 2024-01-31T05:11:32.285 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/171 2024-01-31T05:11:32.342 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/171 2024-01-31T05:11:32.367 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 155/171 2024-01-31T05:11:32.392 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 156/171 2024-01-31T05:11:32.417 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-31T05:11:32.444 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 158/171 2024-01-31T05:11:32.444 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-31T05:11:32.532 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-31T05:11:32.570 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 160/171 2024-01-31T05:11:32.595 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 161/171 2024-01-31T05:11:32.621 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 162/171 2024-01-31T05:11:32.638 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 163/171 2024-01-31T05:11:32.638 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-31T05:11:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T05:11:32.730 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-31T05:11:32.755 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 165/171 2024-01-31T05:11:32.761 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T05:11:32.762 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T05:11:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T05:11:32.817 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T05:11:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 150/171 2024-01-31T05:11:32.950 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T05:11:32.950 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-31T05:11:33.042 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-31T05:11:33.066 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 168/171 2024-01-31T05:11:33.093 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 169/171 2024-01-31T05:11:33.134 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 170/171 2024-01-31T05:11:33.198 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-31T05:11:33.738 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 151/171 2024-01-31T05:11:33.831 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 152/171 2024-01-31T05:11:33.945 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/171 2024-01-31T05:11:33.975 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/171 2024-01-31T05:11:33.998 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 155/171 2024-01-31T05:11:34.024 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 156/171 2024-01-31T05:11:34.049 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-31T05:11:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 158/171 2024-01-31T05:11:34.076 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-31T05:11:34.164 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-31T05:11:34.204 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 160/171 2024-01-31T05:11:34.236 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 161/171 2024-01-31T05:11:34.262 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 162/171 2024-01-31T05:11:34.287 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 163/171 2024-01-31T05:11:34.287 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-31T05:11:34.371 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-31T05:11:34.396 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 165/171 2024-01-31T05:11:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T05:11:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T05:11:34.590 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T05:11:34.591 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-31T05:11:34.658 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-31T05:11:34.682 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 168/171 2024-01-31T05:11:34.709 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 169/171 2024-01-31T05:11:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 170/171 2024-01-31T05:11:34.810 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-31T05:11:38.130 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-31T05:11:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:19.0.0-885.g661946e7.el9.x86_64 1/171 2024-01-31T05:11:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2/171 2024-01-31T05:11:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 3/171 2024-01-31T05:11:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 4/171 2024-01-31T05:11:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-885.g661946 5/171 2024-01-31T05:11:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 6/171 2024-01-31T05:11:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 7/171 2024-01-31T05:11:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T05:11:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T05:11:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-31T05:11:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 11/171 2024-01-31T05:11:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 12/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 13/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 14/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 15/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 16/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 17/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 18/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 19/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 20/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 21/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 22/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 23/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 24/171 2024-01-31T05:11:38.132 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 25/171 2024-01-31T05:11:38.133 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 26/171 2024-01-31T05:11:38.133 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 27/171 2024-01-31T05:11:38.133 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 28/171 2024-01-31T05:11:38.146 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 29/171 2024-01-31T05:11:38.146 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 30/171 2024-01-31T05:11:38.146 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 31/171 2024-01-31T05:11:38.146 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 32/171 2024-01-31T05:11:38.146 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 33/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 34/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 35/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 36/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 37/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:19.0.0-885.g661946e7.el9.noarch 38/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 39/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : jq-1.6-16.el9.x86_64 42/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/171 2024-01-31T05:11:38.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/171 2024-01-31T05:11:38.148 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/171 2024-01-31T05:11:38.149 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/171 2024-01-31T05:11:38.149 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/171 2024-01-31T05:11:38.149 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/171 2024-01-31T05:11:38.149 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/171 2024-01-31T05:11:38.149 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/171 2024-01-31T05:11:38.149 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/171 2024-01-31T05:11:38.149 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/171 2024-01-31T05:11:38.149 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/171 2024-01-31T05:11:38.149 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/171 2024-01-31T05:11:38.149 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/171 2024-01-31T05:11:38.149 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/171 2024-01-31T05:11:38.149 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 77/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 78/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/171 2024-01-31T05:11:38.150 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 87/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 88/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 89/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 90/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 91/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 92/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 93/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 94/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 95/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 96/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 97/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 98/171 2024-01-31T05:11:38.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 99/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 100/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 101/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 102/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 103/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 104/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 105/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 106/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 107/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 108/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 109/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 110/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 111/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 112/171 2024-01-31T05:11:38.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 113/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 114/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 115/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 116/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 117/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 118/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 119/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 120/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 121/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 122/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 123/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 124/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 125/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 126/171 2024-01-31T05:11:38.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 127/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 128/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 129/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 130/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 131/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 132/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 133/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 134/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 135/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 136/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 137/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : re2-1:20211101-3.el9.x86_64 138/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 139/171 2024-01-31T05:11:38.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:19.0.0-885.g661946e7.el9.x86_64 140/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 141/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 142/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 143/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 144/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 145/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 146/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 147/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 148/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 149/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 150/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 151/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : acl-2.3.1-4.el9.x86_64 152/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : acl-2.3.1-3.el9.x86_64 153/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 154/171 2024-01-31T05:11:38.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 155/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 156/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 157/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 158/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 159/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 160/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 161/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 162/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 163/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 164/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 165/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 166/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 167/171 2024-01-31T05:11:38.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 168/171 2024-01-31T05:11:38.157 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 169/171 2024-01-31T05:11:38.157 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 170/171 2024-01-31T05:11:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-31T05:11:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.0.0-885.g661946e7.el9.x86_64 1/171 2024-01-31T05:11:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2/171 2024-01-31T05:11:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 3/171 2024-01-31T05:11:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 4/171 2024-01-31T05:11:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-885.g661946 5/171 2024-01-31T05:11:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 6/171 2024-01-31T05:11:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 7/171 2024-01-31T05:11:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T05:11:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T05:11:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-31T05:11:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 11/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 12/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 13/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 14/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 15/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 16/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 17/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 18/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 19/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 20/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 21/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 22/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 23/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 24/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 25/171 2024-01-31T05:11:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 26/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 27/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 28/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 29/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 30/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 31/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 32/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 33/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 34/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 35/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 36/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 37/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.0.0-885.g661946e7.el9.noarch 38/171 2024-01-31T05:11:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 39/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : jq-1.6-16.el9.x86_64 42/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/171 2024-01-31T05:11:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/171 2024-01-31T05:11:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 77/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 78/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/171 2024-01-31T05:11:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 87/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 88/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 89/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 90/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 91/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 92/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 93/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 94/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 95/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 96/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 97/171 2024-01-31T05:11:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 98/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 99/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 100/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 101/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 102/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 103/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 104/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 105/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 106/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 107/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 108/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 109/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 110/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 111/171 2024-01-31T05:11:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 112/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 113/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 114/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 115/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 116/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 117/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 118/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 119/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 120/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 121/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 122/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 123/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 124/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 125/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 126/171 2024-01-31T05:11:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 127/171 2024-01-31T05:11:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 128/171 2024-01-31T05:11:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 129/171 2024-01-31T05:11:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 130/171 2024-01-31T05:11:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 131/171 2024-01-31T05:11:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 132/171 2024-01-31T05:11:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 133/171 2024-01-31T05:11:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 134/171 2024-01-31T05:11:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 135/171 2024-01-31T05:11:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 136/171 2024-01-31T05:11:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 137/171 2024-01-31T05:11:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-3.el9.x86_64 138/171 2024-01-31T05:11:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 139/171 2024-01-31T05:11:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.0.0-885.g661946e7.el9.x86_64 140/171 2024-01-31T05:11:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 141/171 2024-01-31T05:11:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 142/171 2024-01-31T05:11:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 143/171 2024-01-31T05:11:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 144/171 2024-01-31T05:11:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 145/171 2024-01-31T05:11:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 146/171 2024-01-31T05:11:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 147/171 2024-01-31T05:11:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 148/171 2024-01-31T05:11:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 149/171 2024-01-31T05:11:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 150/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 151/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : acl-2.3.1-4.el9.x86_64 152/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : acl-2.3.1-3.el9.x86_64 153/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 154/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 155/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 156/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 157/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 158/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 159/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 160/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 161/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 162/171 2024-01-31T05:11:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 163/171 2024-01-31T05:11:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 164/171 2024-01-31T05:11:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 165/171 2024-01-31T05:11:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 166/171 2024-01-31T05:11:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 167/171 2024-01-31T05:11:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 168/171 2024-01-31T05:11:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 169/171 2024-01-31T05:11:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 170/171 2024-01-31T05:11:40.952 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 171/171 2024-01-31T05:11:40.952 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:40.952 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2024-01-31T05:11:40.952 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-31T05:11:40.952 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-31T05:11:40.952 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-31T05:11:40.952 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-31T05:11:40.952 INFO:teuthology.orchestra.run.smithi186.stdout: acl-2.3.1-4.el9.x86_64 2024-01-31T05:11:40.953 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-31T05:11:40.953 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-31T05:11:40.953 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-31T05:11:40.953 INFO:teuthology.orchestra.run.smithi186.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-31T05:11:40.953 INFO:teuthology.orchestra.run.smithi186.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-31T05:11:40.953 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.954 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.954 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-31T05:11:40.954 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-31T05:11:40.955 INFO:teuthology.orchestra.run.smithi186.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-31T05:11:40.955 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-31T05:11:40.955 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-01-31T05:11:40.955 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.2-8.el9.noarch 2024-01-31T05:11:40.955 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.7.4-5.el9.x86_64 2024-01-31T05:11:40.955 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-31T05:11:40.955 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.955 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.956 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.956 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.956 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:40.956 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.956 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.956 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.956 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:40.956 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:40.956 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:40.956 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-31T05:11:40.957 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.4-9.el9.x86_64 2024-01-31T05:11:40.958 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-31T05:11:40.958 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-31T05:11:40.958 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-31T05:11:40.958 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-31T05:11:40.958 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-31T05:11:40.958 INFO:teuthology.orchestra.run.smithi186.stdout: jq-1.6-16.el9.x86_64 2024-01-31T05:11:40.958 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-31T05:11:40.958 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-31T05:11:40.958 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-31T05:11:40.958 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.958 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.959 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.959 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-31T05:11:40.959 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-31T05:11:40.959 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-31T05:11:40.959 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-31T05:11:40.959 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-31T05:11:40.959 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.959 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.959 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-31T05:11:40.959 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.959 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-31T05:11:40.959 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-31T05:11:40.960 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-31T05:11:40.960 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-31T05:11:40.960 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-31T05:11:40.960 INFO:teuthology.orchestra.run.smithi186.stdout: lua-5.4.4-4.el9.x86_64 2024-01-31T05:11:40.960 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-31T05:11:40.960 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-31T05:11:40.960 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-31T05:11:40.960 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-31T05:11:40.960 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-31T05:11:40.960 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-31T05:11:40.960 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-31T05:11:40.960 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-13.11-1.el9.x86_64 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-31T05:11:40.961 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-31T05:11:40.962 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-31T05:11:40.963 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-31T05:11:40.963 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-31T05:11:40.963 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-31T05:11:40.963 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-31T05:11:40.963 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-31T05:11:40.963 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-31T05:11:40.963 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-31T05:11:40.963 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-31T05:11:40.963 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-31T05:11:40.963 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-31T05:11:40.963 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-31T05:11:40.963 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply-3.11-14.el9.noarch 2024-01-31T05:11:40.964 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-31T05:11:40.964 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-31T05:11:40.964 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-31T05:11:40.964 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-31T05:11:40.964 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-31T05:11:40.964 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-31T05:11:40.964 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-31T05:11:40.964 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-31T05:11:40.964 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-2.25.1-8.el9.noarch 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-31T05:11:40.965 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-31T05:11:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-31T05:11:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-31T05:11:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-31T05:11:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: re2-1:20211101-3.el9.x86_64 2024-01-31T05:11:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-31T05:11:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-31T05:11:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-31T05:11:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-31T05:11:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-31T05:11:40.967 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-31T05:11:40.967 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:11:40.967 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-01-31T05:11:41.417 DEBUG:teuthology.parallel:result is None 2024-01-31T05:11:42.400 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 171/171 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: acl-2.3.1-4.el9.x86_64 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-31T05:11:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.2-8.el9.noarch 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.7.4-5.el9.x86_64 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.4-9.el9.x86_64 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-31T05:11:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: jq-1.6-16.el9.x86_64 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-31T05:11:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-31T05:11:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-13.11-1.el9.x86_64 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-31T05:11:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-31T05:11:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-31T05:11:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-31T05:11:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-31T05:11:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-31T05:11:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-31T05:11:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-31T05:11:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-01-31T05:11:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-31T05:11:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-31T05:11:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-31T05:11:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-31T05:11:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-31T05:11:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-31T05:11:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T05:11:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-3.el9.x86_64 2024-01-31T05:11:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-31T05:11:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-31T05:11:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-31T05:11:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-31T05:11:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-31T05:11:42.413 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-31T05:11:42.413 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:11:42.413 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-01-31T05:11:42.994 DEBUG:teuthology.parallel:result is None 2024-01-31T05:11:42.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:11:43.216 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-31T05:11:43.237 INFO:teuthology.orchestra.run.smithi045.stdout:19.0.0-885.g661946e7.el9 2024-01-31T05:11:43.238 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-885.g661946e7.el9 2024-01-31T05:11:43.238 INFO:teuthology.task.install:The correct ceph version 19.0.0-885.g661946e7 is installed. 2024-01-31T05:11:43.240 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:11:43.382 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-31T05:11:43.409 INFO:teuthology.orchestra.run.smithi156.stdout:19.0.0-885.g661946e7.el9 2024-01-31T05:11:43.409 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-885.g661946e7.el9 2024-01-31T05:11:43.410 INFO:teuthology.task.install:The correct ceph version 19.0.0-885.g661946e7 is installed. 2024-01-31T05:11:43.411 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:11:43.632 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-31T05:11:43.658 INFO:teuthology.orchestra.run.smithi186.stdout:19.0.0-885.g661946e7.el9 2024-01-31T05:11:43.658 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-885.g661946e7.el9 2024-01-31T05:11:43.658 INFO:teuthology.task.install:The correct ceph version 19.0.0-885.g661946e7 is installed. 2024-01-31T05:11:43.661 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-31T05:11:43.662 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:11:43.662 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-31T05:11:43.694 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-01-31T05:11:43.694 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-31T05:11:43.728 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-01-31T05:11:43.728 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-31T05:11:43.762 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-31T05:11:43.763 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:11:43.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-01-31T05:11:43.792 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-31T05:11:43.860 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-01-31T05:11:43.860 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2024-01-31T05:11:43.891 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-31T05:11:43.962 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-01-31T05:11:43.962 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2024-01-31T05:11:43.993 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-31T05:11:44.064 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-31T05:11:44.064 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:11:44.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-31T05:11:44.093 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-31T05:11:44.162 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-01-31T05:11:44.162 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-31T05:11:44.193 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-31T05:11:44.263 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-01-31T05:11:44.263 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-31T05:11:44.295 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-31T05:11:44.367 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-31T05:11:44.367 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:11:44.368 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-01-31T05:11:44.398 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-31T05:11:44.468 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-01-31T05:11:44.502 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2024-01-31T05:11:44.531 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-31T05:11:44.601 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-01-31T05:11:44.601 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2024-01-31T05:11:44.633 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-31T05:11:44.703 INFO:teuthology.run_tasks:Running task cephadm... 2024-01-31T05:11:44.801 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-01-31T05:11:44.801 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:11:44.801 INFO:tasks.cephadm:Cluster fsid is 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:11:44.802 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-01-31T05:11:44.802 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.45', 'mon.b': '172.21.15.156', 'mon.c': '172.21.15.186'} 2024-01-31T05:11:44.802 INFO:tasks.cephadm:First mon is mon.a on smithi045 2024-01-31T05:11:44.802 INFO:tasks.cephadm:First mgr is x 2024-01-31T05:11:44.802 INFO:tasks.cephadm:Normalizing hostnames... 2024-01-31T05:11:44.802 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2024-01-31T05:11:44.834 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname $(hostname -s) 2024-01-31T05:11:44.865 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname $(hostname -s) 2024-01-31T05:11:44.895 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-01-31T05:11:44.896 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:11:45.110 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76233/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-885-g661946e7', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-885.g661946e7'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-01-25 07:26:33.077331', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240125.044502', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-01-31T05:11:45.322 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-vshankar-testing-20240125.044502, sha1 661946e7044723130c03627323f6a879a9848cb5 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:11:45.323 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/x86_64/flavors/default/cephadm 2024-01-31T05:11:45.324 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-01-31T05:11:45.911 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r--. 1 ubuntu ubuntu 760779 Jan 31 05:11 /home/ubuntu/cephtest/cephadm 2024-01-31T05:11:45.911 DEBUG:teuthology.orchestra.run.smithi156:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-01-31T05:11:46.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r--. 1 ubuntu ubuntu 760779 Jan 31 05:11 /home/ubuntu/cephtest/cephadm 2024-01-31T05:11:46.316 DEBUG:teuthology.orchestra.run.smithi186:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-01-31T05:11:46.868 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r--. 1 ubuntu ubuntu 760779 Jan 31 05:11 /home/ubuntu/cephtest/cephadm 2024-01-31T05:11:46.869 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-01-31T05:11:46.887 DEBUG:teuthology.orchestra.run.smithi156:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-01-31T05:11:46.906 DEBUG:teuthology.orchestra.run.smithi186:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-01-31T05:11:46.946 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 on all hosts... 2024-01-31T05:11:46.946 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 pull 2024-01-31T05:11:46.949 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 pull 2024-01-31T05:11:46.951 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 pull 2024-01-31T05:11:47.749 INFO:teuthology.orchestra.run.smithi186.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5... 2024-01-31T05:11:47.798 INFO:teuthology.orchestra.run.smithi156.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5... 2024-01-31T05:11:47.826 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5... 2024-01-31T05:12:30.596 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-01-31T05:12:30.597 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:12:30.597 INFO:teuthology.orchestra.run.smithi156.stdout: "image_id": "0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775", 2024-01-31T05:12:30.597 INFO:teuthology.orchestra.run.smithi156.stdout: "repo_digests": [ 2024-01-31T05:12:30.597 INFO:teuthology.orchestra.run.smithi156.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06" 2024-01-31T05:12:30.597 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2024-01-31T05:12:30.598 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-01-31T05:12:31.123 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-01-31T05:12:31.123 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:12:31.124 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775", 2024-01-31T05:12:31.124 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2024-01-31T05:12:31.124 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06" 2024-01-31T05:12:31.124 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-01-31T05:12:31.124 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-01-31T05:12:31.514 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-01-31T05:12:31.514 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:12:31.514 INFO:teuthology.orchestra.run.smithi186.stdout: "image_id": "0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775", 2024-01-31T05:12:31.514 INFO:teuthology.orchestra.run.smithi186.stdout: "repo_digests": [ 2024-01-31T05:12:31.514 INFO:teuthology.orchestra.run.smithi186.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06" 2024-01-31T05:12:31.515 INFO:teuthology.orchestra.run.smithi186.stdout: ] 2024-01-31T05:12:31.515 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-01-31T05:12:31.539 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2024-01-31T05:12:31.571 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph 2024-01-31T05:12:31.602 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph 2024-01-31T05:12:31.632 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2024-01-31T05:12:31.660 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /etc/ceph 2024-01-31T05:12:31.688 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /etc/ceph 2024-01-31T05:12:31.715 INFO:tasks.cephadm:Writing seed config... 2024-01-31T05:12:31.716 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-01-31T05:12:31.716 INFO:tasks.cephadm: override: [client] debug client = 20 2024-01-31T05:12:31.716 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-01-31T05:12:31.716 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-01-31T05:12:31.716 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-01-31T05:12:31.716 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-01-31T05:12:31.717 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-01-31T05:12:31.718 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-01-31T05:12:31.719 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-01-31T05:12:31.720 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-01-31T05:12:31.721 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:12:31.721 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-01-31T05:12:31.736 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3a1f4122-bff7-11ee-95b3-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-01-31T05:12:31.737 DEBUG:teuthology.orchestra.run.smithi045:mon.a> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.a.service 2024-01-31T05:12:31.779 DEBUG:teuthology.orchestra.run.smithi045:mgr.x> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.x.service 2024-01-31T05:12:31.822 INFO:tasks.cephadm:Bootstrapping... 2024-01-31T05:12:31.822 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 -v bootstrap --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.45 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-01-31T05:12:32.019 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-01-31T05:12:32.019 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5', '-v', 'bootstrap', '--fsid', '3a1f4122-bff7-11ee-95b3-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.45', '--skip-admin-label'] 2024-01-31T05:12:32.043 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.8.1 2024-01-31T05:12:32.044 INFO:teuthology.orchestra.run.smithi045.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-01-31T05:12:32.044 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying podman|docker is present... 2024-01-31T05:12:32.066 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.8.1 2024-01-31T05:12:32.067 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying lvm2 is present... 2024-01-31T05:12:32.067 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying time synchronization is in place... 2024-01-31T05:12:32.074 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-01-31T05:12:32.074 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-01-31T05:12:32.080 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-01-31T05:12:32.080 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-01-31T05:12:32.086 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-01-31T05:12:32.092 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-01-31T05:12:32.092 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-01-31T05:12:32.092 INFO:teuthology.orchestra.run.smithi045.stdout:Repeating the final host check... 2024-01-31T05:12:32.113 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.8.1 2024-01-31T05:12:32.114 INFO:teuthology.orchestra.run.smithi045.stdout:podman (/bin/podman) version 4.8.1 is present 2024-01-31T05:12:32.114 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl is present 2024-01-31T05:12:32.114 INFO:teuthology.orchestra.run.smithi045.stdout:lvcreate is present 2024-01-31T05:12:32.120 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-01-31T05:12:32.120 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-01-31T05:12:32.125 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-01-31T05:12:32.125 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-01-31T05:12:32.131 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-01-31T05:12:32.138 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-01-31T05:12:32.138 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-01-31T05:12:32.138 INFO:teuthology.orchestra.run.smithi045.stdout:Host looks OK 2024-01-31T05:12:32.138 INFO:teuthology.orchestra.run.smithi045.stdout:Cluster fsid: 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:12:32.138 INFO:teuthology.orchestra.run.smithi045.stdout:Acquiring lock 140301897947360 on /run/cephadm/3a1f4122-bff7-11ee-95b3-87774f69a715.lock 2024-01-31T05:12:32.138 INFO:teuthology.orchestra.run.smithi045.stdout:Lock 140301897947360 acquired on /run/cephadm/3a1f4122-bff7-11ee-95b3-87774f69a715.lock 2024-01-31T05:12:32.138 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 3300 ... 2024-01-31T05:12:32.139 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 6789 ... 2024-01-31T05:12:32.139 INFO:teuthology.orchestra.run.smithi045.stdout:Base mon IP(s) is [172.21.15.45:3300, 172.21.15.45:6789], mon addrv is [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-01-31T05:12:32.142 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.45 metric 100 2024-01-31T05:12:32.142 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.45 metric 100 2024-01-31T05:12:32.144 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-01-31T05:12:32.144 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-01-31T05:12:32.144 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-01-31T05:12:32.147 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-01-31T05:12:32.147 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-01-31T05:12:32.147 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-01-31T05:12:32.147 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-01-31T05:12:32.147 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-01-31T05:12:32.147 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-01-31T05:12:32.148 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-01-31T05:12:32.148 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-01-31T05:12:32.148 INFO:teuthology.orchestra.run.smithi045.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-01-31T05:12:32.148 INFO:teuthology.orchestra.run.smithi045.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-01-31T05:12:32.149 INFO:teuthology.orchestra.run.smithi045.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5... 2024-01-31T05:12:32.635 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 2024-01-31T05:12:32.636 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5... 2024-01-31T05:12:32.636 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Getting image source signatures 2024-01-31T05:12:32.636 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:7526f7e228dfc90a8cd9a697e920af939ccb44066789c4e5461c5efe2a22fe5d 2024-01-31T05:12:32.636 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:ea3366b8a90265ae3f6d3736cb0de4b52da4ffec919b46c6cb5bc5be1372efe8 2024-01-31T05:12:32.636 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying config sha256:0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 2024-01-31T05:12:32.636 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Writing manifest to image destination 2024-01-31T05:12:33.489 INFO:teuthology.orchestra.run.smithi045.stdout:ceph: stdout ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:12:33.490 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph version: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:12:33.490 INFO:teuthology.orchestra.run.smithi045.stdout:Extracting ceph user uid/gid from container image... 2024-01-31T05:12:34.487 INFO:teuthology.orchestra.run.smithi045.stdout:stat: stdout 167 167 2024-01-31T05:12:34.488 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial keys... 2024-01-31T05:12:38.077 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQDC1rllYXsqMRAADz1iII242cD+KlS86NZEYw== 2024-01-31T05:12:46.859 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQDI1rllmeJxNhAA7Dr5+Q2/3Kjy3ikWNWJwIw== 2024-01-31T05:12:48.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQDP1rllg9x/GRAAMP6IPzS2GIWxO6aEbwZlig== 2024-01-31T05:12:48.184 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial monmap... 2024-01-31T05:12:49.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-01-31T05:12:49.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-01-31T05:12:49.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:12:49.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-01-31T05:12:49.071 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool for a [v2:172.21.15.45:3300,v1:172.21.15.45:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-01-31T05:12:49.071 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-01-31T05:12:49.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: set fsid to 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:12:49.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-01-31T05:12:49.072 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:12:49.072 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mon... 2024-01-31T05:12:49.910 INFO:teuthology.orchestra.run.smithi045.stdout:create mon.a on 2024-01-31T05:12:50.113 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-01-31T05:12:50.280 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-01-31T05:12:50.457 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3a1f4122-bff7-11ee-95b3-87774f69a715.target → /etc/systemd/system/ceph-3a1f4122-bff7-11ee-95b3-87774f69a715.target. 2024-01-31T05:12:50.457 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3a1f4122-bff7-11ee-95b3-87774f69a715.target → /etc/systemd/system/ceph-3a1f4122-bff7-11ee-95b3-87774f69a715.target. 2024-01-31T05:12:50.705 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.a 2024-01-31T05:12:50.705 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.a.service: Unit ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.a.service not loaded. 2024-01-31T05:12:50.872 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3a1f4122-bff7-11ee-95b3-87774f69a715.target.wants/ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.a.service → /etc/systemd/system/ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@.service. 2024-01-31T05:12:51.384 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-01-31T05:12:51.384 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-01-31T05:12:51.391 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-01-31T05:12:51.391 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-01-31T05:12:51.391 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-01-31T05:12:51.391 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-01-31T05:12:51.391 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon to start... 2024-01-31T05:12:51.391 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon... 2024-01-31T05:12:53.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout cluster: 2024-01-31T05:12:53.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout id: 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:12:53.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-01-31T05:12:53.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:12:53.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout services: 2024-01-31T05:12:53.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.553605s) 2024-01-31T05:12:53.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-01-31T05:12:53.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-01-31T05:12:53.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:12:53.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout data: 2024-01-31T05:12:53.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-01-31T05:12:53.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-01-31T05:12:53.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-01-31T05:12:53.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pgs: 2024-01-31T05:12:53.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:12:53.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.814+0000 7f4ae979b700 1 Processor -- start 2024-01-31T05:12:53.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.814+0000 7f4ae979b700 1 -- start start 2024-01-31T05:12:53.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.815+0000 7f4ae979b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 0x7f4ae4108d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:12:53.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.815+0000 7f4ae979b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae41092d0 con 0x7f4ae41088e0 2024-01-31T05:12:53.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.815+0000 7f4ae2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 0x7f4ae4108d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:12:53.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.815+0000 7f4ae2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 0x7f4ae4108d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48474/0 (socket says 172.21.15.45:48474) 2024-01-31T05:12:53.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.815+0000 7f4ae2ffd700 1 -- 172.21.15.45:0/1248097388 learned_addr learned my addr 172.21.15.45:0/1248097388 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:12:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.815+0000 7f4ae2ffd700 1 -- 172.21.15.45:0/1248097388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae4106ec0 con 0x7f4ae41088e0 2024-01-31T05:12:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.816+0000 7f4ae2ffd700 1 --2- 172.21.15.45:0/1248097388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 0x7f4ae4108d00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0098a0 tx=0x7f4acc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a079635380ed226f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:12:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.816+0000 7f4ae1ffb700 1 -- 172.21.15.45:0/1248097388 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4acc004160 con 0x7f4ae41088e0 2024-01-31T05:12:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.816+0000 7f4ae1ffb700 1 -- 172.21.15.45:0/1248097388 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4acc0042c0 con 0x7f4ae41088e0 2024-01-31T05:12:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.816+0000 7f4ae1ffb700 1 -- 172.21.15.45:0/1248097388 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4acc0045a0 con 0x7f4ae41088e0 2024-01-31T05:12:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.816+0000 7f4ae979b700 1 -- 172.21.15.45:0/1248097388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 msgr2=0x7f4ae4108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:12:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.816+0000 7f4ae979b700 1 --2- 172.21.15.45:0/1248097388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 0x7f4ae4108d00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0098a0 tx=0x7f4acc009bb0 comp rx=0 tx=0).stop 2024-01-31T05:12:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.816+0000 7f4ae979b700 1 -- 172.21.15.45:0/1248097388 shutdown_connections 2024-01-31T05:12:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.816+0000 7f4ae979b700 1 --2- 172.21.15.45:0/1248097388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 0x7f4ae4108d00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:12:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.816+0000 7f4ae979b700 1 -- 172.21.15.45:0/1248097388 >> 172.21.15.45:0/1248097388 conn(0x7f4ae41028a0 msgr2=0x7f4ae4104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:12:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.816+0000 7f4ae979b700 1 -- 172.21.15.45:0/1248097388 shutdown_connections 2024-01-31T05:12:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.817+0000 7f4ae979b700 1 -- 172.21.15.45:0/1248097388 wait complete. 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.817+0000 7f4ae979b700 1 Processor -- start 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.817+0000 7f4ae979b700 1 -- start start 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.817+0000 7f4ae979b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 0x7f4ae407cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.817+0000 7f4ae979b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae407d0c0 con 0x7f4ae41088e0 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.817+0000 7f4ae2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 0x7f4ae407cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.817+0000 7f4ae2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 0x7f4ae407cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48478/0 (socket says 172.21.15.45:48478) 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.817+0000 7f4ae2ffd700 1 -- 172.21.15.45:0/981158369 learned_addr learned my addr 172.21.15.45:0/981158369 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.818+0000 7f4ae2ffd700 1 -- 172.21.15.45:0/981158369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc009550 con 0x7f4ae41088e0 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.818+0000 7f4ae2ffd700 1 --2- 172.21.15.45:0/981158369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 0x7f4ae407cb80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4acc004000 tx=0x7f4acc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.818+0000 7f4adbfff700 1 -- 172.21.15.45:0/981158369 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4acc011630 con 0x7f4ae41088e0 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.818+0000 7f4adbfff700 1 -- 172.21.15.45:0/981158369 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4acc011790 con 0x7f4ae41088e0 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.818+0000 7f4ae979b700 1 -- 172.21.15.45:0/981158369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae407b2b0 con 0x7f4ae41088e0 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.818+0000 7f4adbfff700 1 -- 172.21.15.45:0/981158369 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4acc011a30 con 0x7f4ae41088e0 2024-01-31T05:12:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.818+0000 7f4ae979b700 1 -- 172.21.15.45:0/981158369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae407b750 con 0x7f4ae41088e0 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.818+0000 7f4adbfff700 1 -- 172.21.15.45:0/981158369 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4acc022070 con 0x7f4ae41088e0 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.818+0000 7f4adbfff700 1 -- 172.21.15.45:0/981158369 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4acc01c450 con 0x7f4ae41088e0 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.819+0000 7f4ae979b700 1 -- 172.21.15.45:0/981158369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae4062e10 con 0x7f4ae41088e0 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.821+0000 7f4adbfff700 1 -- 172.21.15.45:0/981158369 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f4acc011b90 con 0x7f4ae41088e0 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.893+0000 7f4ae979b700 1 -- 172.21.15.45:0/981158369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f4ae4107230 con 0x7f4ae41088e0 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.894+0000 7f4adbfff700 1 -- 172.21.15.45:0/981158369 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f4acc011d30 con 0x7f4ae41088e0 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.894+0000 7f4ae979b700 1 -- 172.21.15.45:0/981158369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 msgr2=0x7f4ae407cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.894+0000 7f4ae979b700 1 --2- 172.21.15.45:0/981158369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 0x7f4ae407cb80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4acc004000 tx=0x7f4acc004750 comp rx=0 tx=0).stop 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.894+0000 7f4ae979b700 1 -- 172.21.15.45:0/981158369 shutdown_connections 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.894+0000 7f4ae979b700 1 --2- 172.21.15.45:0/981158369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae41088e0 0x7f4ae407cb80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.894+0000 7f4ae979b700 1 -- 172.21.15.45:0/981158369 >> 172.21.15.45:0/981158369 conn(0x7f4ae41028a0 msgr2=0x7f4ae4191550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.895+0000 7f4ae979b700 1 -- 172.21.15.45:0/981158369 shutdown_connections 2024-01-31T05:12:53.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:51.895+0000 7f4ae979b700 1 -- 172.21.15.45:0/981158369 wait complete. 2024-01-31T05:12:53.799 INFO:teuthology.orchestra.run.smithi045.stdout:mon is available 2024-01-31T05:12:53.799 INFO:teuthology.orchestra.run.smithi045.stdout:Assimilating anything we can from ceph.conf... 2024-01-31T05:12:54.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:54 smithi045 ceph-mon[31213]: from='client.? 172.21.15.45:0/1909839944' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-01-31T05:12:54.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:12:54.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-01-31T05:12:54.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:12:54.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-01-31T05:12:54.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-01-31T05:12:54.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-01-31T05:12:54.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-01-31T05:12:54.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-01-31T05:12:54.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:12:54.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-01-31T05:12:54.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-01-31T05:12:54.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:12:54.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-01-31T05:12:54.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-01-31T05:12:54.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-01-31T05:12:54.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-01-31T05:12:54.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-01-31T05:12:54.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-01-31T05:12:54.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.204+0000 7fd5969e1700 1 Processor -- start 2024-01-31T05:12:54.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.204+0000 7fd5969e1700 1 -- start start 2024-01-31T05:12:54.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.204+0000 7fd5969e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 0x7fd590106580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:12:54.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.204+0000 7fd5969e1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd590106b50 con 0x7fd590106160 2024-01-31T05:12:54.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.204+0000 7fd58ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 0x7fd590106580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:12:54.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.204+0000 7fd58ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 0x7fd590106580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48488/0 (socket says 172.21.15.45:48488) 2024-01-31T05:12:54.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.204+0000 7fd58ffff700 1 -- 172.21.15.45:0/1788081371 learned_addr learned my addr 172.21.15.45:0/1788081371 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:12:54.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.205+0000 7fd58ffff700 1 -- 172.21.15.45:0/1788081371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd590107360 con 0x7fd590106160 2024-01-31T05:12:54.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.205+0000 7fd58ffff700 1 --2- 172.21.15.45:0/1788081371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 0x7fd590106580 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd5780098a0 tx=0x7fd578009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e26b18f0448e94 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:12:54.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.205+0000 7fd58effd700 1 -- 172.21.15.45:0/1788081371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd578004160 con 0x7fd590106160 2024-01-31T05:12:54.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.205+0000 7fd58effd700 1 -- 172.21.15.45:0/1788081371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5780042c0 con 0x7fd590106160 2024-01-31T05:12:54.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.205+0000 7fd58effd700 1 -- 172.21.15.45:0/1788081371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5780045a0 con 0x7fd590106160 2024-01-31T05:12:54.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.205+0000 7fd5969e1700 1 -- 172.21.15.45:0/1788081371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 msgr2=0x7fd590106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:12:54.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.206+0000 7fd5969e1700 1 --2- 172.21.15.45:0/1788081371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 0x7fd590106580 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd5780098a0 tx=0x7fd578009bb0 comp rx=0 tx=0).stop 2024-01-31T05:12:54.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.206+0000 7fd5969e1700 1 -- 172.21.15.45:0/1788081371 shutdown_connections 2024-01-31T05:12:54.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.206+0000 7fd5969e1700 1 --2- 172.21.15.45:0/1788081371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 0x7fd590106580 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:12:54.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.206+0000 7fd5969e1700 1 -- 172.21.15.45:0/1788081371 >> 172.21.15.45:0/1788081371 conn(0x7fd590101740 msgr2=0x7fd590103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:12:54.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.206+0000 7fd5969e1700 1 -- 172.21.15.45:0/1788081371 shutdown_connections 2024-01-31T05:12:54.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.206+0000 7fd5969e1700 1 -- 172.21.15.45:0/1788081371 wait complete. 2024-01-31T05:12:54.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.206+0000 7fd5969e1700 1 Processor -- start 2024-01-31T05:12:54.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.206+0000 7fd5969e1700 1 -- start start 2024-01-31T05:12:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.207+0000 7fd5969e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 0x7fd59018ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:12:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.207+0000 7fd5969e1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59018f260 con 0x7fd590106160 2024-01-31T05:12:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.207+0000 7fd58ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 0x7fd59018ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:12:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.207+0000 7fd58ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 0x7fd59018ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48498/0 (socket says 172.21.15.45:48498) 2024-01-31T05:12:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.207+0000 7fd58ffff700 1 -- 172.21.15.45:0/1909839944 learned_addr learned my addr 172.21.15.45:0/1909839944 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:12:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.207+0000 7fd58ffff700 1 -- 172.21.15.45:0/1909839944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd578009550 con 0x7fd590106160 2024-01-31T05:12:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.207+0000 7fd58ffff700 1 --2- 172.21.15.45:0/1909839944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 0x7fd59018ed20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd578004000 tx=0x7fd578004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:12:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.207+0000 7fd58d7fa700 1 -- 172.21.15.45:0/1909839944 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd578011630 con 0x7fd590106160 2024-01-31T05:12:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.207+0000 7fd58d7fa700 1 -- 172.21.15.45:0/1909839944 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd578011790 con 0x7fd590106160 2024-01-31T05:12:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.207+0000 7fd5969e1700 1 -- 172.21.15.45:0/1909839944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59018d450 con 0x7fd590106160 2024-01-31T05:12:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.207+0000 7fd58d7fa700 1 -- 172.21.15.45:0/1909839944 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd578011a30 con 0x7fd590106160 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.207+0000 7fd5969e1700 1 -- 172.21.15.45:0/1909839944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59018d8f0 con 0x7fd590106160 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.208+0000 7fd58d7fa700 1 -- 172.21.15.45:0/1909839944 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd578022070 con 0x7fd590106160 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.208+0000 7fd58d7fa700 1 -- 172.21.15.45:0/1909839944 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd57801c450 con 0x7fd590106160 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.208+0000 7fd5969e1700 1 -- 172.21.15.45:0/1909839944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59018e300 con 0x7fd590106160 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.211+0000 7fd58d7fa700 1 -- 172.21.15.45:0/1909839944 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7fd578011b90 con 0x7fd590106160 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.280+0000 7fd5969e1700 1 -- 172.21.15.45:0/1909839944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd5901076d0 con 0x7fd590106160 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.283+0000 7fd58d7fa700 1 -- 172.21.15.45:0/1909839944 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fd578011de0 con 0x7fd590106160 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.284+0000 7fd58d7fa700 1 -- 172.21.15.45:0/1909839944 <== mon.0 v2:172.21.15.45:3300/0 8 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd57801c6b0 con 0x7fd590106160 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.285+0000 7fd5969e1700 1 -- 172.21.15.45:0/1909839944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 msgr2=0x7fd59018ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.285+0000 7fd5969e1700 1 --2- 172.21.15.45:0/1909839944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 0x7fd59018ed20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd578004000 tx=0x7fd578004750 comp rx=0 tx=0).stop 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.285+0000 7fd5969e1700 1 -- 172.21.15.45:0/1909839944 shutdown_connections 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.285+0000 7fd5969e1700 1 --2- 172.21.15.45:0/1909839944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd590106160 0x7fd59018ed20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.285+0000 7fd5969e1700 1 -- 172.21.15.45:0/1909839944 >> 172.21.15.45:0/1909839944 conn(0x7fd590101740 msgr2=0x7fd5901914b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.285+0000 7fd5969e1700 1 -- 172.21.15.45:0/1909839944 shutdown_connections 2024-01-31T05:12:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:54.285+0000 7fd5969e1700 1 -- 172.21.15.45:0/1909839944 wait complete. 2024-01-31T05:12:54.788 INFO:teuthology.orchestra.run.smithi045.stdout:Generating new minimal ceph.conf... 2024-01-31T05:12:55.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:55 smithi045 ceph-mon[31213]: from='client.? 172.21.15.45:0/3769776043' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:12:55.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.187+0000 7f66374c5700 1 Processor -- start 2024-01-31T05:12:55.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.187+0000 7f66374c5700 1 -- start start 2024-01-31T05:12:55.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.188+0000 7f66374c5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 0x7f663007b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:12:55.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.188+0000 7f66374c5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663007b8e0 con 0x7f663007ceb0 2024-01-31T05:12:55.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.188+0000 7f6635261700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 0x7f663007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:12:55.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.188+0000 7f6635261700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 0x7f663007b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48500/0 (socket says 172.21.15.45:48500) 2024-01-31T05:12:55.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.188+0000 7f6635261700 1 -- 172.21.15.45:0/3152465362 learned_addr learned my addr 172.21.15.45:0/3152465362 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:12:55.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.188+0000 7f6635261700 1 -- 172.21.15.45:0/3152465362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663007ba20 con 0x7f663007ceb0 2024-01-31T05:12:55.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.188+0000 7f6635261700 1 --2- 172.21.15.45:0/3152465362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 0x7f663007b310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f66200098a0 tx=0x7f6620009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29346d8fbd64a17 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:12:55.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.188+0000 7f6627fff700 1 -- 172.21.15.45:0/3152465362 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6620004160 con 0x7f663007ceb0 2024-01-31T05:12:55.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.188+0000 7f6627fff700 1 -- 172.21.15.45:0/3152465362 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f66200042c0 con 0x7f663007ceb0 2024-01-31T05:12:55.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.188+0000 7f6627fff700 1 -- 172.21.15.45:0/3152465362 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6620011a00 con 0x7f663007ceb0 2024-01-31T05:12:55.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.189+0000 7f66374c5700 1 -- 172.21.15.45:0/3152465362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 msgr2=0x7f663007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:12:55.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.189+0000 7f66374c5700 1 --2- 172.21.15.45:0/3152465362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 0x7f663007b310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f66200098a0 tx=0x7f6620009bb0 comp rx=0 tx=0).stop 2024-01-31T05:12:55.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.189+0000 7f66374c5700 1 -- 172.21.15.45:0/3152465362 shutdown_connections 2024-01-31T05:12:55.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.189+0000 7f66374c5700 1 --2- 172.21.15.45:0/3152465362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 0x7f663007b310 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:12:55.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.189+0000 7f66374c5700 1 -- 172.21.15.45:0/3152465362 >> 172.21.15.45:0/3152465362 conn(0x7f6630102890 msgr2=0x7f6630104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:12:55.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.189+0000 7f66374c5700 1 -- 172.21.15.45:0/3152465362 shutdown_connections 2024-01-31T05:12:55.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.189+0000 7f66374c5700 1 -- 172.21.15.45:0/3152465362 wait complete. 2024-01-31T05:12:55.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.190+0000 7f66374c5700 1 Processor -- start 2024-01-31T05:12:55.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.190+0000 7f66374c5700 1 -- start start 2024-01-31T05:12:55.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.190+0000 7f66374c5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 0x7f663019d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:12:55.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.190+0000 7f66374c5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663019d6b0 con 0x7f663007ceb0 2024-01-31T05:12:55.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.190+0000 7f6635261700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 0x7f663019d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:12:55.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.190+0000 7f6635261700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 0x7f663019d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48504/0 (socket says 172.21.15.45:48504) 2024-01-31T05:12:55.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.190+0000 7f6635261700 1 -- 172.21.15.45:0/3769776043 learned_addr learned my addr 172.21.15.45:0/3769776043 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:12:55.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.190+0000 7f6635261700 1 -- 172.21.15.45:0/3769776043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620009550 con 0x7f663007ceb0 2024-01-31T05:12:55.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.190+0000 7f6635261700 1 --2- 172.21.15.45:0/3769776043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 0x7f663019d170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6620011710 tx=0x7f6620011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:12:55.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.190+0000 7f66267fc700 1 -- 172.21.15.45:0/3769776043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6620003aa0 con 0x7f663007ceb0 2024-01-31T05:12:55.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.190+0000 7f66267fc700 1 -- 172.21.15.45:0/3769776043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6620003c00 con 0x7f663007ceb0 2024-01-31T05:12:55.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.191+0000 7f66374c5700 1 -- 172.21.15.45:0/3769776043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663019d8b0 con 0x7f663007ceb0 2024-01-31T05:12:55.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.191+0000 7f66267fc700 1 -- 172.21.15.45:0/3769776043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6620024800 con 0x7f663007ceb0 2024-01-31T05:12:55.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.191+0000 7f66374c5700 1 -- 172.21.15.45:0/3769776043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663019dd50 con 0x7f663007ceb0 2024-01-31T05:12:55.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.191+0000 7f66267fc700 1 -- 172.21.15.45:0/3769776043 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6620024960 con 0x7f663007ceb0 2024-01-31T05:12:55.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.191+0000 7f66267fc700 1 -- 172.21.15.45:0/3769776043 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f66200193c0 con 0x7f663007ceb0 2024-01-31T05:12:55.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.191+0000 7f66374c5700 1 -- 172.21.15.45:0/3769776043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663010d5a0 con 0x7f663007ceb0 2024-01-31T05:12:55.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.194+0000 7f66267fc700 1 -- 172.21.15.45:0/3769776043 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f6620024d50 con 0x7f663007ceb0 2024-01-31T05:12:55.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.263+0000 7f66374c5700 1 -- 172.21.15.45:0/3769776043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f66301a07e0 con 0x7f663007ceb0 2024-01-31T05:12:55.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.264+0000 7f66267fc700 1 -- 172.21.15.45:0/3769776043 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f662001abd0 con 0x7f663007ceb0 2024-01-31T05:12:55.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.264+0000 7f66374c5700 1 -- 172.21.15.45:0/3769776043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 msgr2=0x7f663019d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:12:55.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.264+0000 7f66374c5700 1 --2- 172.21.15.45:0/3769776043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 0x7f663019d170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6620011710 tx=0x7f6620011ad0 comp rx=0 tx=0).stop 2024-01-31T05:12:55.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.265+0000 7f66374c5700 1 -- 172.21.15.45:0/3769776043 shutdown_connections 2024-01-31T05:12:55.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.265+0000 7f66374c5700 1 --2- 172.21.15.45:0/3769776043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f663007ceb0 0x7f663019d170 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:12:55.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.265+0000 7f66374c5700 1 -- 172.21.15.45:0/3769776043 >> 172.21.15.45:0/3769776043 conn(0x7f6630102890 msgr2=0x7f6630103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:12:55.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.265+0000 7f66374c5700 1 -- 172.21.15.45:0/3769776043 shutdown_connections 2024-01-31T05:12:55.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:55.265+0000 7f66374c5700 1 -- 172.21.15.45:0/3769776043 wait complete. 2024-01-31T05:12:55.833 INFO:teuthology.orchestra.run.smithi045.stdout:Restarting the monitor... 2024-01-31T05:12:56.093 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:55 smithi045 systemd[1]: Stopping Ceph mon.a for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:12:56.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:56 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a[31209]: 2024-01-31T05:12:56.093+0000 7f278bdf7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:12:56.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:56 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a[31209]: 2024-01-31T05:12:56.093+0000 7f278bdf7700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-01-31T05:12:56.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:56 smithi045 podman[31458]: 2024-01-31 05:12:56.206436524 +0000 UTC m=+0.214743276 container died 8f4c862479ef56b8a79924ec4a758932209658a23494d1c438d7fd853aae5590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3) 2024-01-31T05:12:56.879 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:56 smithi045 podman[31458]: 2024-01-31 05:12:56.55154187 +0000 UTC m=+0.559848617 container cleanup 8f4c862479ef56b8a79924ec4a758932209658a23494d1c438d7fd853aae5590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, org.label-schema.build-date=20240123) 2024-01-31T05:12:56.879 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:56 smithi045 bash[31458]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a 2024-01-31T05:12:56.879 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:56 smithi045 podman[31470]: 2024-01-31 05:12:56.685861442 +0000 UTC m=+0.474973953 container remove 8f4c862479ef56b8a79924ec4a758932209658a23494d1c438d7fd853aae5590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.license=GPLv2) 2024-01-31T05:12:57.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:56 smithi045 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.a.service: Deactivated successfully. 2024-01-31T05:12:57.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:56 smithi045 systemd[1]: Stopped Ceph mon.a for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:12:57.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:56 smithi045 systemd[1]: Starting Ceph mon.a for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:12:57.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 podman[31566]: 2024-01-31 05:12:57.012114335 +0000 UTC m=+0.018668603 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:12:57.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 podman[31566]: 2024-01-31 05:12:57.111760131 +0000 UTC m=+0.118314397 container create 6db52a688cd2808402d8eb14120a95d364d10b03eb95d6a7cc24d269d5770ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:12:57.391 INFO:teuthology.orchestra.run.smithi045.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-01-31T05:12:57.506 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 podman[31566]: 2024-01-31 05:12:57.314210442 +0000 UTC m=+0.320764710 container init 6db52a688cd2808402d8eb14120a95d364d10b03eb95d6a7cc24d269d5770ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2) 2024-01-31T05:12:57.506 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 podman[31566]: 2024-01-31 05:12:57.316974012 +0000 UTC m=+0.323528291 container start 6db52a688cd2808402d8eb14120a95d364d10b03eb95d6a7cc24d269d5770ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, maintainer=Guillaume Abrioux ) 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: set uid:gid to 167:167 (ceph:ceph) 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev), process ceph-mon, pid 2 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: pidfile_write: ignore empty --pid-file 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: load: jerasure load: lrc 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: RocksDB version: 7.9.2 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Git sha 0 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Compile date 2024-01-25 05:03:27 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: DB SUMMARY 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: DB Session ID: B0YZNH65W65SEGPADG5G 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: CURRENT file: CURRENT 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: IDENTITY file: IDENTITY 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-01-31T05:12:57.507 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 101166 ; 2024-01-31T05:12:57.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.error_if_exists: 0 2024-01-31T05:12:57.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.create_if_missing: 0 2024-01-31T05:12:57.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.paranoid_checks: 1 2024-01-31T05:12:57.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.flush_verify_memtable_count: 1 2024-01-31T05:12:57.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-01-31T05:12:57.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-01-31T05:12:57.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.env: 0x559f513aeae0 2024-01-31T05:12:57.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.fs: PosixFileSystem 2024-01-31T05:12:57.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.info_log: 0x559f52e3d780 2024-01-31T05:12:57.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_file_opening_threads: 16 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.statistics: (nil) 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.use_fsync: 0 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_log_file_size: 0 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.log_file_time_to_roll: 0 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.keep_log_file_num: 1000 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.recycle_log_file_num: 0 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.allow_fallocate: 1 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.allow_mmap_reads: 0 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.allow_mmap_writes: 0 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.use_direct_reads: 0 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-01-31T05:12:57.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.create_missing_column_families: 0 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.db_log_dir: 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.wal_dir: 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.table_cache_numshardbits: 6 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.WAL_ttl_seconds: 0 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.WAL_size_limit_MB: 0 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.is_fd_close_on_exec: 1 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.advise_random_on_open: 1 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.db_write_buffer_size: 0 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.write_buffer_manager: 0x559f52e11400 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-01-31T05:12:57.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.use_adaptive_mutex: 0 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.rate_limiter: (nil) 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.wal_recovery_mode: 2 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.enable_thread_tracking: 0 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.enable_pipelined_write: 0 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.unordered_write: 0 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.row_cache: None 2024-01-31T05:12:57.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.wal_filter: None 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.allow_ingest_behind: 0 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.two_write_queues: 0 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.manual_wal_flush: 0 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.wal_compression: 0 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.atomic_flush: 0 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.persist_stats_to_disk: 0 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.write_dbid_to_manifest: 0 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.log_readahead_size: 0 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.best_efforts_recovery: 0 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-01-31T05:12:57.513 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.allow_data_in_errors: 0 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.db_host_id: __hostname__ 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.enforce_single_del_contracts: true 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_background_jobs: 2 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_background_compactions: -1 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_subcompactions: 1 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.delayed_write_rate : 16777216 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_total_wal_size: 0 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.stats_dump_period_sec: 600 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.stats_persist_period_sec: 600 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-01-31T05:12:57.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_open_files: -1 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bytes_per_sync: 0 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.wal_bytes_per_sync: 0 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.strict_bytes_per_sync: 0 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_readahead_size: 0 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_background_flushes: -1 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Compression algorithms supported: 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: kZSTD supported: 0 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: kXpressCompression supported: 0 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: kLZ4HCCompression supported: 1 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: kZlibCompression supported: 1 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: kSnappyCompression supported: 1 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: kLZ4Compression supported: 1 2024-01-31T05:12:57.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: kBZip2Compression supported: 0 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Fast CRC32 supported: Supported on x86 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: DMutex implementation: pthread_mutex_t 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.merge_operator: 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_filter: None 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_filter_factory: None 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.sst_partitioner_factory: None 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.memtable_factory: SkipListFactory 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.table_factory: BlockBasedTable 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559f52e3d8a0) 2024-01-31T05:12:57.516 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks: 1 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_top_level_index_and_filter: 1 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_type: 0 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_index_type: 0 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_shortening: 1 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: checksum: 4 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: no_block_cache: 0 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache: 0x559f52e41090 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_name: BinnedLRUCache 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_options: 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: capacity : 536870912 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_shard_bits : 4 2024-01-31T05:12:57.517 INFO:journalctl@ceph.mon.a.smithi045.stdout: strict_capacity_limit : 0 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: high_pri_pool_ratio: 0.000 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_compressed: (nil) 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: persistent_cache: (nil) 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size: 4096 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size_deviation: 10 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_restart_interval: 16 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_block_restart_interval: 1 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: metadata_block_size: 4096 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: partition_filters: 0 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: use_delta_encoding: 1 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: filter_policy: bloomfilter 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: whole_key_filtering: 1 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: verify_compression: 0 2024-01-31T05:12:57.518 INFO:journalctl@ceph.mon.a.smithi045.stdout: read_amp_bytes_per_bit: 0 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout: format_version: 5 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_index_compression: 1 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_align: 0 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout: max_auto_readahead_size: 262144 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout: prepopulate_block_cache: 0 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout: initial_auto_readahead_size: 8192 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.write_buffer_size: 33554432 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_write_buffer_number: 2 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compression: NoCompression 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bottommost_compression: Disabled 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.prefix_extractor: nullptr 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.num_levels: 7 2024-01-31T05:12:57.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compression_opts.window_bits: -14 2024-01-31T05:12:57.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compression_opts.level: 32767 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compression_opts.strategy: 0 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compression_opts.enabled: false 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.target_file_size_base: 67108864 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.target_file_size_multiplier: 1 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-01-31T05:12:57.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.arena_block_size: 1048576 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.disable_auto_compactions: 0 2024-01-31T05:12:57.522 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.table_properties_collectors: 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.inplace_update_support: 0 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.inplace_update_num_locks: 10000 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-01-31T05:12:57.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.memtable_huge_page_size: 0 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.bloom_locality: 0 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.max_successive_merges: 0 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.optimize_filters_for_hits: 0 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.paranoid_file_checks: 0 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.force_consistency_checks: 1 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.report_bg_io_stats: 0 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.ttl: 2592000 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.periodic_compaction_seconds: 0 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.enable_blob_files: false 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.min_blob_size: 0 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.blob_file_size: 268435456 2024-01-31T05:12:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.blob_compression_type: NoCompression 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.enable_blob_garbage_collection: false 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.blob_file_starting_level: 0 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 59cfae46-93f5-49ef-b41b-7e0a56af56de 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706677977345307, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706677977346257, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 96223, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 94211, "index_size": 294, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 14132, "raw_average_key_size": 50, "raw_value_size": 86525, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1706677977, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "59cfae46-93f5-49ef-b41b-7e0a56af56de", "db_session_id": "B0YZNH65W65SEGPADG5G", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706677977346317, "job": 1, "event": "recovery_finished"} 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-01-31T05:12:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559f52f2e000 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: DB pointer 0x559f52f1c000 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** DB Stats ** 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-01-31T05:12:57.526 INFO:journalctl@ceph.mon.a.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: L0 2/0 95.80 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: Sum 2/0 95.80 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 157.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-01-31T05:12:57.527 INFO:journalctl@ceph.mon.a.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative compaction: 0.00 GB write, 31.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval compaction: 0.00 GB write, 31.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache BinnedLRUCache@0x559f52e41090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: starting mon.a rank 0 at public addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] at bind addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:12:57.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: mon.a@-1(???) e1 preinit fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: mon.a@-1(???).mds e1 new map 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: mon.a@-1(???).mds e1 print_map 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout: e1 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout: legacy client fscid: -1 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout: No filesystems configured 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 ceph-mon[31585]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 bash[31566]: 6db52a688cd2808402d8eb14120a95d364d10b03eb95d6a7cc24d269d5770ccc 2024-01-31T05:12:57.529 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:57 smithi045 systemd[1]: Started Ceph mon.a for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:12:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:58 smithi045 ceph-mon[31585]: mon.a is new leader, mons a in quorum (ranks 0) 2024-01-31T05:12:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:58 smithi045 ceph-mon[31585]: monmap e1: 1 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-31T05:12:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:58 smithi045 ceph-mon[31585]: fsmap 2024-01-31T05:12:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:58 smithi045 ceph-mon[31585]: osdmap e1: 0 total, 0 up, 0 in 2024-01-31T05:12:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:58 smithi045 ceph-mon[31585]: mgrmap e1: no daemons active 2024-01-31T05:12:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:12:58 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2702311180' entity='client.admin' 2024-01-31T05:12:58.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.870+0000 7f06fbd7d700 1 Processor -- start 2024-01-31T05:12:58.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.870+0000 7f06fbd7d700 1 -- start start 2024-01-31T05:12:58.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.870+0000 7f06fbd7d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 0x7f06f410a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:12:58.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.870+0000 7f06fbd7d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f407ae40 con 0x7f06f4107d50 2024-01-31T05:12:58.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.870+0000 7f06f9b19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 0x7f06f410a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:12:58.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.870+0000 7f06f9b19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 0x7f06f410a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47270/0 (socket says 172.21.15.45:47270) 2024-01-31T05:12:58.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.870+0000 7f06f9b19700 1 -- 172.21.15.45:0/1147893746 learned_addr learned my addr 172.21.15.45:0/1147893746 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.871+0000 7f06f9b19700 1 -- 172.21.15.45:0/1147893746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f410a680 con 0x7f06f4107d50 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.871+0000 7f06f9b19700 1 --2- 172.21.15.45:0/1147893746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 0x7f06f410a140 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f06f00098a0 tx=0x7f06f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06cdb3a93a90028 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.871+0000 7f06f8b17700 1 -- 172.21.15.45:0/1147893746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06f0004160 con 0x7f06f4107d50 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.871+0000 7f06f8b17700 1 -- 172.21.15.45:0/1147893746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f06f00042c0 con 0x7f06f4107d50 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.871+0000 7f06f8b17700 1 -- 172.21.15.45:0/1147893746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06f0011a00 con 0x7f06f4107d50 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.872+0000 7f06fbd7d700 1 -- 172.21.15.45:0/1147893746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 msgr2=0x7f06f410a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.872+0000 7f06fbd7d700 1 --2- 172.21.15.45:0/1147893746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 0x7f06f410a140 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f06f00098a0 tx=0x7f06f0009bb0 comp rx=0 tx=0).stop 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.872+0000 7f06fbd7d700 1 -- 172.21.15.45:0/1147893746 shutdown_connections 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.872+0000 7f06fbd7d700 1 --2- 172.21.15.45:0/1147893746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 0x7f06f410a140 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.872+0000 7f06fbd7d700 1 -- 172.21.15.45:0/1147893746 >> 172.21.15.45:0/1147893746 conn(0x7f06f4101720 msgr2=0x7f06f4103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.872+0000 7f06fbd7d700 1 -- 172.21.15.45:0/1147893746 shutdown_connections 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.872+0000 7f06fbd7d700 1 -- 172.21.15.45:0/1147893746 wait complete. 2024-01-31T05:12:58.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.872+0000 7f06fbd7d700 1 Processor -- start 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.872+0000 7f06fbd7d700 1 -- start start 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.872+0000 7f06fbd7d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 0x7f06f419af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.872+0000 7f06fbd7d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f419b4d0 con 0x7f06f4107d50 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.873+0000 7f06f9b19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 0x7f06f419af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.873+0000 7f06f9b19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 0x7f06f419af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47284/0 (socket says 172.21.15.45:47284) 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.873+0000 7f06f9b19700 1 -- 172.21.15.45:0/2702311180 learned_addr learned my addr 172.21.15.45:0/2702311180 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.873+0000 7f06f9b19700 1 -- 172.21.15.45:0/2702311180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f0009550 con 0x7f06f4107d50 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.873+0000 7f06f9b19700 1 --2- 172.21.15.45:0/2702311180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 0x7f06f419af90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009130 tx=0x7f06f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.873+0000 7f06eaffd700 1 -- 172.21.15.45:0/2702311180 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06f00038b0 con 0x7f06f4107d50 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.873+0000 7f06eaffd700 1 -- 172.21.15.45:0/2702311180 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f06f0003a10 con 0x7f06f4107d50 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.873+0000 7f06fbd7d700 1 -- 172.21.15.45:0/2702311180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f419b6d0 con 0x7f06f4107d50 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.873+0000 7f06eaffd700 1 -- 172.21.15.45:0/2702311180 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06f0019550 con 0x7f06f4107d50 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.873+0000 7f06fbd7d700 1 -- 172.21.15.45:0/2702311180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f419bb70 con 0x7f06f4107d50 2024-01-31T05:12:58.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.873+0000 7f06eaffd700 1 -- 172.21.15.45:0/2702311180 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f06f0019b50 con 0x7f06f4107d50 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.873+0000 7f06eaffd700 1 -- 172.21.15.45:0/2702311180 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f06f00245c0 con 0x7f06f4107d50 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.874+0000 7f06fbd7d700 1 -- 172.21.15.45:0/2702311180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f410b360 con 0x7f06f4107d50 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.877+0000 7f06eaffd700 1 -- 172.21.15.45:0/2702311180 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f06f0003b80 con 0x7f06f4107d50 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.946+0000 7f06fbd7d700 1 -- 172.21.15.45:0/2702311180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f06f410b550 con 0x7f06f4107d50 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.948+0000 7f06eaffd700 1 -- 172.21.15.45:0/2702311180 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f06f001a960 con 0x7f06f4107d50 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.949+0000 7f06fbd7d700 1 -- 172.21.15.45:0/2702311180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 msgr2=0x7f06f419af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.949+0000 7f06fbd7d700 1 --2- 172.21.15.45:0/2702311180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 0x7f06f419af90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009130 tx=0x7f06f0011ad0 comp rx=0 tx=0).stop 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.949+0000 7f06fbd7d700 1 -- 172.21.15.45:0/2702311180 shutdown_connections 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.949+0000 7f06fbd7d700 1 --2- 172.21.15.45:0/2702311180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f4107d50 0x7f06f419af90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.949+0000 7f06fbd7d700 1 -- 172.21.15.45:0/2702311180 >> 172.21.15.45:0/2702311180 conn(0x7f06f4101720 msgr2=0x7f06f4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.949+0000 7f06fbd7d700 1 -- 172.21.15.45:0/2702311180 shutdown_connections 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:12:57.949+0000 7f06fbd7d700 1 -- 172.21.15.45:0/2702311180 wait complete. 2024-01-31T05:12:58.755 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote config to /etc/ceph/ceph.conf 2024-01-31T05:12:58.795 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-01-31T05:12:58.795 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mgr... 2024-01-31T05:12:58.795 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:9283 ... 2024-01-31T05:12:58.796 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:8765 ... 2024-01-31T05:12:59.056 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.x 2024-01-31T05:12:59.056 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.x.service: Unit ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.x.service not loaded. 2024-01-31T05:12:59.207 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3a1f4122-bff7-11ee-95b3-87774f69a715.target.wants/ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@.service. 2024-01-31T05:12:59.754 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-01-31T05:12:59.754 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-01-31T05:12:59.761 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-01-31T05:12:59.761 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-01-31T05:12:59.761 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-01-31T05:12:59.761 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-01-31T05:12:59.770 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-01-31T05:12:59.770 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-01-31T05:12:59.777 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-01-31T05:12:59.777 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-01-31T05:12:59.777 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-01-31T05:12:59.778 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-01-31T05:12:59.778 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr to start... 2024-01-31T05:12:59.778 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr... 2024-01-31T05:12:59.884 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:12:59 smithi045 systemd[1]: Started Ceph mgr.x for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:13:00.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:00 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3268892523' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-01-31T05:13:00.573 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:00.217+0000 7feedb383200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-31T05:13:00.573 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:00.313+0000 7feedb383200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-31T05:13:00.886 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:13:00.886 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-01-31T05:13:00.886 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "3a1f4122-bff7-11ee-95b3-87774f69a715", 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-01-31T05:13:00.887 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-01-31T05:13:00.888 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:00.888 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-01-31T05:13:00.888 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-01-31T05:13:00.888 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:00.888 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-01-31T05:13:00.888 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:00.888 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-01-31T05:13:00.888 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-01-31T05:13:00.888 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-01-31T05:13:00.896 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-01-31T05:13:00.896 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-01-31T05:13:00.896 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-01-31T05:13:00.896 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:00.896 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-01-31T05:13:00.896 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-01-31T05:13:00.896 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:00.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-01-31T05:12:51.342330+0000", 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-31T05:13:00.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.197+0000 7f9b4ea6c700 1 Processor -- start 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.197+0000 7f9b4ea6c700 1 -- start start 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.197+0000 7f9b4ea6c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 0x7f9b481076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.197+0000 7f9b4ea6c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b48107c90 con 0x7f9b481072a0 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.197+0000 7f9b4c808700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 0x7f9b481076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.197+0000 7f9b4c808700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 0x7f9b481076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47320/0 (socket says 172.21.15.45:47320) 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.197+0000 7f9b4c808700 1 -- 172.21.15.45:0/3590549643 learned_addr learned my addr 172.21.15.45:0/3590549643 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.197+0000 7f9b4c808700 1 -- 172.21.15.45:0/3590549643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b48108400 con 0x7f9b481072a0 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.198+0000 7f9b4c808700 1 --2- 172.21.15.45:0/3590549643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 0x7f9b481076c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0098a0 tx=0x7f9b3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea1292f32ac5906 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.198+0000 7f9b477fe700 1 -- 172.21.15.45:0/3590549643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b3c004160 con 0x7f9b481072a0 2024-01-31T05:13:00.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.198+0000 7f9b477fe700 1 -- 172.21.15.45:0/3590549643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9b3c0042c0 con 0x7f9b481072a0 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.198+0000 7f9b477fe700 1 -- 172.21.15.45:0/3590549643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b3c011a00 con 0x7f9b481072a0 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.198+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3590549643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 msgr2=0x7f9b481076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.198+0000 7f9b4ea6c700 1 --2- 172.21.15.45:0/3590549643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 0x7f9b481076c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0098a0 tx=0x7f9b3c009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.198+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3590549643 shutdown_connections 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.198+0000 7f9b4ea6c700 1 --2- 172.21.15.45:0/3590549643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 0x7f9b481076c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.198+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3590549643 >> 172.21.15.45:0/3590549643 conn(0x7f9b481028a0 msgr2=0x7f9b48104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.199+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3590549643 shutdown_connections 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.199+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3590549643 wait complete. 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.199+0000 7f9b4ea6c700 1 Processor -- start 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.199+0000 7f9b4ea6c700 1 -- start start 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.199+0000 7f9b4ea6c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 0x7f9b4807cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.199+0000 7f9b4ea6c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4807d0c0 con 0x7f9b481072a0 2024-01-31T05:13:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.199+0000 7f9b4c808700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 0x7f9b4807cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.199+0000 7f9b4c808700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 0x7f9b4807cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47330/0 (socket says 172.21.15.45:47330) 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.199+0000 7f9b4c808700 1 -- 172.21.15.45:0/3268892523 learned_addr learned my addr 172.21.15.45:0/3268892523 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.200+0000 7f9b4c808700 1 -- 172.21.15.45:0/3268892523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c009550 con 0x7f9b481072a0 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.200+0000 7f9b4c808700 1 --2- 172.21.15.45:0/3268892523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 0x7f9b4807cb80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009130 tx=0x7f9b3c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.200+0000 7f9b45ffb700 1 -- 172.21.15.45:0/3268892523 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b3c003ab0 con 0x7f9b481072a0 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.200+0000 7f9b45ffb700 1 -- 172.21.15.45:0/3268892523 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9b3c003c10 con 0x7f9b481072a0 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.200+0000 7f9b45ffb700 1 -- 172.21.15.45:0/3268892523 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b3c0247d0 con 0x7f9b481072a0 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.200+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3268892523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4807b2b0 con 0x7f9b481072a0 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.200+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3268892523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4807b750 con 0x7f9b481072a0 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.201+0000 7f9b45ffb700 1 -- 172.21.15.45:0/3268892523 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9b3c024a00 con 0x7f9b481072a0 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.201+0000 7f9b45ffb700 1 -- 172.21.15.45:0/3268892523 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9b3c0193c0 con 0x7f9b481072a0 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.201+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3268892523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b48062e10 con 0x7f9b481072a0 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.205+0000 7f9b45ffb700 1 -- 172.21.15.45:0/3268892523 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f9b3c024e40 con 0x7f9b481072a0 2024-01-31T05:13:00.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.280+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3268892523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9b4807b990 con 0x7f9b481072a0 2024-01-31T05:13:00.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.280+0000 7f9b45ffb700 1 -- 172.21.15.45:0/3268892523 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f9b3c035090 con 0x7f9b481072a0 2024-01-31T05:13:00.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.282+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3268892523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 msgr2=0x7f9b4807cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:00.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.282+0000 7f9b4ea6c700 1 --2- 172.21.15.45:0/3268892523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 0x7f9b4807cb80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009130 tx=0x7f9b3c011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:00.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.282+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3268892523 shutdown_connections 2024-01-31T05:13:00.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.282+0000 7f9b4ea6c700 1 --2- 172.21.15.45:0/3268892523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b481072a0 0x7f9b4807cb80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:00.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.282+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3268892523 >> 172.21.15.45:0/3268892523 conn(0x7f9b481028a0 msgr2=0x7f9b48103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:00.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.282+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3268892523 shutdown_connections 2024-01-31T05:13:00.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:00.282+0000 7f9b4ea6c700 1 -- 172.21.15.45:0/3268892523 wait complete. 2024-01-31T05:13:00.902 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (1/15)... 2024-01-31T05:13:01.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:00.978+0000 7feedb383200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-31T05:13:01.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:01.071+0000 7feedb383200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-31T05:13:01.573 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:01.158+0000 7feedb383200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-31T05:13:01.574 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:01.310+0000 7feedb383200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-31T05:13:01.574 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:01.394+0000 7feedb383200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-31T05:13:01.846 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:01.577+0000 7feedb383200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-31T05:13:01.846 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:01.759+0000 7feedb383200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-31T05:13:01.846 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:01.847+0000 7feedb383200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-31T05:13:02.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:01.942+0000 7feedb383200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-31T05:13:02.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:02 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:02.032+0000 7feedb383200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-31T05:13:03.260 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:03 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:03.125+0000 7feedb383200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-31T05:13:03.261 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:03 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:03.261+0000 7feedb383200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-31T05:13:03.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:03 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/385505602' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-01-31T05:13:03.643 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:03 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:03.491+0000 7feedb383200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-31T05:13:04.007 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:03 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:03.729+0000 7feedb383200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-31T05:13:04.007 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:04 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:03.914+0000 7feedb383200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:13:04.379 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:04 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:04.105+0000 7feedb383200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-31T05:13:04.494 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "3a1f4122-bff7-11ee-95b3-87774f69a715", 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-01-31T05:13:04.495 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-01-31T05:13:04.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-01-31T05:13:04.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-01-31T05:13:04.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-01-31T05:12:51.342330+0000", 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.309+0000 7fd8cefcd700 1 Processor -- start 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.309+0000 7fd8cefcd700 1 -- start start 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.309+0000 7fd8cefcd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 0x7fd8c8106550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.309+0000 7fd8cefcd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8106b20 con 0x7fd8c8106130 2024-01-31T05:13:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.310+0000 7fd8ccd69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 0x7fd8c8106550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.310+0000 7fd8ccd69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 0x7fd8c8106550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47342/0 (socket says 172.21.15.45:47342) 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.310+0000 7fd8ccd69700 1 -- 172.21.15.45:0/428498162 learned_addr learned my addr 172.21.15.45:0/428498162 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.310+0000 7fd8ccd69700 1 -- 172.21.15.45:0/428498162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8107330 con 0x7fd8c8106130 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.310+0000 7fd8ccd69700 1 --2- 172.21.15.45:0/428498162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 0x7fd8c8106550 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00bb40 tx=0x7fd8bc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957225569c1c7357 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.310+0000 7fd8c77fe700 1 -- 172.21.15.45:0/428498162 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8bc00dbe0 con 0x7fd8c8106130 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.310+0000 7fd8c77fe700 1 -- 172.21.15.45:0/428498162 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd8bc0044d0 con 0x7fd8c8106130 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.310+0000 7fd8c77fe700 1 -- 172.21.15.45:0/428498162 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8bc010470 con 0x7fd8c8106130 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.311+0000 7fd8cefcd700 1 -- 172.21.15.45:0/428498162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 msgr2=0x7fd8c8106550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.311+0000 7fd8cefcd700 1 --2- 172.21.15.45:0/428498162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 0x7fd8c8106550 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00bb40 tx=0x7fd8bc00be50 comp rx=0 tx=0).stop 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.311+0000 7fd8cefcd700 1 -- 172.21.15.45:0/428498162 shutdown_connections 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.311+0000 7fd8cefcd700 1 --2- 172.21.15.45:0/428498162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 0x7fd8c8106550 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.311+0000 7fd8cefcd700 1 -- 172.21.15.45:0/428498162 >> 172.21.15.45:0/428498162 conn(0x7fd8c8101710 msgr2=0x7fd8c8103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:04.500 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.311+0000 7fd8cefcd700 1 -- 172.21.15.45:0/428498162 shutdown_connections 2024-01-31T05:13:04.501 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.311+0000 7fd8cefcd700 1 -- 172.21.15.45:0/428498162 wait complete. 2024-01-31T05:13:04.501 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8cefcd700 1 Processor -- start 2024-01-31T05:13:04.501 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8cefcd700 1 -- start start 2024-01-31T05:13:04.501 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8cefcd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 0x7fd8c8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:04.501 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8cefcd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c81992f0 con 0x7fd8c8106130 2024-01-31T05:13:04.501 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8ccd69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 0x7fd8c8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:04.501 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8ccd69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 0x7fd8c8198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47348/0 (socket says 172.21.15.45:47348) 2024-01-31T05:13:04.501 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8ccd69700 1 -- 172.21.15.45:0/385505602 learned_addr learned my addr 172.21.15.45:0/385505602 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:04.501 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8ccd69700 1 -- 172.21.15.45:0/385505602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc00b7f0 con 0x7fd8c8106130 2024-01-31T05:13:04.501 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8ccd69700 1 --2- 172.21.15.45:0/385505602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 0x7fd8c8198db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00bfd0 tx=0x7fd8bc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8c5ffb700 1 -- 172.21.15.45:0/385505602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8bc01e070 con 0x7fd8c8106130 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8c5ffb700 1 -- 172.21.15.45:0/385505602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd8bc0042a0 con 0x7fd8c8106130 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8cefcd700 1 -- 172.21.15.45:0/385505602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c81994f0 con 0x7fd8c8106130 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.312+0000 7fd8c5ffb700 1 -- 172.21.15.45:0/385505602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8bc022ac0 con 0x7fd8c8106130 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.313+0000 7fd8cefcd700 1 -- 172.21.15.45:0/385505602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c8199990 con 0x7fd8c8106130 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.313+0000 7fd8c5ffb700 1 -- 172.21.15.45:0/385505602 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd8bc02c440 con 0x7fd8c8106130 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.313+0000 7fd8c5ffb700 1 -- 172.21.15.45:0/385505602 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd8bc019a70 con 0x7fd8c8106130 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.313+0000 7fd8cefcd700 1 -- 172.21.15.45:0/385505602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c8062e10 con 0x7fd8c8106130 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.316+0000 7fd8c5ffb700 1 -- 172.21.15.45:0/385505602 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7fd8bc027070 con 0x7fd8c8106130 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.390+0000 7fd8cefcd700 1 -- 172.21.15.45:0/385505602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd8c8192490 con 0x7fd8c8106130 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.390+0000 7fd8c5ffb700 1 -- 172.21.15.45:0/385505602 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fd8bc022410 con 0x7fd8c8106130 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.391+0000 7fd8cefcd700 1 -- 172.21.15.45:0/385505602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 msgr2=0x7fd8c8198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:04.502 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.391+0000 7fd8cefcd700 1 --2- 172.21.15.45:0/385505602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 0x7fd8c8198db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00bfd0 tx=0x7fd8bc003d40 comp rx=0 tx=0).stop 2024-01-31T05:13:04.503 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.391+0000 7fd8cefcd700 1 -- 172.21.15.45:0/385505602 shutdown_connections 2024-01-31T05:13:04.503 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.391+0000 7fd8cefcd700 1 --2- 172.21.15.45:0/385505602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8c8106130 0x7fd8c8198db0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:04.503 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.391+0000 7fd8cefcd700 1 -- 172.21.15.45:0/385505602 >> 172.21.15.45:0/385505602 conn(0x7fd8c8101710 msgr2=0x7fd8c818f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:04.503 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.392+0000 7fd8cefcd700 1 -- 172.21.15.45:0/385505602 shutdown_connections 2024-01-31T05:13:04.503 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:03.392+0000 7fd8cefcd700 1 -- 172.21.15.45:0/385505602 wait complete. 2024-01-31T05:13:04.503 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (2/15)... 2024-01-31T05:13:04.642 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:04 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:04.380+0000 7feedb383200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:13:04.643 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:04 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:04.469+0000 7feedb383200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-31T05:13:05.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:04 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:04.868+0000 7feedb383200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-31T05:13:06.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:05 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:05.701+0000 7feedb383200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-31T05:13:06.588 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:06.282+0000 7feedb383200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-31T05:13:06.588 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:06.381+0000 7feedb383200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-31T05:13:06.588 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:06.589+0000 7feedb383200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-31T05:13:06.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: Activating manager daemon x 2024-01-31T05:13:06.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: mgrmap e2: x(active, starting, since 0.00301427s) 2024-01-31T05:13:06.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: from='mgr.14100 172.21.15.45:0/3689172206' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-31T05:13:06.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: from='mgr.14100 172.21.15.45:0/3689172206' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-31T05:13:06.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: from='mgr.14100 172.21.15.45:0/3689172206' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-31T05:13:06.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: from='mgr.14100 172.21.15.45:0/3689172206' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:13:06.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: from='mgr.14100 172.21.15.45:0/3689172206' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-31T05:13:06.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: Manager daemon x is now available 2024-01-31T05:13:06.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: from='mgr.14100 172.21.15.45:0/3689172206' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-31T05:13:06.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: from='mgr.14100 172.21.15.45:0/3689172206' entity='mgr.x' 2024-01-31T05:13:06.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: from='mgr.14100 172.21.15.45:0/3689172206' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-31T05:13:06.868 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: from='mgr.14100 172.21.15.45:0/3689172206' entity='mgr.x' 2024-01-31T05:13:06.868 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:06 smithi045 ceph-mon[31585]: from='mgr.14100 172.21.15.45:0/3689172206' entity='mgr.x' 2024-01-31T05:13:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:13:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-01-31T05:13:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "3a1f4122-bff7-11ee-95b3-87774f69a715", 2024-01-31T05:13:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-01-31T05:13:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-01-31T05:13:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-01-31T05:13:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-01-31T05:13:07.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-01-31T05:13:07.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-01-31T05:13:07.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-01-31T05:12:51.342330+0000", 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-01-31T05:13:07.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.891+0000 7f28ae57f700 1 Processor -- start 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.891+0000 7f28ae57f700 1 -- start start 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.891+0000 7f28ae57f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 0x7f28a8105f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.891+0000 7f28ae57f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a807ae40 con 0x7f28a8105b10 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.891+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 0x7f28a8105f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.891+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 0x7f28a8105f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47450/0 (socket says 172.21.15.45:47450) 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.891+0000 7f28a7fff700 1 -- 172.21.15.45:0/2301318869 learned_addr learned my addr 172.21.15.45:0/2301318869 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.892+0000 7f28a7fff700 1 -- 172.21.15.45:0/2301318869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a8106470 con 0x7f28a8105b10 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.892+0000 7f28a7fff700 1 --2- 172.21.15.45:0/2301318869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 0x7f28a8105f30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f28980098a0 tx=0x7f2898009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60b4c47a7500f36 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.892+0000 7f28a6ffd700 1 -- 172.21.15.45:0/2301318869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2898004160 con 0x7f28a8105b10 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.892+0000 7f28a6ffd700 1 -- 172.21.15.45:0/2301318869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f28980042c0 con 0x7f28a8105b10 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.892+0000 7f28a6ffd700 1 -- 172.21.15.45:0/2301318869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2898011a00 con 0x7f28a8105b10 2024-01-31T05:13:07.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.892+0000 7f28ae57f700 1 -- 172.21.15.45:0/2301318869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 msgr2=0x7f28a8105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.892+0000 7f28ae57f700 1 --2- 172.21.15.45:0/2301318869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 0x7f28a8105f30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f28980098a0 tx=0x7f2898009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.893+0000 7f28ae57f700 1 -- 172.21.15.45:0/2301318869 shutdown_connections 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.893+0000 7f28ae57f700 1 --2- 172.21.15.45:0/2301318869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 0x7f28a8105f30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.893+0000 7f28ae57f700 1 -- 172.21.15.45:0/2301318869 >> 172.21.15.45:0/2301318869 conn(0x7f28a8101710 msgr2=0x7f28a8103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.893+0000 7f28ae57f700 1 -- 172.21.15.45:0/2301318869 shutdown_connections 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.893+0000 7f28ae57f700 1 -- 172.21.15.45:0/2301318869 wait complete. 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.893+0000 7f28ae57f700 1 Processor -- start 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.893+0000 7f28ae57f700 1 -- start start 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.893+0000 7f28ae57f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 0x7f28a819d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.893+0000 7f28ae57f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a819d6c0 con 0x7f28a8105b10 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.894+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 0x7f28a819d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.894+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 0x7f28a819d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47452/0 (socket says 172.21.15.45:47452) 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.894+0000 7f28a7fff700 1 -- 172.21.15.45:0/466945627 learned_addr learned my addr 172.21.15.45:0/466945627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.894+0000 7f28a7fff700 1 -- 172.21.15.45:0/466945627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2898009550 con 0x7f28a8105b10 2024-01-31T05:13:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.894+0000 7f28a7fff700 1 --2- 172.21.15.45:0/466945627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 0x7f28a819d180 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2898011a20 tx=0x7f2898011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.894+0000 7f28a57fa700 1 -- 172.21.15.45:0/466945627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2898003af0 con 0x7f28a8105b10 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.894+0000 7f28a57fa700 1 -- 172.21.15.45:0/466945627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2898003c50 con 0x7f28a8105b10 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.894+0000 7f28ae57f700 1 -- 172.21.15.45:0/466945627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a819d8c0 con 0x7f28a8105b10 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.894+0000 7f28a57fa700 1 -- 172.21.15.45:0/466945627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2898019760 con 0x7f28a8105b10 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.894+0000 7f28ae57f700 1 -- 172.21.15.45:0/466945627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a819dd60 con 0x7f28a8105b10 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.895+0000 7f28a57fa700 1 -- 172.21.15.45:0/466945627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 2) v1 ==== 45397+0+0 (secure 0 0 0) 0x7f2898019950 con 0x7f28a8105b10 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.895+0000 7f28ae57f700 1 -- 172.21.15.45:0/466945627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a8196df0 con 0x7f28a8105b10 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.895+0000 7f28a57fa700 1 -- 172.21.15.45:0/466945627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f289804c800 con 0x7f28a8105b10 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.897+0000 7f28a57fa700 1 -- 172.21.15.45:0/466945627 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f2898018d90 con 0x7f28a8105b10 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.970+0000 7f28ae57f700 1 -- 172.21.15.45:0/466945627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f28a8107270 con 0x7f28a8105b10 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.970+0000 7f28a57fa700 1 -- 172.21.15.45:0/466945627 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f289800ed40 con 0x7f28a8105b10 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.971+0000 7f28ae57f700 1 -- 172.21.15.45:0/466945627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 msgr2=0x7f28a819d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:07.531 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.971+0000 7f28ae57f700 1 --2- 172.21.15.45:0/466945627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 0x7f28a819d180 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2898011a20 tx=0x7f2898011b00 comp rx=0 tx=0).stop 2024-01-31T05:13:07.532 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.971+0000 7f28ae57f700 1 -- 172.21.15.45:0/466945627 shutdown_connections 2024-01-31T05:13:07.532 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.971+0000 7f28ae57f700 1 --2- 172.21.15.45:0/466945627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a8105b10 0x7f28a819d180 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:07.532 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.971+0000 7f28ae57f700 1 -- 172.21.15.45:0/466945627 >> 172.21.15.45:0/466945627 conn(0x7f28a8101710 msgr2=0x7f28a8193f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:07.532 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.971+0000 7f28ae57f700 1 -- 172.21.15.45:0/466945627 shutdown_connections 2024-01-31T05:13:07.532 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:06.971+0000 7f28ae57f700 1 -- 172.21.15.45:0/466945627 wait complete. 2024-01-31T05:13:07.532 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (3/15)... 2024-01-31T05:13:07.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:07 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/466945627' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-01-31T05:13:07.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:07 smithi045 ceph-mon[31585]: mgrmap e3: x(active, since 1.00614s) 2024-01-31T05:13:09.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:09 smithi045 ceph-mon[31585]: mgrmap e4: x(active, since 2s) 2024-01-31T05:13:10.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:13:10.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-01-31T05:13:10.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "3a1f4122-bff7-11ee-95b3-87774f69a715", 2024-01-31T05:13:10.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-01-31T05:13:10.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-01-31T05:13:10.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-01-31T05:13:10.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-01-31T05:12:51.342330+0000", 2024-01-31T05:13:10.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.948+0000 7fb045dce700 1 Processor -- start 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.948+0000 7fb045dce700 1 -- start start 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.948+0000 7fb045dce700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 0x7fb040107620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.948+0000 7fb045dce700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb040107bf0 con 0x7fb040107200 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.948+0000 7fb03f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 0x7fb040107620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.948+0000 7fb03f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 0x7fb040107620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60658/0 (socket says 172.21.15.45:60658) 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.948+0000 7fb03f7fe700 1 -- 172.21.15.45:0/2491490271 learned_addr learned my addr 172.21.15.45:0/2491490271 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.949+0000 7fb03f7fe700 1 -- 172.21.15.45:0/2491490271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb040108400 con 0x7fb040107200 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.949+0000 7fb03f7fe700 1 --2- 172.21.15.45:0/2491490271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 0x7fb040107620 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb0300098a0 tx=0x7fb030009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e04aac6b0bea9f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.949+0000 7fb03e7fc700 1 -- 172.21.15.45:0/2491490271 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb030004160 con 0x7fb040107200 2024-01-31T05:13:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.949+0000 7fb03e7fc700 1 -- 172.21.15.45:0/2491490271 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb0300042c0 con 0x7fb040107200 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.949+0000 7fb03e7fc700 1 -- 172.21.15.45:0/2491490271 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb030011a00 con 0x7fb040107200 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.950+0000 7fb045dce700 1 -- 172.21.15.45:0/2491490271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 msgr2=0x7fb040107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.950+0000 7fb045dce700 1 --2- 172.21.15.45:0/2491490271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 0x7fb040107620 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb0300098a0 tx=0x7fb030009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.950+0000 7fb045dce700 1 -- 172.21.15.45:0/2491490271 shutdown_connections 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.950+0000 7fb045dce700 1 --2- 172.21.15.45:0/2491490271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 0x7fb040107620 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.950+0000 7fb045dce700 1 -- 172.21.15.45:0/2491490271 >> 172.21.15.45:0/2491490271 conn(0x7fb0401027c0 msgr2=0x7fb040104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.950+0000 7fb045dce700 1 -- 172.21.15.45:0/2491490271 shutdown_connections 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.950+0000 7fb045dce700 1 -- 172.21.15.45:0/2491490271 wait complete. 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.950+0000 7fb045dce700 1 Processor -- start 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.951+0000 7fb045dce700 1 -- start start 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.951+0000 7fb045dce700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 0x7fb04019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.951+0000 7fb045dce700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04019b450 con 0x7fb040107200 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.951+0000 7fb03f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 0x7fb04019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:10.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.951+0000 7fb03f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 0x7fb04019af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60660/0 (socket says 172.21.15.45:60660) 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.951+0000 7fb03f7fe700 1 -- 172.21.15.45:0/3271398006 learned_addr learned my addr 172.21.15.45:0/3271398006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.951+0000 7fb03f7fe700 1 -- 172.21.15.45:0/3271398006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb030009550 con 0x7fb040107200 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.951+0000 7fb03f7fe700 1 --2- 172.21.15.45:0/3271398006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 0x7fb04019af10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb030009fb0 tx=0x7fb030011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.952+0000 7fb03cff9700 1 -- 172.21.15.45:0/3271398006 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb030003a40 con 0x7fb040107200 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.952+0000 7fb045dce700 1 -- 172.21.15.45:0/3271398006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04019b650 con 0x7fb040107200 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.952+0000 7fb03cff9700 1 -- 172.21.15.45:0/3271398006 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb030003ba0 con 0x7fb040107200 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.952+0000 7fb03cff9700 1 -- 172.21.15.45:0/3271398006 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb03001a750 con 0x7fb040107200 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.952+0000 7fb045dce700 1 -- 172.21.15.45:0/3271398006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04019baf0 con 0x7fb040107200 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.952+0000 7fb03cff9700 1 -- 172.21.15.45:0/3271398006 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 45630+0+0 (secure 0 0 0) 0x7fb03001a9d0 con 0x7fb040107200 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.953+0000 7fb03cff9700 1 --2- 172.21.15.45:0/3271398006 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fb0280389a0 0x7fb02803ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.953+0000 7fb045dce700 1 -- 172.21.15.45:0/3271398006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04010b630 con 0x7fb040107200 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.953+0000 7fb03cff9700 1 -- 172.21.15.45:0/3271398006 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb03004c800 con 0x7fb040107200 2024-01-31T05:13:10.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.953+0000 7fb03effd700 1 --2- 172.21.15.45:0/3271398006 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fb0280389a0 0x7fb02803ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.953+0000 7fb03effd700 1 --2- 172.21.15.45:0/3271398006 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fb0280389a0 0x7fb02803ae60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb034006fd0 tx=0x7fb034006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:09.958+0000 7fb03cff9700 1 -- 172.21.15.45:0/3271398006 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb030018db0 con 0x7fb040107200 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:10.240+0000 7fb045dce700 1 -- 172.21.15.45:0/3271398006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb04019e7a0 con 0x7fb040107200 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:10.241+0000 7fb03cff9700 1 -- 172.21.15.45:0/3271398006 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fb030018350 con 0x7fb040107200 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:10.243+0000 7fb045dce700 1 -- 172.21.15.45:0/3271398006 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fb0280389a0 msgr2=0x7fb02803ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:10.243+0000 7fb045dce700 1 --2- 172.21.15.45:0/3271398006 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fb0280389a0 0x7fb02803ae60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb034006fd0 tx=0x7fb034006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:10.243+0000 7fb045dce700 1 -- 172.21.15.45:0/3271398006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 msgr2=0x7fb04019af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:10.243+0000 7fb045dce700 1 --2- 172.21.15.45:0/3271398006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 0x7fb04019af10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb030009fb0 tx=0x7fb030011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:10.243+0000 7fb045dce700 1 -- 172.21.15.45:0/3271398006 shutdown_connections 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:10.243+0000 7fb045dce700 1 --2- 172.21.15.45:0/3271398006 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fb0280389a0 0x7fb02803ae60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:10.243+0000 7fb045dce700 1 --2- 172.21.15.45:0/3271398006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb040107200 0x7fb04019af10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:10.243+0000 7fb045dce700 1 -- 172.21.15.45:0/3271398006 >> 172.21.15.45:0/3271398006 conn(0x7fb0401027c0 msgr2=0x7fb040103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:10.244+0000 7fb045dce700 1 -- 172.21.15.45:0/3271398006 shutdown_connections 2024-01-31T05:13:10.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:10.244+0000 7fb045dce700 1 -- 172.21.15.45:0/3271398006 wait complete. 2024-01-31T05:13:10.790 INFO:teuthology.orchestra.run.smithi045.stdout:mgr is available 2024-01-31T05:13:10.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:10 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3271398006' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-01-31T05:13:11.949 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:11 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2607731535' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-01-31T05:13:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:13:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-01-31T05:13:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:13:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-01-31T05:13:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-01-31T05:13:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-01-31T05:13:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.190+0000 7fde34dcf700 1 Processor -- start 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.190+0000 7fde34dcf700 1 -- start start 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.190+0000 7fde34dcf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 0x7fde3007b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.190+0000 7fde34dcf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3007b850 con 0x7fde3007ce10 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.190+0000 7fde2e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 0x7fde3007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.190+0000 7fde2e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 0x7fde3007b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60674/0 (socket says 172.21.15.45:60674) 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.190+0000 7fde2e59c700 1 -- 172.21.15.45:0/1749240047 learned_addr learned my addr 172.21.15.45:0/1749240047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.190+0000 7fde2e59c700 1 -- 172.21.15.45:0/1749240047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3007b990 con 0x7fde3007ce10 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.190+0000 7fde2e59c700 1 --2- 172.21.15.45:0/1749240047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 0x7fde3007b310 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde200098a0 tx=0x7fde20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d5701a1c121ccd server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.191+0000 7fde2d59a700 1 -- 172.21.15.45:0/1749240047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde20004160 con 0x7fde3007ce10 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.191+0000 7fde2d59a700 1 -- 172.21.15.45:0/1749240047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fde200042c0 con 0x7fde3007ce10 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.191+0000 7fde2d59a700 1 -- 172.21.15.45:0/1749240047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde20011a00 con 0x7fde3007ce10 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.191+0000 7fde34dcf700 1 -- 172.21.15.45:0/1749240047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 msgr2=0x7fde3007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.191+0000 7fde34dcf700 1 --2- 172.21.15.45:0/1749240047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 0x7fde3007b310 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde200098a0 tx=0x7fde20009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.191+0000 7fde34dcf700 1 -- 172.21.15.45:0/1749240047 shutdown_connections 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.191+0000 7fde34dcf700 1 --2- 172.21.15.45:0/1749240047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 0x7fde3007b310 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.191+0000 7fde34dcf700 1 -- 172.21.15.45:0/1749240047 >> 172.21.15.45:0/1749240047 conn(0x7fde301028a0 msgr2=0x7fde30104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.191+0000 7fde34dcf700 1 -- 172.21.15.45:0/1749240047 shutdown_connections 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.191+0000 7fde34dcf700 1 -- 172.21.15.45:0/1749240047 wait complete. 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.192+0000 7fde34dcf700 1 Processor -- start 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.192+0000 7fde34dcf700 1 -- start start 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.192+0000 7fde34dcf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 0x7fde3019d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.192+0000 7fde34dcf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3019d610 con 0x7fde3007ce10 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.192+0000 7fde2e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 0x7fde3019d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.192+0000 7fde2e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 0x7fde3019d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60684/0 (socket says 172.21.15.45:60684) 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.193+0000 7fde2e59c700 1 -- 172.21.15.45:0/2607731535 learned_addr learned my addr 172.21.15.45:0/2607731535 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.193+0000 7fde2e59c700 1 -- 172.21.15.45:0/2607731535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde20009550 con 0x7fde3007ce10 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.193+0000 7fde2e59c700 1 --2- 172.21.15.45:0/2607731535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 0x7fde3019d0d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fde20009130 tx=0x7fde20011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.193+0000 7fde1f7fe700 1 -- 172.21.15.45:0/2607731535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde20003b70 con 0x7fde3007ce10 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.193+0000 7fde1f7fe700 1 -- 172.21.15.45:0/2607731535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fde20003cd0 con 0x7fde3007ce10 2024-01-31T05:13:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.193+0000 7fde1f7fe700 1 -- 172.21.15.45:0/2607731535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde200197d0 con 0x7fde3007ce10 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.193+0000 7fde34dcf700 1 -- 172.21.15.45:0/2607731535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3019d810 con 0x7fde3007ce10 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.193+0000 7fde34dcf700 1 -- 172.21.15.45:0/2607731535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3019dcb0 con 0x7fde3007ce10 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.194+0000 7fde1f7fe700 1 -- 172.21.15.45:0/2607731535 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 45630+0+0 (secure 0 0 0) 0x7fde200199c0 con 0x7fde3007ce10 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.194+0000 7fde1f7fe700 1 --2- 172.21.15.45:0/2607731535 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fde18038a40 0x7fde1803af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.194+0000 7fde1f7fe700 1 -- 172.21.15.45:0/2607731535 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fde2004cd30 con 0x7fde3007ce10 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.194+0000 7fde34dcf700 1 -- 172.21.15.45:0/2607731535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3010d550 con 0x7fde3007ce10 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.194+0000 7fde2dd9b700 1 --2- 172.21.15.45:0/2607731535 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fde18038a40 0x7fde1803af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.195+0000 7fde2dd9b700 1 --2- 172.21.15.45:0/2607731535 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fde18038a40 0x7fde1803af00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde24006fd0 tx=0x7fde24006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.199+0000 7fde1f7fe700 1 -- 172.21.15.45:0/2607731535 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fde20018db0 con 0x7fde3007ce10 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.396+0000 7fde34dcf700 1 -- 172.21.15.45:0/2607731535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fde3010d710 con 0x7fde3007ce10 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.397+0000 7fde1f7fe700 1 -- 172.21.15.45:0/2607731535 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fde20018350 con 0x7fde3007ce10 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.399+0000 7fde34dcf700 1 -- 172.21.15.45:0/2607731535 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fde18038a40 msgr2=0x7fde1803af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.399+0000 7fde34dcf700 1 --2- 172.21.15.45:0/2607731535 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fde18038a40 0x7fde1803af00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde24006fd0 tx=0x7fde24006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.399+0000 7fde34dcf700 1 -- 172.21.15.45:0/2607731535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 msgr2=0x7fde3019d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:11.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.399+0000 7fde34dcf700 1 --2- 172.21.15.45:0/2607731535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 0x7fde3019d0d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fde20009130 tx=0x7fde20011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:11.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.399+0000 7fde34dcf700 1 -- 172.21.15.45:0/2607731535 shutdown_connections 2024-01-31T05:13:11.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.399+0000 7fde34dcf700 1 --2- 172.21.15.45:0/2607731535 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fde18038a40 0x7fde1803af00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:11.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.399+0000 7fde34dcf700 1 --2- 172.21.15.45:0/2607731535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3007ce10 0x7fde3019d0d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:11.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.399+0000 7fde34dcf700 1 -- 172.21.15.45:0/2607731535 >> 172.21.15.45:0/2607731535 conn(0x7fde301028a0 msgr2=0x7fde30103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:11.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.400+0000 7fde34dcf700 1 -- 172.21.15.45:0/2607731535 shutdown_connections 2024-01-31T05:13:11.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:11.400+0000 7fde34dcf700 1 -- 172.21.15.45:0/2607731535 wait complete. 2024-01-31T05:13:11.964 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling cephadm module... 2024-01-31T05:13:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:13 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2345400615' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-01-31T05:13:13.643 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:13 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: ignoring --setuser ceph since I am not root 2024-01-31T05:13:13.643 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:13 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: ignoring --setgroup ceph since I am not root 2024-01-31T05:13:13.948 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:13 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:13.921+0000 7f5bccd89200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-31T05:13:13.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.339+0000 7f09ce10c700 1 Processor -- start 2024-01-31T05:13:13.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.339+0000 7f09ce10c700 1 -- start start 2024-01-31T05:13:13.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.339+0000 7f09ce10c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 0x7f09c81076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:13.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.339+0000 7f09ce10c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c8107c70 con 0x7f09c8107280 2024-01-31T05:13:13.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.339+0000 7f09c77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 0x7f09c81076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:13.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.339+0000 7f09c77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 0x7f09c81076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60688/0 (socket says 172.21.15.45:60688) 2024-01-31T05:13:13.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.339+0000 7f09c77fe700 1 -- 172.21.15.45:0/1858669544 learned_addr learned my addr 172.21.15.45:0/1858669544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:13.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.340+0000 7f09c77fe700 1 -- 172.21.15.45:0/1858669544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c81083e0 con 0x7f09c8107280 2024-01-31T05:13:13.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.340+0000 7f09c77fe700 1 --2- 172.21.15.45:0/1858669544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 0x7f09c81076a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f09b80098a0 tx=0x7f09b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded1a5f4b089ccde server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.340+0000 7f09c67fc700 1 -- 172.21.15.45:0/1858669544 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09b8004160 con 0x7f09c8107280 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.340+0000 7f09c67fc700 1 -- 172.21.15.45:0/1858669544 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f09b80042c0 con 0x7f09c8107280 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.340+0000 7f09c67fc700 1 -- 172.21.15.45:0/1858669544 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09b8011a00 con 0x7f09c8107280 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.341+0000 7f09ce10c700 1 -- 172.21.15.45:0/1858669544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 msgr2=0x7f09c81076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.341+0000 7f09ce10c700 1 --2- 172.21.15.45:0/1858669544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 0x7f09c81076a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f09b80098a0 tx=0x7f09b8009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.341+0000 7f09ce10c700 1 -- 172.21.15.45:0/1858669544 shutdown_connections 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.341+0000 7f09ce10c700 1 --2- 172.21.15.45:0/1858669544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 0x7f09c81076a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.341+0000 7f09ce10c700 1 -- 172.21.15.45:0/1858669544 >> 172.21.15.45:0/1858669544 conn(0x7f09c8102880 msgr2=0x7f09c8104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.341+0000 7f09ce10c700 1 -- 172.21.15.45:0/1858669544 shutdown_connections 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.341+0000 7f09ce10c700 1 -- 172.21.15.45:0/1858669544 wait complete. 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.341+0000 7f09ce10c700 1 Processor -- start 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.341+0000 7f09ce10c700 1 -- start start 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.342+0000 7f09ce10c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 0x7f09c819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:13.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.342+0000 7f09ce10c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c819b480 con 0x7f09c8107280 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.342+0000 7f09c77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 0x7f09c819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.342+0000 7f09c77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 0x7f09c819af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60690/0 (socket says 172.21.15.45:60690) 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.342+0000 7f09c77fe700 1 -- 172.21.15.45:0/2345400615 learned_addr learned my addr 172.21.15.45:0/2345400615 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.342+0000 7f09c77fe700 1 -- 172.21.15.45:0/2345400615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b8009550 con 0x7f09c8107280 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.342+0000 7f09c77fe700 1 --2- 172.21.15.45:0/2345400615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 0x7f09c819af40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009130 tx=0x7f09b8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.342+0000 7f09c4ff9700 1 -- 172.21.15.45:0/2345400615 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09b8003ab0 con 0x7f09c8107280 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.342+0000 7f09c4ff9700 1 -- 172.21.15.45:0/2345400615 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f09b8003c10 con 0x7f09c8107280 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.342+0000 7f09ce10c700 1 -- 172.21.15.45:0/2345400615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c819b680 con 0x7f09c8107280 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.342+0000 7f09c4ff9700 1 -- 172.21.15.45:0/2345400615 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09b80247d0 con 0x7f09c8107280 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.342+0000 7f09ce10c700 1 -- 172.21.15.45:0/2345400615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c819bb20 con 0x7f09c8107280 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.343+0000 7f09c4ff9700 1 -- 172.21.15.45:0/2345400615 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 45630+0+0 (secure 0 0 0) 0x7f09b802d3f0 con 0x7f09c8107280 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.343+0000 7f09c4ff9700 1 --2- 172.21.15.45:0/2345400615 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f09b0038a40 0x7f09b003af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.343+0000 7f09c4ff9700 1 -- 172.21.15.45:0/2345400615 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f09b804ca70 con 0x7f09c8107280 2024-01-31T05:13:13.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.343+0000 7f09c6ffd700 1 --2- 172.21.15.45:0/2345400615 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f09b0038a40 0x7f09b003af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.343+0000 7f09ce10c700 1 -- 172.21.15.45:0/2345400615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c8062e10 con 0x7f09c8107280 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.344+0000 7f09c6ffd700 1 --2- 172.21.15.45:0/2345400615 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f09b0038a40 0x7f09b003af00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09bc006fd0 tx=0x7f09bc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.348+0000 7f09c4ff9700 1 -- 172.21.15.45:0/2345400615 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f09b8010970 con 0x7f09c8107280 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:12.605+0000 7f09ce10c700 1 -- 172.21.15.45:0/2345400615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f09c819e630 con 0x7f09c8107280 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:13.356+0000 7f09c4ff9700 1 -- 172.21.15.45:0/2345400615 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f09b8020220 con 0x7f09c8107280 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:13.358+0000 7f09c4ff9700 1 -- 172.21.15.45:0/2345400615 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 5) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f09b8016d60 con 0x7f09c8107280 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:13.360+0000 7f09ce10c700 1 -- 172.21.15.45:0/2345400615 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f09b0038a40 msgr2=0x7f09b003af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:13.360+0000 7f09ce10c700 1 --2- 172.21.15.45:0/2345400615 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f09b0038a40 0x7f09b003af00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09bc006fd0 tx=0x7f09bc006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:13.360+0000 7f09ce10c700 1 -- 172.21.15.45:0/2345400615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 msgr2=0x7f09c819af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:13.360+0000 7f09ce10c700 1 --2- 172.21.15.45:0/2345400615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 0x7f09c819af40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009130 tx=0x7f09b8011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:13.360+0000 7f09ce10c700 1 -- 172.21.15.45:0/2345400615 shutdown_connections 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:13.360+0000 7f09ce10c700 1 --2- 172.21.15.45:0/2345400615 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f09b0038a40 0x7f09b003af00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:13.360+0000 7f09ce10c700 1 --2- 172.21.15.45:0/2345400615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c8107280 0x7f09c819af40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:13.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:13.360+0000 7f09ce10c700 1 -- 172.21.15.45:0/2345400615 >> 172.21.15.45:0/2345400615 conn(0x7f09c8102880 msgr2=0x7f09c8103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:13.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:13.360+0000 7f09ce10c700 1 -- 172.21.15.45:0/2345400615 shutdown_connections 2024-01-31T05:13:13.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:13.360+0000 7f09ce10c700 1 -- 172.21.15.45:0/2345400615 wait complete. 2024-01-31T05:13:14.234 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:14 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:14.019+0000 7f5bccd89200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-31T05:13:14.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:14 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2345400615' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-01-31T05:13:14.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:14 smithi045 ceph-mon[31585]: mgrmap e5: x(active, since 6s) 2024-01-31T05:13:14.868 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:14 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:14.687+0000 7f5bccd89200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-31T05:13:14.868 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:14 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:14.780+0000 7f5bccd89200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-31T05:13:14.869 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:14 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:14.869+0000 7f5bccd89200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-31T05:13:15.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:15 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:15.023+0000 7f5bccd89200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-31T05:13:15.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:15 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:15.107+0000 7f5bccd89200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-31T05:13:15.238 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-01-31T05:13:15.238 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-01-31T05:13:15.238 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.334+0000 7fcfac056700 1 Processor -- start 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.334+0000 7fcfac056700 1 -- start start 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.334+0000 7fcfac056700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 0x7fcfa4105930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.334+0000 7fcfac056700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4105f00 con 0x7fcfa4105510 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.334+0000 7fcfa9df2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 0x7fcfa4105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.334+0000 7fcfa9df2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 0x7fcfa4105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60716/0 (socket says 172.21.15.45:60716) 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.334+0000 7fcfa9df2700 1 -- 172.21.15.45:0/2672371904 learned_addr learned my addr 172.21.15.45:0/2672371904 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.334+0000 7fcfa9df2700 1 -- 172.21.15.45:0/2672371904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa41084a0 con 0x7fcfa4105510 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.335+0000 7fcfa9df2700 1 --2- 172.21.15.45:0/2672371904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 0x7fcfa4105930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00098a0 tx=0x7fcfa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181d8fb21913f9d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.335+0000 7fcfa8df0700 1 -- 172.21.15.45:0/2672371904 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfa0004160 con 0x7fcfa4105510 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.335+0000 7fcfa8df0700 1 -- 172.21.15.45:0/2672371904 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfa00042c0 con 0x7fcfa4105510 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.335+0000 7fcfa8df0700 1 -- 172.21.15.45:0/2672371904 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfa0011a00 con 0x7fcfa4105510 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.335+0000 7fcfac056700 1 -- 172.21.15.45:0/2672371904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 msgr2=0x7fcfa4105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.335+0000 7fcfac056700 1 --2- 172.21.15.45:0/2672371904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 0x7fcfa4105930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00098a0 tx=0x7fcfa0009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.336+0000 7fcfac056700 1 -- 172.21.15.45:0/2672371904 shutdown_connections 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.336+0000 7fcfac056700 1 --2- 172.21.15.45:0/2672371904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 0x7fcfa4105930 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.336+0000 7fcfac056700 1 -- 172.21.15.45:0/2672371904 >> 172.21.15.45:0/2672371904 conn(0x7fcfa4100ee0 msgr2=0x7fcfa4103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.336+0000 7fcfac056700 1 -- 172.21.15.45:0/2672371904 shutdown_connections 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.336+0000 7fcfac056700 1 -- 172.21.15.45:0/2672371904 wait complete. 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.336+0000 7fcfac056700 1 Processor -- start 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.336+0000 7fcfac056700 1 -- start start 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.336+0000 7fcfac056700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 0x7fcfa4107d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.336+0000 7fcfac056700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4108260 con 0x7fcfa4105510 2024-01-31T05:13:15.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.337+0000 7fcfa9df2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 0x7fcfa4107d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.337+0000 7fcfa9df2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 0x7fcfa4107d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60718/0 (socket says 172.21.15.45:60718) 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.337+0000 7fcfa9df2700 1 -- 172.21.15.45:0/2700755215 learned_addr learned my addr 172.21.15.45:0/2700755215 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.337+0000 7fcfa9df2700 1 -- 172.21.15.45:0/2700755215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa0009550 con 0x7fcfa4105510 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.337+0000 7fcfa9df2700 1 --2- 172.21.15.45:0/2700755215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 0x7fcfa4107d20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0011710 tx=0x7fcfa0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.337+0000 7fcf9affd700 1 -- 172.21.15.45:0/2700755215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfa0003aa0 con 0x7fcfa4105510 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.337+0000 7fcfac056700 1 -- 172.21.15.45:0/2700755215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa4106450 con 0x7fcfa4105510 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.337+0000 7fcf9affd700 1 -- 172.21.15.45:0/2700755215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfa0003c00 con 0x7fcfa4105510 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.337+0000 7fcf9affd700 1 -- 172.21.15.45:0/2700755215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfa0024800 con 0x7fcfa4105510 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.337+0000 7fcfac056700 1 -- 172.21.15.45:0/2700755215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa41068f0 con 0x7fcfa4105510 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.338+0000 7fcf9affd700 1 -- 172.21.15.45:0/2700755215 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 45641+0+0 (secure 0 0 0) 0x7fcfa0024a30 con 0x7fcfa4105510 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.338+0000 7fcf9affd700 1 --2- 172.21.15.45:0/2700755215 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fcf90038a40 0x7fcf9003af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.338+0000 7fcf9affd700 1 -- 172.21.15.45:0/2700755215 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcfa004c660 con 0x7fcfa4105510 2024-01-31T05:13:15.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.338+0000 7fcfa95f1700 1 -- 172.21.15.45:0/2700755215 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fcf90038a40 msgr2=0x7fcf9003af00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2712881513 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.338+0000 7fcfa95f1700 1 --2- 172.21.15.45:0/2700755215 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fcf90038a40 0x7fcf9003af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.338+0000 7fcfac056700 1 -- 172.21.15.45:0/2700755215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa4062e10 con 0x7fcfa4105510 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.343+0000 7fcf9affd700 1 -- 172.21.15.45:0/2700755215 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fcfa0024ce0 con 0x7fcfa4105510 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.539+0000 7fcfa95f1700 1 -- 172.21.15.45:0/2700755215 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fcf90038a40 msgr2=0x7fcf9003af00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2712881513 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.539+0000 7fcfa95f1700 1 --2- 172.21.15.45:0/2700755215 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fcf90038a40 0x7fcf9003af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.600+0000 7fcfac056700 1 -- 172.21.15.45:0/2700755215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fcfa4106f80 con 0x7fcfa4105510 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.601+0000 7fcf9affd700 1 -- 172.21.15.45:0/2700755215 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fcfa002c350 con 0x7fcfa4105510 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.603+0000 7fcfac056700 1 -- 172.21.15.45:0/2700755215 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fcf90038a40 msgr2=0x7fcf9003af00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.603+0000 7fcfac056700 1 --2- 172.21.15.45:0/2700755215 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fcf90038a40 0x7fcf9003af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.603+0000 7fcfac056700 1 -- 172.21.15.45:0/2700755215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 msgr2=0x7fcfa4107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.603+0000 7fcfac056700 1 --2- 172.21.15.45:0/2700755215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 0x7fcfa4107d20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0011710 tx=0x7fcfa0011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.603+0000 7fcfac056700 1 -- 172.21.15.45:0/2700755215 shutdown_connections 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.603+0000 7fcfac056700 1 --2- 172.21.15.45:0/2700755215 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7fcf90038a40 0x7fcf9003af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.603+0000 7fcfac056700 1 --2- 172.21.15.45:0/2700755215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa4105510 0x7fcfa4107d20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.603+0000 7fcfac056700 1 -- 172.21.15.45:0/2700755215 >> 172.21.15.45:0/2700755215 conn(0x7fcfa4100ee0 msgr2=0x7fcfa4101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:15.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.603+0000 7fcfac056700 1 -- 172.21.15.45:0/2700755215 shutdown_connections 2024-01-31T05:13:15.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:14.603+0000 7fcfac056700 1 -- 172.21.15.45:0/2700755215 wait complete. 2024-01-31T05:13:15.243 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-01-31T05:13:15.243 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 5... 2024-01-31T05:13:15.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:15 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2700755215' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-01-31T05:13:15.566 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:15 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:15.292+0000 7f5bccd89200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-31T05:13:15.566 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:15 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:15.479+0000 7f5bccd89200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-31T05:13:15.872 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:15 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:15.567+0000 7f5bccd89200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-31T05:13:15.872 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:15 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:15.667+0000 7f5bccd89200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-31T05:13:15.872 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:15 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:15.770+0000 7f5bccd89200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-31T05:13:17.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:16 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:16.848+0000 7f5bccd89200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-31T05:13:17.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:16 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:16.981+0000 7f5bccd89200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-31T05:13:17.593 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:17 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:17.180+0000 7f5bccd89200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-31T05:13:17.593 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:17 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:17.410+0000 7f5bccd89200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-31T05:13:17.594 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:17 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:17.593+0000 7f5bccd89200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:13:17.893 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:17 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:17.782+0000 7f5bccd89200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-31T05:13:18.392 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:18 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:18.056+0000 7f5bccd89200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:13:18.393 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:18 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:18.145+0000 7f5bccd89200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-31T05:13:18.892 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:18 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:18.542+0000 7f5bccd89200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-31T05:13:19.643 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:19 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:19.355+0000 7f5bccd89200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-31T05:13:20.235 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:19 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:19.931+0000 7f5bccd89200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-31T05:13:20.235 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:20.031+0000 7f5bccd89200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-31T05:13:20.235 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:20.234+0000 7f5bccd89200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-31T05:13:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: Active manager daemon x restarted 2024-01-31T05:13:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: Activating manager daemon x 2024-01-31T05:13:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: osdmap e2: 0 total, 0 up, 0 in 2024-01-31T05:13:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: mgrmap e6: x(active, starting, since 0.00331488s) 2024-01-31T05:13:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:13:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-31T05:13:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-31T05:13:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-31T05:13:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-31T05:13:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: Manager daemon x is now available 2024-01-31T05:13:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:20.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:20.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:13:20.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:20 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:13:21.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:21 smithi045 ceph-mon[31585]: Found migration_current of "None". Setting to last migration. 2024-01-31T05:13:21.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:21 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:13:21.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:21 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-31T05:13:21.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:21 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-31T05:13:21.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:21 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:21.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:21 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:21.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:21 smithi045 ceph-mon[31585]: mgrmap e7: x(active, since 1.00503s) 2024-01-31T05:13:21.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-01-31T05:13:21.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-01-31T05:13:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-01-31T05:13:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-01-31T05:13:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.667+0000 7f86230ef700 1 Processor -- start 2024-01-31T05:13:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.667+0000 7f86230ef700 1 -- start start 2024-01-31T05:13:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.667+0000 7f86230ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 0x7f861c10a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.667+0000 7f86230ef700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c07ae80 con 0x7f861c107d10 2024-01-31T05:13:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.667+0000 7f8620e8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 0x7f861c10a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.667+0000 7f8620e8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 0x7f861c10a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60732/0 (socket says 172.21.15.45:60732) 2024-01-31T05:13:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.667+0000 7f8620e8b700 1 -- 172.21.15.45:0/4014569777 learned_addr learned my addr 172.21.15.45:0/4014569777 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.668+0000 7f8620e8b700 1 -- 172.21.15.45:0/4014569777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c10a640 con 0x7f861c107d10 2024-01-31T05:13:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.668+0000 7f8620e8b700 1 --2- 172.21.15.45:0/4014569777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 0x7f861c10a100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86100098a0 tx=0x7f8610009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c614eeff785d3007 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.668+0000 7f861b7fe700 1 -- 172.21.15.45:0/4014569777 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8610004160 con 0x7f861c107d10 2024-01-31T05:13:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.668+0000 7f861b7fe700 1 -- 172.21.15.45:0/4014569777 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f86100042c0 con 0x7f861c107d10 2024-01-31T05:13:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.668+0000 7f861b7fe700 1 -- 172.21.15.45:0/4014569777 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8610011a00 con 0x7f861c107d10 2024-01-31T05:13:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.669+0000 7f86230ef700 1 -- 172.21.15.45:0/4014569777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 msgr2=0x7f861c10a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.669+0000 7f86230ef700 1 --2- 172.21.15.45:0/4014569777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 0x7f861c10a100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86100098a0 tx=0x7f8610009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.669+0000 7f86230ef700 1 -- 172.21.15.45:0/4014569777 shutdown_connections 2024-01-31T05:13:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.669+0000 7f86230ef700 1 --2- 172.21.15.45:0/4014569777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 0x7f861c10a100 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.669+0000 7f86230ef700 1 -- 172.21.15.45:0/4014569777 >> 172.21.15.45:0/4014569777 conn(0x7f861c101740 msgr2=0x7f861c103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.669+0000 7f86230ef700 1 -- 172.21.15.45:0/4014569777 shutdown_connections 2024-01-31T05:13:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.669+0000 7f86230ef700 1 -- 172.21.15.45:0/4014569777 wait complete. 2024-01-31T05:13:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.669+0000 7f86230ef700 1 Processor -- start 2024-01-31T05:13:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.669+0000 7f86230ef700 1 -- start start 2024-01-31T05:13:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.669+0000 7f86230ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 0x7f861c107540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.669+0000 7f86230ef700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c107a80 con 0x7f861c107d10 2024-01-31T05:13:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.670+0000 7f8620e8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 0x7f861c107540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.670+0000 7f8620e8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 0x7f861c107540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60744/0 (socket says 172.21.15.45:60744) 2024-01-31T05:13:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.670+0000 7f8620e8b700 1 -- 172.21.15.45:0/3200712411 learned_addr learned my addr 172.21.15.45:0/3200712411 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.670+0000 7f8620e8b700 1 -- 172.21.15.45:0/3200712411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8610009550 con 0x7f861c107d10 2024-01-31T05:13:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.670+0000 7f8620e8b700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 0x7f861c107540 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8610009fb0 tx=0x7f8610011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.670+0000 7f8619ffb700 1 -- 172.21.15.45:0/3200712411 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8610003a40 con 0x7f861c107d10 2024-01-31T05:13:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.670+0000 7f8619ffb700 1 -- 172.21.15.45:0/3200712411 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8610003ba0 con 0x7f861c107d10 2024-01-31T05:13:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.670+0000 7f86230ef700 1 -- 172.21.15.45:0/3200712411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861c105cc0 con 0x7f861c107d10 2024-01-31T05:13:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.670+0000 7f8619ffb700 1 -- 172.21.15.45:0/3200712411 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f861001a750 con 0x7f861c107d10 2024-01-31T05:13:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.670+0000 7f86230ef700 1 -- 172.21.15.45:0/3200712411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861c106160 con 0x7f861c107d10 2024-01-31T05:13:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.671+0000 7f8619ffb700 1 -- 172.21.15.45:0/3200712411 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f861001a9d0 con 0x7f861c107d10 2024-01-31T05:13:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.671+0000 7f8619ffb700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 0x7f860c03aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.671+0000 7f86230ef700 1 -- 172.21.15.45:0/3200712411 --> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f861c062e10 con 0x7f860c0389f0 2024-01-31T05:13:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.672+0000 7f8619ffb700 1 -- 172.21.15.45:0/3200712411 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f861004d6f0 con 0x7f861c107d10 2024-01-31T05:13:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.671+0000 7f861bfff700 1 -- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 msgr2=0x7f860c03aeb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2712881513 2024-01-31T05:13:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.672+0000 7f861bfff700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 0x7f860c03aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-31T05:13:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.873+0000 7f861bfff700 1 -- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 msgr2=0x7f860c03aeb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2712881513 2024-01-31T05:13:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:15.873+0000 7f861bfff700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 0x7f860c03aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-31T05:13:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:16.274+0000 7f861bfff700 1 -- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 msgr2=0x7f860c03aeb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2712881513 2024-01-31T05:13:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:16.274+0000 7f861bfff700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 0x7f860c03aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-31T05:13:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:17.075+0000 7f861bfff700 1 -- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 msgr2=0x7f860c03aeb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2712881513 2024-01-31T05:13:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:17.075+0000 7f861bfff700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 0x7f860c03aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-31T05:13:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:18.675+0000 7f861bfff700 1 -- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 msgr2=0x7f860c03aeb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2712881513 2024-01-31T05:13:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:18.675+0000 7f861bfff700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 0x7f860c03aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-01-31T05:13:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:20.249+0000 7f8619ffb700 1 -- 172.21.15.45:0/3200712411 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 6) v1 ==== 45408+0+0 (secure 0 0 0) 0x7f861002d9b0 con 0x7f861c107d10 2024-01-31T05:13:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:20.249+0000 7f8619ffb700 1 -- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 msgr2=0x7f860c03aeb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:13:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:20.249+0000 7f8619ffb700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 0x7f860c03aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:21.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.249+0000 7f8619ffb700 1 -- 172.21.15.45:0/3200712411 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 7) v1 ==== 45535+0+0 (secure 0 0 0) 0x7f861004e530 con 0x7f861c107d10 2024-01-31T05:13:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.250+0000 7f8619ffb700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f860c03bfe0 0x7f860c03e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.250+0000 7f8619ffb700 1 -- 172.21.15.45:0/3200712411 --> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f861c062e10 con 0x7f860c03bfe0 2024-01-31T05:13:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.250+0000 7f861bfff700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f860c03bfe0 0x7f860c03e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.251+0000 7f861bfff700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f860c03bfe0 0x7f860c03e3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8608003a10 tx=0x7f8608009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.252+0000 7f8619ffb700 1 -- 172.21.15.45:0/3200712411 <== mgr.14120 v2:172.21.15.45:6800/4050317147 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f861c062e10 con 0x7f860c03bfe0 2024-01-31T05:13:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.258+0000 7f86230ef700 1 -- 172.21.15.45:0/3200712411 --> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f861c10b370 con 0x7f860c03bfe0 2024-01-31T05:13:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.258+0000 7f8619ffb700 1 -- 172.21.15.45:0/3200712411 <== mgr.14120 v2:172.21.15.45:6800/4050317147 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f861c10b370 con 0x7f860c03bfe0 2024-01-31T05:13:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.259+0000 7f86230ef700 1 -- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f860c03bfe0 msgr2=0x7f860c03e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.259+0000 7f86230ef700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f860c03bfe0 0x7f860c03e3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8608003a10 tx=0x7f8608009290 comp rx=0 tx=0).stop 2024-01-31T05:13:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.259+0000 7f86230ef700 1 -- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 msgr2=0x7f861c107540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.259+0000 7f86230ef700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 0x7f861c107540 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8610009fb0 tx=0x7f8610011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.259+0000 7f86230ef700 1 -- 172.21.15.45:0/3200712411 shutdown_connections 2024-01-31T05:13:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.259+0000 7f86230ef700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f860c03bfe0 0x7f860c03e3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.259+0000 7f86230ef700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f861c107d10 0x7f861c107540 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.259+0000 7f86230ef700 1 --2- 172.21.15.45:0/3200712411 >> [v2:172.21.15.45:6800/2712881513,v1:172.21.15.45:6801/2712881513] conn(0x7f860c0389f0 0x7f860c03aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.259+0000 7f86230ef700 1 -- 172.21.15.45:0/3200712411 >> 172.21.15.45:0/3200712411 conn(0x7f861c101740 msgr2=0x7f861c102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.259+0000 7f86230ef700 1 -- 172.21.15.45:0/3200712411 shutdown_connections 2024-01-31T05:13:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:21.259+0000 7f86230ef700 1 -- 172.21.15.45:0/3200712411 wait complete. 2024-01-31T05:13:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 5 is available 2024-01-31T05:13:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:Setting orchestrator backend to cephadm... 2024-01-31T05:13:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:22 smithi045 ceph-mon[31585]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-01-31T05:13:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:22 smithi045 ceph-mon[31585]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-01-31T05:13:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:22 smithi045 ceph-mon[31585]: [31/Jan/2024:05:13:21] ENGINE Bus STARTING 2024-01-31T05:13:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:22 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:13:22.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.132+0000 7f317dc48700 1 Processor -- start 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.132+0000 7f317dc48700 1 -- start start 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.132+0000 7f317dc48700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 0x7f317810a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.132+0000 7f317dc48700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317807ae00 con 0x7f3178107d40 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.133+0000 7f31777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 0x7f317810a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.133+0000 7f31777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 0x7f317810a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42328/0 (socket says 172.21.15.45:42328) 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.133+0000 7f31777fe700 1 -- 172.21.15.45:0/238195786 learned_addr learned my addr 172.21.15.45:0/238195786 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.133+0000 7f31777fe700 1 -- 172.21.15.45:0/238195786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317807af40 con 0x7f3178107d40 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.133+0000 7f31777fe700 1 --2- 172.21.15.45:0/238195786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 0x7f317810a130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f31680098a0 tx=0x7f3168009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f979d43374e68aa server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.133+0000 7f31767fc700 1 -- 172.21.15.45:0/238195786 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3168004160 con 0x7f3178107d40 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.133+0000 7f31767fc700 1 -- 172.21.15.45:0/238195786 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f31680042c0 con 0x7f3178107d40 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.133+0000 7f31767fc700 1 -- 172.21.15.45:0/238195786 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3168011a00 con 0x7f3178107d40 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.134+0000 7f317dc48700 1 -- 172.21.15.45:0/238195786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 msgr2=0x7f317810a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.134+0000 7f317dc48700 1 --2- 172.21.15.45:0/238195786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 0x7f317810a130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f31680098a0 tx=0x7f3168009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.134+0000 7f317dc48700 1 -- 172.21.15.45:0/238195786 shutdown_connections 2024-01-31T05:13:22.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.134+0000 7f317dc48700 1 --2- 172.21.15.45:0/238195786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 0x7f317810a130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:22.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.134+0000 7f317dc48700 1 -- 172.21.15.45:0/238195786 >> 172.21.15.45:0/238195786 conn(0x7f31781016d0 msgr2=0x7f3178103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:22.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.134+0000 7f317dc48700 1 -- 172.21.15.45:0/238195786 shutdown_connections 2024-01-31T05:13:22.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.134+0000 7f317dc48700 1 -- 172.21.15.45:0/238195786 wait complete. 2024-01-31T05:13:22.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.135+0000 7f317dc48700 1 Processor -- start 2024-01-31T05:13:22.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.135+0000 7f317dc48700 1 -- start start 2024-01-31T05:13:22.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.135+0000 7f317dc48700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 0x7f317819d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:22.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.135+0000 7f317dc48700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317819d600 con 0x7f3178107d40 2024-01-31T05:13:22.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.135+0000 7f31777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 0x7f317819d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.135+0000 7f31777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 0x7f317819d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42336/0 (socket says 172.21.15.45:42336) 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.135+0000 7f31777fe700 1 -- 172.21.15.45:0/2249606543 learned_addr learned my addr 172.21.15.45:0/2249606543 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.135+0000 7f31777fe700 1 -- 172.21.15.45:0/2249606543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3168009550 con 0x7f3178107d40 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.136+0000 7f31777fe700 1 --2- 172.21.15.45:0/2249606543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 0x7f317819d0c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3168009130 tx=0x7f3168011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.136+0000 7f3174ff9700 1 -- 172.21.15.45:0/2249606543 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3168003b70 con 0x7f3178107d40 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.136+0000 7f3174ff9700 1 -- 172.21.15.45:0/2249606543 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3168003cd0 con 0x7f3178107d40 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.136+0000 7f317dc48700 1 -- 172.21.15.45:0/2249606543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317819d800 con 0x7f3178107d40 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.136+0000 7f3174ff9700 1 -- 172.21.15.45:0/2249606543 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31680197d0 con 0x7f3178107d40 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.136+0000 7f317dc48700 1 -- 172.21.15.45:0/2249606543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317819dca0 con 0x7f3178107d40 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.136+0000 7f3174ff9700 1 -- 172.21.15.45:0/2249606543 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 45535+0+0 (secure 0 0 0) 0x7f3168019a90 con 0x7f3178107d40 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.137+0000 7f3174ff9700 1 --2- 172.21.15.45:0/2249606543 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3160038920 0x7f316003ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.137+0000 7f3174ff9700 1 -- 172.21.15.45:0/2249606543 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f316804cac0 con 0x7f3178107d40 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.137+0000 7f317dc48700 1 -- 172.21.15.45:0/2249606543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3178197000 con 0x7f3178107d40 2024-01-31T05:13:22.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.137+0000 7f3176ffd700 1 --2- 172.21.15.45:0/2249606543 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3160038920 0x7f316003ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.137+0000 7f3176ffd700 1 --2- 172.21.15.45:0/2249606543 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3160038920 0x7f316003ade0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f316c006fd0 tx=0x7f316c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.142+0000 7f3174ff9700 1 -- 172.21.15.45:0/2249606543 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f3168018db0 con 0x7f3178107d40 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.304+0000 7f3174ff9700 1 -- 172.21.15.45:0/2249606543 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f316800ed40 con 0x7f3178107d40 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.363+0000 7f317dc48700 1 -- 172.21.15.45:0/2249606543 --> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f317802d5f0 con 0x7f3160038920 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.367+0000 7f3174ff9700 1 -- 172.21.15.45:0/2249606543 <== mgr.14120 v2:172.21.15.45:6800/4050317147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f317802d5f0 con 0x7f3160038920 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.369+0000 7f317dc48700 1 -- 172.21.15.45:0/2249606543 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3160038920 msgr2=0x7f316003ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.369+0000 7f317dc48700 1 --2- 172.21.15.45:0/2249606543 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3160038920 0x7f316003ade0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f316c006fd0 tx=0x7f316c006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.369+0000 7f317dc48700 1 -- 172.21.15.45:0/2249606543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 msgr2=0x7f317819d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.369+0000 7f317dc48700 1 --2- 172.21.15.45:0/2249606543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 0x7f317819d0c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3168009130 tx=0x7f3168011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.370+0000 7f317dc48700 1 -- 172.21.15.45:0/2249606543 shutdown_connections 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.370+0000 7f317dc48700 1 --2- 172.21.15.45:0/2249606543 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3160038920 0x7f316003ade0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.370+0000 7f317dc48700 1 --2- 172.21.15.45:0/2249606543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178107d40 0x7f317819d0c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:22.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.370+0000 7f317dc48700 1 -- 172.21.15.45:0/2249606543 >> 172.21.15.45:0/2249606543 conn(0x7f31781016d0 msgr2=0x7f3178101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:22.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.370+0000 7f317dc48700 1 -- 172.21.15.45:0/2249606543 shutdown_connections 2024-01-31T05:13:22.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:22.370+0000 7f317dc48700 1 -- 172.21.15.45:0/2249606543 wait complete. 2024-01-31T05:13:23.586 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:23 smithi045 ceph-mon[31585]: [31/Jan/2024:05:13:21] ENGINE Serving on http://172.21.15.45:8765 2024-01-31T05:13:23.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:23 smithi045 ceph-mon[31585]: [31/Jan/2024:05:13:21] ENGINE Serving on https://172.21.15.45:7150 2024-01-31T05:13:23.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:23 smithi045 ceph-mon[31585]: [31/Jan/2024:05:13:21] ENGINE Bus STARTED 2024-01-31T05:13:23.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:23 smithi045 ceph-mon[31585]: mgrmap e8: x(active, since 2s) 2024-01-31T05:13:23.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:23 smithi045 ceph-mon[31585]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:13:23.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:23 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:23.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:23 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:13:24.140 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout value unchanged 2024-01-31T05:13:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.375+0000 7f87950cd700 1 Processor -- start 2024-01-31T05:13:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.375+0000 7f87950cd700 1 -- start start 2024-01-31T05:13:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.375+0000 7f87950cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 0x7f8790106560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.375+0000 7f87950cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8790106b30 con 0x7f8790106140 2024-01-31T05:13:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.375+0000 7f878ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 0x7f8790106560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.375+0000 7f878ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 0x7f8790106560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42342/0 (socket says 172.21.15.45:42342) 2024-01-31T05:13:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.375+0000 7f878ed9d700 1 -- 172.21.15.45:0/1434125359 learned_addr learned my addr 172.21.15.45:0/1434125359 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.375+0000 7f878ed9d700 1 -- 172.21.15.45:0/1434125359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8790107340 con 0x7f8790106140 2024-01-31T05:13:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.375+0000 7f878ed9d700 1 --2- 172.21.15.45:0/1434125359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 0x7f8790106560 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f87800098a0 tx=0x7f8780009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e323ec0d0ad375 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.376+0000 7f878dd9b700 1 -- 172.21.15.45:0/1434125359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8780004160 con 0x7f8790106140 2024-01-31T05:13:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.376+0000 7f878dd9b700 1 -- 172.21.15.45:0/1434125359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f87800042c0 con 0x7f8790106140 2024-01-31T05:13:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.376+0000 7f878dd9b700 1 -- 172.21.15.45:0/1434125359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8780011a00 con 0x7f8790106140 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.376+0000 7f87950cd700 1 -- 172.21.15.45:0/1434125359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 msgr2=0x7f8790106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.376+0000 7f87950cd700 1 --2- 172.21.15.45:0/1434125359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 0x7f8790106560 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f87800098a0 tx=0x7f8780009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.376+0000 7f87950cd700 1 -- 172.21.15.45:0/1434125359 shutdown_connections 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.376+0000 7f87950cd700 1 --2- 172.21.15.45:0/1434125359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 0x7f8790106560 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.376+0000 7f87950cd700 1 -- 172.21.15.45:0/1434125359 >> 172.21.15.45:0/1434125359 conn(0x7f87901016e0 msgr2=0x7f8790103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.376+0000 7f87950cd700 1 -- 172.21.15.45:0/1434125359 shutdown_connections 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.376+0000 7f87950cd700 1 -- 172.21.15.45:0/1434125359 wait complete. 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.377+0000 7f87950cd700 1 Processor -- start 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.377+0000 7f87950cd700 1 -- start start 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.377+0000 7f87950cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 0x7f879019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.377+0000 7f87950cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879019b540 con 0x7f8790106140 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.378+0000 7f878ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 0x7f879019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.378+0000 7f878ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 0x7f879019b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42350/0 (socket says 172.21.15.45:42350) 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.378+0000 7f878ed9d700 1 -- 172.21.15.45:0/2956630125 learned_addr learned my addr 172.21.15.45:0/2956630125 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.378+0000 7f878ed9d700 1 -- 172.21.15.45:0/2956630125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8780009550 con 0x7f8790106140 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.378+0000 7f878ed9d700 1 --2- 172.21.15.45:0/2956630125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 0x7f879019b000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8780011710 tx=0x7f8780011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.378+0000 7f8777fff700 1 -- 172.21.15.45:0/2956630125 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8780003aa0 con 0x7f8790106140 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.378+0000 7f8777fff700 1 -- 172.21.15.45:0/2956630125 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8780003c00 con 0x7f8790106140 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.378+0000 7f87950cd700 1 -- 172.21.15.45:0/2956630125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879019b740 con 0x7f8790106140 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.378+0000 7f8777fff700 1 -- 172.21.15.45:0/2956630125 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8780024800 con 0x7f8790106140 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.378+0000 7f87950cd700 1 -- 172.21.15.45:0/2956630125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879019bbe0 con 0x7f8790106140 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.379+0000 7f8777fff700 1 -- 172.21.15.45:0/2956630125 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f8780024960 con 0x7f8790106140 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.379+0000 7f8777fff700 1 --2- 172.21.15.45:0/2956630125 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f8778038a90 0x7f877803af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.379+0000 7f8777fff700 1 -- 172.21.15.45:0/2956630125 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f878004c9c0 con 0x7f8790106140 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.379+0000 7f87950cd700 1 -- 172.21.15.45:0/2956630125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87901950b0 con 0x7f8790106140 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.379+0000 7f878e59c700 1 --2- 172.21.15.45:0/2956630125 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f8778038a90 0x7f877803af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.380+0000 7f878e59c700 1 --2- 172.21.15.45:0/2956630125 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f8778038a90 0x7f877803af50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8784006fd0 tx=0x7f8784006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:24.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.385+0000 7f8777fff700 1 -- 172.21.15.45:0/2956630125 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8780018db0 con 0x7f8790106140 2024-01-31T05:13:24.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.586+0000 7f87950cd700 1 -- 172.21.15.45:0/2956630125 --> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f879002d540 con 0x7f8778038a90 2024-01-31T05:13:24.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.587+0000 7f8777fff700 1 -- 172.21.15.45:0/2956630125 <== mgr.14120 v2:172.21.15.45:6800/4050317147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f879002d540 con 0x7f8778038a90 2024-01-31T05:13:24.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.588+0000 7f87950cd700 1 -- 172.21.15.45:0/2956630125 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f8778038a90 msgr2=0x7f877803af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:24.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.589+0000 7f87950cd700 1 --2- 172.21.15.45:0/2956630125 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f8778038a90 0x7f877803af50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8784006fd0 tx=0x7f8784006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:24.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.589+0000 7f87950cd700 1 -- 172.21.15.45:0/2956630125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 msgr2=0x7f879019b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:24.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.589+0000 7f87950cd700 1 --2- 172.21.15.45:0/2956630125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 0x7f879019b000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8780011710 tx=0x7f8780011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:24.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.589+0000 7f87950cd700 1 -- 172.21.15.45:0/2956630125 shutdown_connections 2024-01-31T05:13:24.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.589+0000 7f87950cd700 1 --2- 172.21.15.45:0/2956630125 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f8778038a90 0x7f877803af50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:24.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.589+0000 7f87950cd700 1 --2- 172.21.15.45:0/2956630125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8790106140 0x7f879019b000 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:24.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.589+0000 7f87950cd700 1 -- 172.21.15.45:0/2956630125 >> 172.21.15.45:0/2956630125 conn(0x7f87901016e0 msgr2=0x7f8790102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:24.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.589+0000 7f87950cd700 1 -- 172.21.15.45:0/2956630125 shutdown_connections 2024-01-31T05:13:24.145 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:23.589+0000 7f87950cd700 1 -- 172.21.15.45:0/2956630125 wait complete. 2024-01-31T05:13:24.145 INFO:teuthology.orchestra.run.smithi045.stdout:Generating ssh key... 2024-01-31T05:13:24.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-mon[31585]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:13:25.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: Generating public/private rsa key pair. 2024-01-31T05:13:25.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: Your identification has been saved in /tmp/tmp9lp29q03/key. 2024-01-31T05:13:25.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: Your public key has been saved in /tmp/tmp9lp29q03/key.pub. 2024-01-31T05:13:25.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: The key fingerprint is: 2024-01-31T05:13:25.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: SHA256:2QxJO3lYEcu5DffciYKSfFJFg17DPgYydu/79HCzFTw ceph-3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:13:25.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: The key's randomart image is: 2024-01-31T05:13:25.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: +---[RSA 3072]----+ 2024-01-31T05:13:25.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: | . B* | 2024-01-31T05:13:25.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: | = @.*. | 2024-01-31T05:13:25.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: | . %.@ o | 2024-01-31T05:13:25.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: | . oO.X oo..| 2024-01-31T05:13:25.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: | =So*.o.oE.| 2024-01-31T05:13:25.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: | + .. o| 2024-01-31T05:13:25.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: | .o o.| 2024-01-31T05:13:25.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: | .. + +| 2024-01-31T05:13:25.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: | .. o | 2024-01-31T05:13:25.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: +----[SHA256]-----+ 2024-01-31T05:13:25.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.610+0000 7fb141438700 1 Processor -- start 2024-01-31T05:13:25.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.611+0000 7fb141438700 1 -- start start 2024-01-31T05:13:25.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.611+0000 7fb141438700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 0x7fb13c108cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:25.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.611+0000 7fb141438700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c1092c0 con 0x7fb13c1088d0 2024-01-31T05:13:25.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.611+0000 7fb13affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 0x7fb13c108cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:25.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.611+0000 7fb13affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 0x7fb13c108cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42354/0 (socket says 172.21.15.45:42354) 2024-01-31T05:13:25.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.611+0000 7fb13affd700 1 -- 172.21.15.45:0/224760364 learned_addr learned my addr 172.21.15.45:0/224760364 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:25.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.611+0000 7fb13affd700 1 -- 172.21.15.45:0/224760364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c106e60 con 0x7fb13c1088d0 2024-01-31T05:13:25.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.611+0000 7fb13affd700 1 --2- 172.21.15.45:0/224760364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 0x7fb13c108cf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb1240098a0 tx=0x7fb124009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f156340ddf91ffbc server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:25.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.612+0000 7fb139ffb700 1 -- 172.21.15.45:0/224760364 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb124004160 con 0x7fb13c1088d0 2024-01-31T05:13:25.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.612+0000 7fb139ffb700 1 -- 172.21.15.45:0/224760364 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb1240042c0 con 0x7fb13c1088d0 2024-01-31T05:13:25.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.612+0000 7fb139ffb700 1 -- 172.21.15.45:0/224760364 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb124011a00 con 0x7fb13c1088d0 2024-01-31T05:13:25.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.612+0000 7fb141438700 1 -- 172.21.15.45:0/224760364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 msgr2=0x7fb13c108cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:25.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.612+0000 7fb141438700 1 --2- 172.21.15.45:0/224760364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 0x7fb13c108cf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb1240098a0 tx=0x7fb124009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:25.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.612+0000 7fb141438700 1 -- 172.21.15.45:0/224760364 shutdown_connections 2024-01-31T05:13:25.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.612+0000 7fb141438700 1 --2- 172.21.15.45:0/224760364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 0x7fb13c108cf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:25.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.612+0000 7fb141438700 1 -- 172.21.15.45:0/224760364 >> 172.21.15.45:0/224760364 conn(0x7fb13c102860 msgr2=0x7fb13c104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:25.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.612+0000 7fb141438700 1 -- 172.21.15.45:0/224760364 shutdown_connections 2024-01-31T05:13:25.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.612+0000 7fb141438700 1 -- 172.21.15.45:0/224760364 wait complete. 2024-01-31T05:13:25.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.613+0000 7fb141438700 1 Processor -- start 2024-01-31T05:13:25.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.613+0000 7fb141438700 1 -- start start 2024-01-31T05:13:25.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.613+0000 7fb141438700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 0x7fb13c19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:25.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.613+0000 7fb141438700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c19b540 con 0x7fb13c1088d0 2024-01-31T05:13:25.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.613+0000 7fb13affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 0x7fb13c19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:25.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.613+0000 7fb13affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 0x7fb13c19b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42366/0 (socket says 172.21.15.45:42366) 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.613+0000 7fb13affd700 1 -- 172.21.15.45:0/3604376148 learned_addr learned my addr 172.21.15.45:0/3604376148 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.614+0000 7fb13affd700 1 -- 172.21.15.45:0/3604376148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb124009550 con 0x7fb13c1088d0 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.614+0000 7fb13affd700 1 --2- 172.21.15.45:0/3604376148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 0x7fb13c19b000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb124009130 tx=0x7fb124011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.614+0000 7fb133fff700 1 -- 172.21.15.45:0/3604376148 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb124003ab0 con 0x7fb13c1088d0 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.614+0000 7fb133fff700 1 -- 172.21.15.45:0/3604376148 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb124003c10 con 0x7fb13c1088d0 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.614+0000 7fb141438700 1 -- 172.21.15.45:0/3604376148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13c19b740 con 0x7fb13c1088d0 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.614+0000 7fb133fff700 1 -- 172.21.15.45:0/3604376148 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1240247d0 con 0x7fb13c1088d0 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.614+0000 7fb141438700 1 -- 172.21.15.45:0/3604376148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c19bbe0 con 0x7fb13c1088d0 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.615+0000 7fb133fff700 1 -- 172.21.15.45:0/3604376148 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7fb12402d3f0 con 0x7fb13c1088d0 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.615+0000 7fb141438700 1 -- 172.21.15.45:0/3604376148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13c062e10 con 0x7fb13c1088d0 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.615+0000 7fb133fff700 1 --2- 172.21.15.45:0/3604376148 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fb128038a40 0x7fb12803af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.615+0000 7fb133fff700 1 -- 172.21.15.45:0/3604376148 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb12404cd10 con 0x7fb13c1088d0 2024-01-31T05:13:25.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.615+0000 7fb13a7fc700 1 --2- 172.21.15.45:0/3604376148 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fb128038a40 0x7fb12803af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.616+0000 7fb13a7fc700 1 --2- 172.21.15.45:0/3604376148 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fb128038a40 0x7fb12803af00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb12c006fd0 tx=0x7fb12c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.620+0000 7fb133fff700 1 -- 172.21.15.45:0/3604376148 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb124018b40 con 0x7fb13c1088d0 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.822+0000 7fb141438700 1 -- 172.21.15.45:0/3604376148 --> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb13c19e520 con 0x7fb128038a40 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.893+0000 7fb133fff700 1 -- 172.21.15.45:0/3604376148 <== mgr.14120 v2:172.21.15.45:6800/4050317147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb13c19e520 con 0x7fb128038a40 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.895+0000 7fb141438700 1 -- 172.21.15.45:0/3604376148 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fb128038a40 msgr2=0x7fb12803af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.895+0000 7fb141438700 1 --2- 172.21.15.45:0/3604376148 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fb128038a40 0x7fb12803af00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb12c006fd0 tx=0x7fb12c006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.895+0000 7fb141438700 1 -- 172.21.15.45:0/3604376148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 msgr2=0x7fb13c19b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.895+0000 7fb141438700 1 --2- 172.21.15.45:0/3604376148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 0x7fb13c19b000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb124009130 tx=0x7fb124011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.895+0000 7fb141438700 1 -- 172.21.15.45:0/3604376148 shutdown_connections 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.895+0000 7fb141438700 1 --2- 172.21.15.45:0/3604376148 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fb128038a40 0x7fb12803af00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.895+0000 7fb141438700 1 --2- 172.21.15.45:0/3604376148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb13c1088d0 0x7fb13c19b000 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.895+0000 7fb141438700 1 -- 172.21.15.45:0/3604376148 >> 172.21.15.45:0/3604376148 conn(0x7fb13c102860 msgr2=0x7fb13c1033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.895+0000 7fb141438700 1 -- 172.21.15.45:0/3604376148 shutdown_connections 2024-01-31T05:13:25.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:24.895+0000 7fb141438700 1 -- 172.21.15.45:0/3604376148 wait complete. 2024-01-31T05:13:26.087 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:25 smithi045 ceph-mon[31585]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:13:26.087 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:25 smithi045 ceph-mon[31585]: Generating ssh key... 2024-01-31T05:13:26.087 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:25 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:26.088 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:25 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:26.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpmPPAWI1o5+wek56GWFejkuztix0b1NWfEkMUdaYVM65tw//L10g+E7vGdDYh1szTf+n7rpQBO5/HS7VZtTMli74qr6aDeKSx/C8o2/JLrg8m+AM2zoAxnEOaulurhbdpdCMIuwLfVF5luB2O2jgH/aS2OnHn3OaGJ/mgD0E1PR/NNSoqI+2uFSR0a7bo9CBZrn3D2gR1MCleYDJihbmzErLTl8msViXnLo9jVBkfcxdNzZCdDqAQNetcsadD62EVR3XBqz+zHahR2GABR90k63GIogGmPoXcdjHCr9CUvbAR939v0RO6XxI5CnXCiupQWwobDz50dUwF8Bke4bqu/78iynQDRFpoLd2Th2Q8HMuoGrr14hrRHf1e1MXFhdFB7hybq3lWuuvgeJojS23vL0iIl5zzDdoP6hNTxnfv9/kT37TKTynARkseKx9Tao9wEJciwvJoGT6Dpj5C15q7RMYOYPYbmFXmaUZvw/9MBT5NmCj6XIdqZTKMuxkzdX0= ceph-3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:13:26.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.863+0000 7f027425e700 1 Processor -- start 2024-01-31T05:13:26.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.863+0000 7f027425e700 1 -- start start 2024-01-31T05:13:26.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.863+0000 7f027425e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 0x7f026c1076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:26.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.863+0000 7f027425e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c107c70 con 0x7f026c107280 2024-01-31T05:13:26.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.863+0000 7f0271ffa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 0x7f026c1076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:26.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.863+0000 7f0271ffa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 0x7f026c1076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42368/0 (socket says 172.21.15.45:42368) 2024-01-31T05:13:26.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.863+0000 7f0271ffa700 1 -- 172.21.15.45:0/511150645 learned_addr learned my addr 172.21.15.45:0/511150645 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:26.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.864+0000 7f0271ffa700 1 -- 172.21.15.45:0/511150645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c108480 con 0x7f026c107280 2024-01-31T05:13:26.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.864+0000 7f0271ffa700 1 --2- 172.21.15.45:0/511150645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 0x7f026c1076a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f02680098a0 tx=0x7f0268009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb68d8ccd3436647 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:26.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.864+0000 7f0270ff8700 1 -- 172.21.15.45:0/511150645 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0268004160 con 0x7f026c107280 2024-01-31T05:13:26.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.864+0000 7f0270ff8700 1 -- 172.21.15.45:0/511150645 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f02680042c0 con 0x7f026c107280 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.864+0000 7f0270ff8700 1 -- 172.21.15.45:0/511150645 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0268011a00 con 0x7f026c107280 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.865+0000 7f027425e700 1 -- 172.21.15.45:0/511150645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 msgr2=0x7f026c1076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.865+0000 7f027425e700 1 --2- 172.21.15.45:0/511150645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 0x7f026c1076a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f02680098a0 tx=0x7f0268009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.865+0000 7f027425e700 1 -- 172.21.15.45:0/511150645 shutdown_connections 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.865+0000 7f027425e700 1 --2- 172.21.15.45:0/511150645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 0x7f026c1076a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.865+0000 7f027425e700 1 -- 172.21.15.45:0/511150645 >> 172.21.15.45:0/511150645 conn(0x7f026c102820 msgr2=0x7f026c104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.865+0000 7f027425e700 1 -- 172.21.15.45:0/511150645 shutdown_connections 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.865+0000 7f027425e700 1 -- 172.21.15.45:0/511150645 wait complete. 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.865+0000 7f027425e700 1 Processor -- start 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.865+0000 7f027425e700 1 -- start start 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.865+0000 7f027425e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 0x7f026c07cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.865+0000 7f027425e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c07d0c0 con 0x7f026c107280 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.866+0000 7f0271ffa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 0x7f026c07cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:26.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.866+0000 7f0271ffa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 0x7f026c07cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42372/0 (socket says 172.21.15.45:42372) 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.866+0000 7f0271ffa700 1 -- 172.21.15.45:0/1085190470 learned_addr learned my addr 172.21.15.45:0/1085190470 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.866+0000 7f0271ffa700 1 -- 172.21.15.45:0/1085190470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0268009550 con 0x7f026c107280 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.866+0000 7f0271ffa700 1 --2- 172.21.15.45:0/1085190470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 0x7f026c07cb80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0268009130 tx=0x7f0268011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.866+0000 7f0262ffd700 1 -- 172.21.15.45:0/1085190470 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0268003ab0 con 0x7f026c107280 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.866+0000 7f0262ffd700 1 -- 172.21.15.45:0/1085190470 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0268003c10 con 0x7f026c107280 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.866+0000 7f027425e700 1 -- 172.21.15.45:0/1085190470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026c07b2b0 con 0x7f026c107280 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.866+0000 7f0262ffd700 1 -- 172.21.15.45:0/1085190470 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02680247d0 con 0x7f026c107280 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.866+0000 7f027425e700 1 -- 172.21.15.45:0/1085190470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c07b750 con 0x7f026c107280 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.867+0000 7f0262ffd700 1 -- 172.21.15.45:0/1085190470 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f026801a430 con 0x7f026c107280 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.867+0000 7f0262ffd700 1 --2- 172.21.15.45:0/1085190470 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f0258038a40 0x7f025803af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.867+0000 7f027425e700 1 -- 172.21.15.45:0/1085190470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026c062e10 con 0x7f026c107280 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.867+0000 7f0262ffd700 1 -- 172.21.15.45:0/1085190470 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f026804cbb0 con 0x7f026c107280 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.867+0000 7f02717f9700 1 --2- 172.21.15.45:0/1085190470 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f0258038a40 0x7f025803af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:26.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.868+0000 7f02717f9700 1 --2- 172.21.15.45:0/1085190470 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f0258038a40 0x7f025803af00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f025c006fd0 tx=0x7f025c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:25.872+0000 7f0262ffd700 1 -- 172.21.15.45:0/1085190470 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f0268018350 con 0x7f026c107280 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:26.075+0000 7f027425e700 1 -- 172.21.15.45:0/1085190470 --> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f026c10b680 con 0x7f0258038a40 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:26.075+0000 7f0262ffd700 1 -- 172.21.15.45:0/1085190470 <== mgr.14120 v2:172.21.15.45:6800/4050317147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f026c10b680 con 0x7f0258038a40 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:26.077+0000 7f027425e700 1 -- 172.21.15.45:0/1085190470 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f0258038a40 msgr2=0x7f025803af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:26.077+0000 7f027425e700 1 --2- 172.21.15.45:0/1085190470 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f0258038a40 0x7f025803af00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f025c006fd0 tx=0x7f025c006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:26.077+0000 7f027425e700 1 -- 172.21.15.45:0/1085190470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 msgr2=0x7f026c07cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:26.077+0000 7f027425e700 1 --2- 172.21.15.45:0/1085190470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 0x7f026c07cb80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0268009130 tx=0x7f0268011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:26.077+0000 7f027425e700 1 -- 172.21.15.45:0/1085190470 shutdown_connections 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:26.077+0000 7f027425e700 1 --2- 172.21.15.45:0/1085190470 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f0258038a40 0x7f025803af00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:26.077+0000 7f027425e700 1 --2- 172.21.15.45:0/1085190470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f026c107280 0x7f026c07cb80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:26.077+0000 7f027425e700 1 -- 172.21.15.45:0/1085190470 >> 172.21.15.45:0/1085190470 conn(0x7f026c102820 msgr2=0x7f026c103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:26.077+0000 7f027425e700 1 -- 172.21.15.45:0/1085190470 shutdown_connections 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:26.077+0000 7f027425e700 1 -- 172.21.15.45:0/1085190470 wait complete. 2024-01-31T05:13:26.707 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-01-31T05:13:26.708 INFO:teuthology.orchestra.run.smithi045.stdout:Adding key to root@localhost authorized_keys... 2024-01-31T05:13:26.708 INFO:teuthology.orchestra.run.smithi045.stdout:Adding host smithi045... 2024-01-31T05:13:27.007 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:26 smithi045 ceph-mon[31585]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:13:28.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:28 smithi045 ceph-mon[31585]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:13:29.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:29 smithi045 ceph-mon[31585]: Deploying cephadm binary to smithi045 2024-01-31T05:13:32.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Added host 'smithi045' with addr '172.21.15.45' 2024-01-31T05:13:32.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.114+0000 7f5fd784b700 1 Processor -- start 2024-01-31T05:13:32.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.115+0000 7f5fd784b700 1 -- start start 2024-01-31T05:13:32.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.115+0000 7f5fd784b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 0x7f5fd0105f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:32.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.115+0000 7f5fd784b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd007ae40 con 0x7f5fd0105b30 2024-01-31T05:13:32.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.115+0000 7f5fd55e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 0x7f5fd0105f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:32.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.115+0000 7f5fd55e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 0x7f5fd0105f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42378/0 (socket says 172.21.15.45:42378) 2024-01-31T05:13:32.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.115+0000 7f5fd55e7700 1 -- 172.21.15.45:0/3741434519 learned_addr learned my addr 172.21.15.45:0/3741434519 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:32.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.115+0000 7f5fd55e7700 1 -- 172.21.15.45:0/3741434519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd0106490 con 0x7f5fd0105b30 2024-01-31T05:13:32.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.115+0000 7f5fd55e7700 1 --2- 172.21.15.45:0/3741434519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 0x7f5fd0105f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0098a0 tx=0x7f5fcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbba7af430d575ee server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:32.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.116+0000 7f5fc7fff700 1 -- 172.21.15.45:0/3741434519 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fcc004160 con 0x7f5fd0105b30 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.116+0000 7f5fc7fff700 1 -- 172.21.15.45:0/3741434519 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5fcc0042c0 con 0x7f5fd0105b30 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.116+0000 7f5fc7fff700 1 -- 172.21.15.45:0/3741434519 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fcc011a00 con 0x7f5fd0105b30 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.116+0000 7f5fd784b700 1 -- 172.21.15.45:0/3741434519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 msgr2=0x7f5fd0105f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.116+0000 7f5fd784b700 1 --2- 172.21.15.45:0/3741434519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 0x7f5fd0105f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0098a0 tx=0x7f5fcc009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.116+0000 7f5fd784b700 1 -- 172.21.15.45:0/3741434519 shutdown_connections 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.116+0000 7f5fd784b700 1 --2- 172.21.15.45:0/3741434519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 0x7f5fd0105f50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.116+0000 7f5fd784b700 1 -- 172.21.15.45:0/3741434519 >> 172.21.15.45:0/3741434519 conn(0x7f5fd0101730 msgr2=0x7f5fd0103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.116+0000 7f5fd784b700 1 -- 172.21.15.45:0/3741434519 shutdown_connections 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.117+0000 7f5fd784b700 1 -- 172.21.15.45:0/3741434519 wait complete. 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.117+0000 7f5fd784b700 1 Processor -- start 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.117+0000 7f5fd784b700 1 -- start start 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.117+0000 7f5fd784b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 0x7f5fd019d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:32.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.117+0000 7f5fd784b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd019d620 con 0x7f5fd0105b30 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.117+0000 7f5fd55e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 0x7f5fd019d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.117+0000 7f5fd55e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 0x7f5fd019d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42390/0 (socket says 172.21.15.45:42390) 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.117+0000 7f5fd55e7700 1 -- 172.21.15.45:0/3669347729 learned_addr learned my addr 172.21.15.45:0/3669347729 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.118+0000 7f5fd55e7700 1 -- 172.21.15.45:0/3669347729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc009550 con 0x7f5fd0105b30 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.118+0000 7f5fd55e7700 1 --2- 172.21.15.45:0/3669347729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 0x7f5fd019d0e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009fb0 tx=0x7f5fcc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.118+0000 7f5fc67fc700 1 -- 172.21.15.45:0/3669347729 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fcc003a40 con 0x7f5fd0105b30 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.118+0000 7f5fc67fc700 1 -- 172.21.15.45:0/3669347729 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5fcc003ba0 con 0x7f5fd0105b30 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.118+0000 7f5fd784b700 1 -- 172.21.15.45:0/3669347729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd019d820 con 0x7f5fd0105b30 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.118+0000 7f5fc67fc700 1 -- 172.21.15.45:0/3669347729 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fcc01a750 con 0x7f5fd0105b30 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.118+0000 7f5fd784b700 1 -- 172.21.15.45:0/3669347729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd019dcc0 con 0x7f5fd0105b30 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.119+0000 7f5fc67fc700 1 -- 172.21.15.45:0/3669347729 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f5fcc01a980 con 0x7f5fd0105b30 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.119+0000 7f5fc67fc700 1 --2- 172.21.15.45:0/3669347729 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f5fbc038a40 0x7f5fbc03af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.119+0000 7f5fc67fc700 1 -- 172.21.15.45:0/3669347729 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5fcc04ca60 con 0x7f5fd0105b30 2024-01-31T05:13:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.119+0000 7f5fd784b700 1 -- 172.21.15.45:0/3669347729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd0196db0 con 0x7f5fd0105b30 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.119+0000 7f5fd4de6700 1 --2- 172.21.15.45:0/3669347729 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f5fbc038a40 0x7f5fbc03af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.120+0000 7f5fd4de6700 1 --2- 172.21.15.45:0/3669347729 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f5fbc038a40 0x7f5fbc03af00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0006fd0 tx=0x7f5fc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.124+0000 7f5fc67fc700 1 -- 172.21.15.45:0/3669347729 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5fcc01ac30 con 0x7f5fd0105b30 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:27.329+0000 7f5fd784b700 1 -- 172.21.15.45:0/3669347729 --> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fd002d4a0 con 0x7f5fbc038a40 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:31.687+0000 7f5fc67fc700 1 -- 172.21.15.45:0/3669347729 <== mgr.14120 v2:172.21.15.45:6800/4050317147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f5fd002d4a0 con 0x7f5fbc038a40 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:31.690+0000 7f5fd784b700 1 -- 172.21.15.45:0/3669347729 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f5fbc038a40 msgr2=0x7f5fbc03af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:31.690+0000 7f5fd784b700 1 --2- 172.21.15.45:0/3669347729 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f5fbc038a40 0x7f5fbc03af00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0006fd0 tx=0x7f5fc0006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:31.690+0000 7f5fd784b700 1 -- 172.21.15.45:0/3669347729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 msgr2=0x7f5fd019d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:31.690+0000 7f5fd784b700 1 --2- 172.21.15.45:0/3669347729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 0x7f5fd019d0e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009fb0 tx=0x7f5fcc011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:31.690+0000 7f5fd784b700 1 -- 172.21.15.45:0/3669347729 shutdown_connections 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:31.690+0000 7f5fd784b700 1 --2- 172.21.15.45:0/3669347729 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f5fbc038a40 0x7f5fbc03af00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:31.690+0000 7f5fd784b700 1 --2- 172.21.15.45:0/3669347729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fd0105b30 0x7f5fd019d0e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:31.690+0000 7f5fd784b700 1 -- 172.21.15.45:0/3669347729 >> 172.21.15.45:0/3669347729 conn(0x7f5fd0101730 msgr2=0x7f5fd0193ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:32.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:31.690+0000 7f5fd784b700 1 -- 172.21.15.45:0/3669347729 shutdown_connections 2024-01-31T05:13:32.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:31.690+0000 7f5fd784b700 1 -- 172.21.15.45:0/3669347729 wait complete. 2024-01-31T05:13:32.325 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mon service... 2024-01-31T05:13:32.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:32 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:32.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:32 smithi045 ceph-mon[31585]: Added host smithi045 2024-01-31T05:13:32.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:32 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:13:33.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-01-31T05:13:33.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.743+0000 7f24df5ef700 1 Processor -- start 2024-01-31T05:13:33.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.743+0000 7f24df5ef700 1 -- start start 2024-01-31T05:13:33.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.743+0000 7f24df5ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 0x7f24d810a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:33.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.743+0000 7f24df5ef700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d807ae60 con 0x7f24d8107d20 2024-01-31T05:13:33.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.744+0000 7f24dd38b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 0x7f24d810a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:33.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.744+0000 7f24dd38b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 0x7f24d810a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34594/0 (socket says 172.21.15.45:34594) 2024-01-31T05:13:33.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.744+0000 7f24dd38b700 1 -- 172.21.15.45:0/4197297509 learned_addr learned my addr 172.21.15.45:0/4197297509 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:33.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.744+0000 7f24dd38b700 1 -- 172.21.15.45:0/4197297509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d810a650 con 0x7f24d8107d20 2024-01-31T05:13:33.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.744+0000 7f24dd38b700 1 --2- 172.21.15.45:0/4197297509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 0x7f24d810a110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f24d00098a0 tx=0x7f24d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4308e3baed1b8ec4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:33.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.744+0000 7f24d7fff700 1 -- 172.21.15.45:0/4197297509 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24d0004160 con 0x7f24d8107d20 2024-01-31T05:13:33.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.744+0000 7f24d7fff700 1 -- 172.21.15.45:0/4197297509 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24d00042c0 con 0x7f24d8107d20 2024-01-31T05:13:33.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.744+0000 7f24d7fff700 1 -- 172.21.15.45:0/4197297509 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24d0011a00 con 0x7f24d8107d20 2024-01-31T05:13:33.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.745+0000 7f24df5ef700 1 -- 172.21.15.45:0/4197297509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 msgr2=0x7f24d810a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:33.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.745+0000 7f24df5ef700 1 --2- 172.21.15.45:0/4197297509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 0x7f24d810a110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f24d00098a0 tx=0x7f24d0009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:33.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.745+0000 7f24df5ef700 1 -- 172.21.15.45:0/4197297509 shutdown_connections 2024-01-31T05:13:33.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.745+0000 7f24df5ef700 1 --2- 172.21.15.45:0/4197297509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 0x7f24d810a110 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:33.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.745+0000 7f24df5ef700 1 -- 172.21.15.45:0/4197297509 >> 172.21.15.45:0/4197297509 conn(0x7f24d8101730 msgr2=0x7f24d8103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:33.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.745+0000 7f24df5ef700 1 -- 172.21.15.45:0/4197297509 shutdown_connections 2024-01-31T05:13:33.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.745+0000 7f24df5ef700 1 -- 172.21.15.45:0/4197297509 wait complete. 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.745+0000 7f24df5ef700 1 Processor -- start 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.746+0000 7f24df5ef700 1 -- start start 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.746+0000 7f24df5ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 0x7f24d8107550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.746+0000 7f24df5ef700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d8107a90 con 0x7f24d8107d20 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.746+0000 7f24dd38b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 0x7f24d8107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.746+0000 7f24dd38b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 0x7f24d8107550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34610/0 (socket says 172.21.15.45:34610) 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.746+0000 7f24dd38b700 1 -- 172.21.15.45:0/721212044 learned_addr learned my addr 172.21.15.45:0/721212044 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.746+0000 7f24dd38b700 1 -- 172.21.15.45:0/721212044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d0009550 con 0x7f24d8107d20 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.746+0000 7f24dd38b700 1 --2- 172.21.15.45:0/721212044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 0x7f24d8107550 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009fb0 tx=0x7f24d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.746+0000 7f24d67fc700 1 -- 172.21.15.45:0/721212044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24d0003a40 con 0x7f24d8107d20 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.746+0000 7f24d67fc700 1 -- 172.21.15.45:0/721212044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24d0003ba0 con 0x7f24d8107d20 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.746+0000 7f24df5ef700 1 -- 172.21.15.45:0/721212044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d8105cd0 con 0x7f24d8107d20 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.746+0000 7f24d67fc700 1 -- 172.21.15.45:0/721212044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24d001a750 con 0x7f24d8107d20 2024-01-31T05:13:33.606 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.747+0000 7f24df5ef700 1 -- 172.21.15.45:0/721212044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d8106170 con 0x7f24d8107d20 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.747+0000 7f24d67fc700 1 -- 172.21.15.45:0/721212044 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f24d0024430 con 0x7f24d8107d20 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.747+0000 7f24d67fc700 1 --2- 172.21.15.45:0/721212044 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f24c0038a40 0x7f24c003af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.747+0000 7f24df5ef700 1 -- 172.21.15.45:0/721212044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d810b380 con 0x7f24d8107d20 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.748+0000 7f24d67fc700 1 -- 172.21.15.45:0/721212044 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f24d004ca00 con 0x7f24d8107d20 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.748+0000 7f24dcb8a700 1 --2- 172.21.15.45:0/721212044 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f24c0038a40 0x7f24c003af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.748+0000 7f24dcb8a700 1 --2- 172.21.15.45:0/721212044 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f24c0038a40 0x7f24c003af00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f24cc006fd0 tx=0x7f24cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.753+0000 7f24d67fc700 1 -- 172.21.15.45:0/721212044 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f24d00189a0 con 0x7f24d8107d20 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.967+0000 7f24df5ef700 1 -- 172.21.15.45:0/721212044 --> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f24d8106740 con 0x7f24c0038a40 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.972+0000 7f24d67fc700 1 -- 172.21.15.45:0/721212044 <== mgr.14120 v2:172.21.15.45:6800/4050317147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f24d8106740 con 0x7f24c0038a40 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.974+0000 7f24df5ef700 1 -- 172.21.15.45:0/721212044 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f24c0038a40 msgr2=0x7f24c003af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.974+0000 7f24df5ef700 1 --2- 172.21.15.45:0/721212044 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f24c0038a40 0x7f24c003af00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f24cc006fd0 tx=0x7f24cc006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.974+0000 7f24df5ef700 1 -- 172.21.15.45:0/721212044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 msgr2=0x7f24d8107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.974+0000 7f24df5ef700 1 --2- 172.21.15.45:0/721212044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 0x7f24d8107550 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009fb0 tx=0x7f24d0011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.974+0000 7f24df5ef700 1 -- 172.21.15.45:0/721212044 shutdown_connections 2024-01-31T05:13:33.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.974+0000 7f24df5ef700 1 --2- 172.21.15.45:0/721212044 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f24c0038a40 0x7f24c003af00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:33.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.974+0000 7f24df5ef700 1 --2- 172.21.15.45:0/721212044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8107d20 0x7f24d8107550 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:33.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.974+0000 7f24df5ef700 1 -- 172.21.15.45:0/721212044 >> 172.21.15.45:0/721212044 conn(0x7f24d8101730 msgr2=0x7f24d8102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:33.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.974+0000 7f24df5ef700 1 -- 172.21.15.45:0/721212044 shutdown_connections 2024-01-31T05:13:33.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:32.974+0000 7f24df5ef700 1 -- 172.21.15.45:0/721212044 wait complete. 2024-01-31T05:13:33.608 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mgr service... 2024-01-31T05:13:34.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:33 smithi045 ceph-mon[31585]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:13:34.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:33 smithi045 ceph-mon[31585]: Saving service mon spec with placement count:5 2024-01-31T05:13:34.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:33 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:35.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:35 smithi045 ceph-mon[31585]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:13:35.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:35 smithi045 ceph-mon[31585]: Saving service mgr spec with placement count:2 2024-01-31T05:13:35.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:35 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:35.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:35 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:35.566 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-01-31T05:13:35.566 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.998+0000 7ff61dcd1700 1 Processor -- start 2024-01-31T05:13:35.566 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.998+0000 7ff61dcd1700 1 -- start start 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.998+0000 7ff61dcd1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 0x7ff6181076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.998+0000 7ff61dcd1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618107c70 con 0x7ff618107280 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.998+0000 7ff6177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 0x7ff6181076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.998+0000 7ff6177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 0x7ff6181076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34622/0 (socket says 172.21.15.45:34622) 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.998+0000 7ff6177fe700 1 -- 172.21.15.45:0/898239084 learned_addr learned my addr 172.21.15.45:0/898239084 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.999+0000 7ff6177fe700 1 -- 172.21.15.45:0/898239084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6181083e0 con 0x7ff618107280 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.999+0000 7ff6177fe700 1 --2- 172.21.15.45:0/898239084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 0x7ff6181076a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff6080098a0 tx=0x7ff608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c700fec308aac9fc server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.999+0000 7ff6167fc700 1 -- 172.21.15.45:0/898239084 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff608004160 con 0x7ff618107280 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.999+0000 7ff6167fc700 1 -- 172.21.15.45:0/898239084 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6080042c0 con 0x7ff618107280 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.999+0000 7ff6167fc700 1 -- 172.21.15.45:0/898239084 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff608011a00 con 0x7ff618107280 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.999+0000 7ff61dcd1700 1 -- 172.21.15.45:0/898239084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 msgr2=0x7ff6181076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.999+0000 7ff61dcd1700 1 --2- 172.21.15.45:0/898239084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 0x7ff6181076a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff6080098a0 tx=0x7ff608009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.999+0000 7ff61dcd1700 1 -- 172.21.15.45:0/898239084 shutdown_connections 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.999+0000 7ff61dcd1700 1 --2- 172.21.15.45:0/898239084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 0x7ff6181076a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:33.999+0000 7ff61dcd1700 1 -- 172.21.15.45:0/898239084 >> 172.21.15.45:0/898239084 conn(0x7ff618102880 msgr2=0x7ff618104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.000+0000 7ff61dcd1700 1 -- 172.21.15.45:0/898239084 shutdown_connections 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.000+0000 7ff61dcd1700 1 -- 172.21.15.45:0/898239084 wait complete. 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.000+0000 7ff61dcd1700 1 Processor -- start 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.000+0000 7ff61dcd1700 1 -- start start 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.000+0000 7ff61dcd1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 0x7ff61819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.000+0000 7ff61dcd1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61819b480 con 0x7ff618107280 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.001+0000 7ff6177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 0x7ff61819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.001+0000 7ff6177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 0x7ff61819af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34634/0 (socket says 172.21.15.45:34634) 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.001+0000 7ff6177fe700 1 -- 172.21.15.45:0/1398274559 learned_addr learned my addr 172.21.15.45:0/1398274559 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.001+0000 7ff6177fe700 1 -- 172.21.15.45:0/1398274559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff608009550 con 0x7ff618107280 2024-01-31T05:13:35.568 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.001+0000 7ff6177fe700 1 --2- 172.21.15.45:0/1398274559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 0x7ff61819af40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff608009130 tx=0x7ff608011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:35.570 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.001+0000 7ff614ff9700 1 -- 172.21.15.45:0/1398274559 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff608003ab0 con 0x7ff618107280 2024-01-31T05:13:35.570 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.001+0000 7ff61dcd1700 1 -- 172.21.15.45:0/1398274559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61819b680 con 0x7ff618107280 2024-01-31T05:13:35.570 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.001+0000 7ff614ff9700 1 -- 172.21.15.45:0/1398274559 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff608003c10 con 0x7ff618107280 2024-01-31T05:13:35.570 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.001+0000 7ff614ff9700 1 -- 172.21.15.45:0/1398274559 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6080247d0 con 0x7ff618107280 2024-01-31T05:13:35.570 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.001+0000 7ff61dcd1700 1 -- 172.21.15.45:0/1398274559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61819bb20 con 0x7ff618107280 2024-01-31T05:13:35.570 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.002+0000 7ff614ff9700 1 -- 172.21.15.45:0/1398274559 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7ff60802d3f0 con 0x7ff618107280 2024-01-31T05:13:35.570 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.002+0000 7ff614ff9700 1 --2- 172.21.15.45:0/1398274559 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7ff600038a40 0x7ff60003af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:35.570 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.002+0000 7ff61dcd1700 1 -- 172.21.15.45:0/1398274559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61810b610 con 0x7ff618107280 2024-01-31T05:13:35.570 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.002+0000 7ff614ff9700 1 -- 172.21.15.45:0/1398274559 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff60804cbb0 con 0x7ff618107280 2024-01-31T05:13:35.571 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.002+0000 7ff616ffd700 1 --2- 172.21.15.45:0/1398274559 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7ff600038a40 0x7ff60003af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:35.571 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.003+0000 7ff616ffd700 1 --2- 172.21.15.45:0/1398274559 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7ff600038a40 0x7ff60003af00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff60c006fd0 tx=0x7ff60c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:35.571 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.007+0000 7ff614ff9700 1 -- 172.21.15.45:0/1398274559 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7ff608010970 con 0x7ff618107280 2024-01-31T05:13:35.571 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.224+0000 7ff61dcd1700 1 -- 172.21.15.45:0/1398274559 --> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff618103b20 con 0x7ff600038a40 2024-01-31T05:13:35.571 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.228+0000 7ff614ff9700 1 -- 172.21.15.45:0/1398274559 <== mgr.14120 v2:172.21.15.45:6800/4050317147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff618103b20 con 0x7ff600038a40 2024-01-31T05:13:35.571 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.230+0000 7ff61dcd1700 1 -- 172.21.15.45:0/1398274559 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7ff600038a40 msgr2=0x7ff60003af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:35.571 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.230+0000 7ff61dcd1700 1 --2- 172.21.15.45:0/1398274559 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7ff600038a40 0x7ff60003af00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff60c006fd0 tx=0x7ff60c006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:35.571 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.230+0000 7ff61dcd1700 1 -- 172.21.15.45:0/1398274559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 msgr2=0x7ff61819af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:35.571 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.230+0000 7ff61dcd1700 1 --2- 172.21.15.45:0/1398274559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 0x7ff61819af40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff608009130 tx=0x7ff608011ad0 comp rx=0 tx=0).stop 2024-01-31T05:13:35.571 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.230+0000 7ff61dcd1700 1 -- 172.21.15.45:0/1398274559 shutdown_connections 2024-01-31T05:13:35.571 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.230+0000 7ff61dcd1700 1 --2- 172.21.15.45:0/1398274559 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7ff600038a40 0x7ff60003af00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:35.571 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.230+0000 7ff61dcd1700 1 --2- 172.21.15.45:0/1398274559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618107280 0x7ff61819af40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:35.572 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.230+0000 7ff61dcd1700 1 -- 172.21.15.45:0/1398274559 >> 172.21.15.45:0/1398274559 conn(0x7ff618102880 msgr2=0x7ff618103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:35.572 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.231+0000 7ff61dcd1700 1 -- 172.21.15.45:0/1398274559 shutdown_connections 2024-01-31T05:13:35.572 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:34.231+0000 7ff61dcd1700 1 -- 172.21.15.45:0/1398274559 wait complete. 2024-01-31T05:13:36.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:36 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:36.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:36 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1935050791' entity='client.admin' 2024-01-31T05:13:37.285 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.052+0000 7fef94ebe700 1 Processor -- start 2024-01-31T05:13:37.285 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.052+0000 7fef94ebe700 1 -- start start 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.052+0000 7fef94ebe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 0x7fef90107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.052+0000 7fef94ebe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9007aec0 con 0x7fef90105ac0 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.052+0000 7fef8e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 0x7fef90107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.052+0000 7fef8e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 0x7fef90107ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34640/0 (socket says 172.21.15.45:34640) 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.052+0000 7fef8e59c700 1 -- 172.21.15.45:0/2115132016 learned_addr learned my addr 172.21.15.45:0/2115132016 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.053+0000 7fef8e59c700 1 -- 172.21.15.45:0/2115132016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef90108430 con 0x7fef90105ac0 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.053+0000 7fef8e59c700 1 --2- 172.21.15.45:0/2115132016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 0x7fef90107ef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef800098a0 tx=0x7fef80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803eb52b28d7ae85 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.053+0000 7fef8d59a700 1 -- 172.21.15.45:0/2115132016 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef80004160 con 0x7fef90105ac0 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.053+0000 7fef8d59a700 1 -- 172.21.15.45:0/2115132016 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef800042c0 con 0x7fef90105ac0 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.053+0000 7fef8d59a700 1 -- 172.21.15.45:0/2115132016 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef800119b0 con 0x7fef90105ac0 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.054+0000 7fef94ebe700 1 -- 172.21.15.45:0/2115132016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 msgr2=0x7fef90107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.054+0000 7fef94ebe700 1 --2- 172.21.15.45:0/2115132016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 0x7fef90107ef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef800098a0 tx=0x7fef80009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:37.286 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.054+0000 7fef94ebe700 1 -- 172.21.15.45:0/2115132016 shutdown_connections 2024-01-31T05:13:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.054+0000 7fef94ebe700 1 --2- 172.21.15.45:0/2115132016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 0x7fef90107ef0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.054+0000 7fef94ebe700 1 -- 172.21.15.45:0/2115132016 >> 172.21.15.45:0/2115132016 conn(0x7fef90101740 msgr2=0x7fef90103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.054+0000 7fef94ebe700 1 -- 172.21.15.45:0/2115132016 shutdown_connections 2024-01-31T05:13:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.054+0000 7fef94ebe700 1 -- 172.21.15.45:0/2115132016 wait complete. 2024-01-31T05:13:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.055+0000 7fef94ebe700 1 Processor -- start 2024-01-31T05:13:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.055+0000 7fef94ebe700 1 -- start start 2024-01-31T05:13:37.287 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.055+0000 7fef94ebe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 0x7fef901a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.055+0000 7fef94ebe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901a1a80 con 0x7fef90105ac0 2024-01-31T05:13:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.056+0000 7fef8e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 0x7fef901a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.056+0000 7fef8e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 0x7fef901a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34648/0 (socket says 172.21.15.45:34648) 2024-01-31T05:13:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.056+0000 7fef8e59c700 1 -- 172.21.15.45:0/1935050791 learned_addr learned my addr 172.21.15.45:0/1935050791 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.056+0000 7fef8e59c700 1 -- 172.21.15.45:0/1935050791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef80009550 con 0x7fef90105ac0 2024-01-31T05:13:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.056+0000 7fef8e59c700 1 --2- 172.21.15.45:0/1935050791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 0x7fef901a1540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef80006af0 tx=0x7fef80011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.057+0000 7fef7f7fe700 1 -- 172.21.15.45:0/1935050791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef80003a40 con 0x7fef90105ac0 2024-01-31T05:13:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.057+0000 7fef94ebe700 1 -- 172.21.15.45:0/1935050791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef901a1c80 con 0x7fef90105ac0 2024-01-31T05:13:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.057+0000 7fef7f7fe700 1 -- 172.21.15.45:0/1935050791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef80003ba0 con 0x7fef90105ac0 2024-01-31T05:13:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.057+0000 7fef7f7fe700 1 -- 172.21.15.45:0/1935050791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef800196c0 con 0x7fef90105ac0 2024-01-31T05:13:37.288 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.057+0000 7fef94ebe700 1 -- 172.21.15.45:0/1935050791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef901a2120 con 0x7fef90105ac0 2024-01-31T05:13:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.057+0000 7fef7f7fe700 1 -- 172.21.15.45:0/1935050791 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7fef80019980 con 0x7fef90105ac0 2024-01-31T05:13:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.058+0000 7fef7f7fe700 1 --2- 172.21.15.45:0/1935050791 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fef780389f0 0x7fef7803aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.058+0000 7fef7f7fe700 1 -- 172.21.15.45:0/1935050791 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fef8004d920 con 0x7fef90105ac0 2024-01-31T05:13:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.058+0000 7fef94ebe700 1 -- 172.21.15.45:0/1935050791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9019ada0 con 0x7fef90105ac0 2024-01-31T05:13:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.058+0000 7fef8dd9b700 1 --2- 172.21.15.45:0/1935050791 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fef780389f0 0x7fef7803aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.058+0000 7fef8dd9b700 1 --2- 172.21.15.45:0/1935050791 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fef780389f0 0x7fef7803aeb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef84006fd0 tx=0x7fef84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.067+0000 7fef7f7fe700 1 -- 172.21.15.45:0/1935050791 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fef800243f0 con 0x7fef90105ac0 2024-01-31T05:13:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.279+0000 7fef94ebe700 1 -- 172.21.15.45:0/1935050791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fef9002d530 con 0x7fef90105ac0 2024-01-31T05:13:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.282+0000 7fef7f7fe700 1 -- 172.21.15.45:0/1935050791 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fef80023390 con 0x7fef90105ac0 2024-01-31T05:13:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.284+0000 7fef94ebe700 1 -- 172.21.15.45:0/1935050791 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fef780389f0 msgr2=0x7fef7803aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.284+0000 7fef94ebe700 1 --2- 172.21.15.45:0/1935050791 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fef780389f0 0x7fef7803aeb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef84006fd0 tx=0x7fef84006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.284+0000 7fef94ebe700 1 -- 172.21.15.45:0/1935050791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 msgr2=0x7fef901a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.284+0000 7fef94ebe700 1 --2- 172.21.15.45:0/1935050791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 0x7fef901a1540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef80006af0 tx=0x7fef80011a10 comp rx=0 tx=0).stop 2024-01-31T05:13:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.284+0000 7fef94ebe700 1 -- 172.21.15.45:0/1935050791 shutdown_connections 2024-01-31T05:13:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.284+0000 7fef94ebe700 1 --2- 172.21.15.45:0/1935050791 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fef780389f0 0x7fef7803aeb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.284+0000 7fef94ebe700 1 --2- 172.21.15.45:0/1935050791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef90105ac0 0x7fef901a1540 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.284+0000 7fef94ebe700 1 -- 172.21.15.45:0/1935050791 >> 172.21.15.45:0/1935050791 conn(0x7fef90101740 msgr2=0x7fef90103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.284+0000 7fef94ebe700 1 -- 172.21.15.45:0/1935050791 shutdown_connections 2024-01-31T05:13:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:36.284+0000 7fef94ebe700 1 -- 172.21.15.45:0/1935050791 wait complete. 2024-01-31T05:13:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.735+0000 7fcee219a700 1 Processor -- start 2024-01-31T05:13:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.735+0000 7fcee219a700 1 -- start start 2024-01-31T05:13:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.735+0000 7fcee219a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 0x7fcedc106520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.735+0000 7fcee219a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc106af0 con 0x7fcedc106100 2024-01-31T05:13:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.735+0000 7fcedb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 0x7fcedc106520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:38.517 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.735+0000 7fcedb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 0x7fcedc106520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60386/0 (socket says 172.21.15.45:60386) 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.735+0000 7fcedb7fe700 1 -- 172.21.15.45:0/3386369035 learned_addr learned my addr 172.21.15.45:0/3386369035 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.736+0000 7fcedb7fe700 1 -- 172.21.15.45:0/3386369035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc107300 con 0x7fcedc106100 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.736+0000 7fcedb7fe700 1 --2- 172.21.15.45:0/3386369035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 0x7fcedc106520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0098a0 tx=0x7fcecc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee1f94fafe672de server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.736+0000 7fceda7fc700 1 -- 172.21.15.45:0/3386369035 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcecc004160 con 0x7fcedc106100 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.736+0000 7fceda7fc700 1 -- 172.21.15.45:0/3386369035 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcecc0042c0 con 0x7fcedc106100 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.736+0000 7fceda7fc700 1 -- 172.21.15.45:0/3386369035 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcecc0119b0 con 0x7fcedc106100 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.737+0000 7fcee219a700 1 -- 172.21.15.45:0/3386369035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 msgr2=0x7fcedc106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.737+0000 7fcee219a700 1 --2- 172.21.15.45:0/3386369035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 0x7fcedc106520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0098a0 tx=0x7fcecc009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.737+0000 7fcee219a700 1 -- 172.21.15.45:0/3386369035 shutdown_connections 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.737+0000 7fcee219a700 1 --2- 172.21.15.45:0/3386369035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 0x7fcedc106520 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.737+0000 7fcee219a700 1 -- 172.21.15.45:0/3386369035 >> 172.21.15.45:0/3386369035 conn(0x7fcedc1016e0 msgr2=0x7fcedc103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.737+0000 7fcee219a700 1 -- 172.21.15.45:0/3386369035 shutdown_connections 2024-01-31T05:13:38.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.737+0000 7fcee219a700 1 -- 172.21.15.45:0/3386369035 wait complete. 2024-01-31T05:13:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.737+0000 7fcee219a700 1 Processor -- start 2024-01-31T05:13:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.738+0000 7fcee219a700 1 -- start start 2024-01-31T05:13:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.738+0000 7fcee219a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 0x7fcedc1911e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.738+0000 7fcee219a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc191720 con 0x7fcedc106100 2024-01-31T05:13:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.738+0000 7fcedb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 0x7fcedc1911e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.738+0000 7fcedb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 0x7fcedc1911e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60390/0 (socket says 172.21.15.45:60390) 2024-01-31T05:13:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.738+0000 7fcedb7fe700 1 -- 172.21.15.45:0/1016235437 learned_addr learned my addr 172.21.15.45:0/1016235437 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.738+0000 7fcedb7fe700 1 -- 172.21.15.45:0/1016235437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc009550 con 0x7fcedc106100 2024-01-31T05:13:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.738+0000 7fcedb7fe700 1 --2- 172.21.15.45:0/1016235437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 0x7fcedc1911e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcecc006af0 tx=0x7fcecc011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.738+0000 7fced8ff9700 1 -- 172.21.15.45:0/1016235437 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcecc003a40 con 0x7fcedc106100 2024-01-31T05:13:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.738+0000 7fcee219a700 1 -- 172.21.15.45:0/1016235437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc18f910 con 0x7fcedc106100 2024-01-31T05:13:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.738+0000 7fced8ff9700 1 -- 172.21.15.45:0/1016235437 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcecc003ba0 con 0x7fcedc106100 2024-01-31T05:13:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.738+0000 7fced8ff9700 1 -- 172.21.15.45:0/1016235437 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcecc019690 con 0x7fcedc106100 2024-01-31T05:13:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.739+0000 7fcee219a700 1 -- 172.21.15.45:0/1016235437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc18fdb0 con 0x7fcedc106100 2024-01-31T05:13:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.739+0000 7fced8ff9700 1 -- 172.21.15.45:0/1016235437 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7fcecc019950 con 0x7fcedc106100 2024-01-31T05:13:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.740+0000 7fcee219a700 1 -- 172.21.15.45:0/1016235437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc10a530 con 0x7fcedc106100 2024-01-31T05:13:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.740+0000 7fced8ff9700 1 --2- 172.21.15.45:0/1016235437 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fcec80389f0 0x7fcec803aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.740+0000 7fced8ff9700 1 -- 172.21.15.45:0/1016235437 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcecc04c9c0 con 0x7fcedc106100 2024-01-31T05:13:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.740+0000 7fcedaffd700 1 --2- 172.21.15.45:0/1016235437 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fcec80389f0 0x7fcec803aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:38.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.740+0000 7fcedaffd700 1 --2- 172.21.15.45:0/1016235437 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fcec80389f0 0x7fcec803aeb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcec4006fd0 tx=0x7fcec4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.745+0000 7fced8ff9700 1 -- 172.21.15.45:0/1016235437 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fcecc018350 con 0x7fcedc106100 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.949+0000 7fcee219a700 1 -- 172.21.15.45:0/1016235437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fcedc10a740 con 0x7fcedc106100 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.951+0000 7fced8ff9700 1 -- 172.21.15.45:0/1016235437 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fcecc023b40 con 0x7fcedc106100 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.954+0000 7fcee219a700 1 -- 172.21.15.45:0/1016235437 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fcec80389f0 msgr2=0x7fcec803aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.954+0000 7fcee219a700 1 --2- 172.21.15.45:0/1016235437 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fcec80389f0 0x7fcec803aeb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcec4006fd0 tx=0x7fcec4006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.954+0000 7fcee219a700 1 -- 172.21.15.45:0/1016235437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 msgr2=0x7fcedc1911e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.954+0000 7fcee219a700 1 --2- 172.21.15.45:0/1016235437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 0x7fcedc1911e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcecc006af0 tx=0x7fcecc011a10 comp rx=0 tx=0).stop 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.954+0000 7fcee219a700 1 -- 172.21.15.45:0/1016235437 shutdown_connections 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.954+0000 7fcee219a700 1 --2- 172.21.15.45:0/1016235437 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7fcec80389f0 0x7fcec803aeb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.954+0000 7fcee219a700 1 --2- 172.21.15.45:0/1016235437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcedc106100 0x7fcedc1911e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.954+0000 7fcee219a700 1 -- 172.21.15.45:0/1016235437 >> 172.21.15.45:0/1016235437 conn(0x7fcedc1016e0 msgr2=0x7fcedc1021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.955+0000 7fcee219a700 1 -- 172.21.15.45:0/1016235437 shutdown_connections 2024-01-31T05:13:38.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:37.955+0000 7fcee219a700 1 -- 172.21.15.45:0/1016235437 wait complete. 2024-01-31T05:13:38.522 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling the dashboard module... 2024-01-31T05:13:38.636 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:38 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:38.636 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:38 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1016235437' entity='client.admin' 2024-01-31T05:13:39.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:39 smithi045 ceph-mon[31585]: from='mgr.14120 172.21.15.45:0/422388090' entity='mgr.x' 2024-01-31T05:13:39.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:39 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1529548484' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-01-31T05:13:39.893 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:39 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: ignoring --setuser ceph since I am not root 2024-01-31T05:13:39.893 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:39 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: ignoring --setgroup ceph since I am not root 2024-01-31T05:13:40.393 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:40 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:40.146+0000 7f4f6c31e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-31T05:13:40.393 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:40 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:40.241+0000 7f4f6c31e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-31T05:13:40.790 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:40 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1529548484' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-01-31T05:13:40.790 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:40 smithi045 ceph-mon[31585]: mgrmap e9: x(active, since 19s) 2024-01-31T05:13:40.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.960+0000 7f67dd7aa700 1 Processor -- start 2024-01-31T05:13:40.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.960+0000 7f67dd7aa700 1 -- start start 2024-01-31T05:13:40.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.960+0000 7f67dd7aa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 0x7f67d810a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:40.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.960+0000 7f67dd7aa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d807ade0 con 0x7f67d8107d50 2024-01-31T05:13:40.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.960+0000 7f67d6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 0x7f67d810a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:40.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.960+0000 7f67d6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 0x7f67d810a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60398/0 (socket says 172.21.15.45:60398) 2024-01-31T05:13:40.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.960+0000 7f67d6ffd700 1 -- 172.21.15.45:0/1314638835 learned_addr learned my addr 172.21.15.45:0/1314638835 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:40.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.960+0000 7f67d6ffd700 1 -- 172.21.15.45:0/1314638835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d807af20 con 0x7f67d8107d50 2024-01-31T05:13:40.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.961+0000 7f67d6ffd700 1 --2- 172.21.15.45:0/1314638835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 0x7f67d810a140 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f67c80098a0 tx=0x7f67c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1781f215ca719620 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:40.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.961+0000 7f67d5ffb700 1 -- 172.21.15.45:0/1314638835 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67c8004160 con 0x7f67d8107d50 2024-01-31T05:13:40.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.961+0000 7f67d5ffb700 1 -- 172.21.15.45:0/1314638835 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67c80042c0 con 0x7f67d8107d50 2024-01-31T05:13:40.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.961+0000 7f67d5ffb700 1 -- 172.21.15.45:0/1314638835 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67c80119b0 con 0x7f67d8107d50 2024-01-31T05:13:40.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.961+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1314638835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 msgr2=0x7f67d810a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:40.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.961+0000 7f67dd7aa700 1 --2- 172.21.15.45:0/1314638835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 0x7f67d810a140 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f67c80098a0 tx=0x7f67c8009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:40.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.961+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1314638835 shutdown_connections 2024-01-31T05:13:40.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.962+0000 7f67dd7aa700 1 --2- 172.21.15.45:0/1314638835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 0x7f67d810a140 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:40.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.962+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1314638835 >> 172.21.15.45:0/1314638835 conn(0x7f67d81016e0 msgr2=0x7f67d8103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:40.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.962+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1314638835 shutdown_connections 2024-01-31T05:13:40.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.962+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1314638835 wait complete. 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.962+0000 7f67dd7aa700 1 Processor -- start 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.962+0000 7f67dd7aa700 1 -- start start 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.963+0000 7f67dd7aa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 0x7f67d81075d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.963+0000 7f67dd7aa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8107b10 con 0x7f67d8107d50 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.963+0000 7f67d6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 0x7f67d81075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.963+0000 7f67d6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 0x7f67d81075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60408/0 (socket says 172.21.15.45:60408) 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.963+0000 7f67d6ffd700 1 -- 172.21.15.45:0/1529548484 learned_addr learned my addr 172.21.15.45:0/1529548484 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.963+0000 7f67d6ffd700 1 -- 172.21.15.45:0/1529548484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c8009550 con 0x7f67d8107d50 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.963+0000 7f67d6ffd700 1 --2- 172.21.15.45:0/1529548484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 0x7f67d81075d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f67c8011670 tx=0x7f67c8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.963+0000 7f67bffff700 1 -- 172.21.15.45:0/1529548484 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67c8003b20 con 0x7f67d8107d50 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.963+0000 7f67bffff700 1 -- 172.21.15.45:0/1529548484 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67c8003c80 con 0x7f67d8107d50 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.963+0000 7f67bffff700 1 -- 172.21.15.45:0/1529548484 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67c8019770 con 0x7f67d8107d50 2024-01-31T05:13:40.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.963+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1529548484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d8105d00 con 0x7f67d8107d50 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.963+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1529548484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d81061a0 con 0x7f67d8107d50 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.964+0000 7f67bffff700 1 -- 172.21.15.45:0/1529548484 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f67c8019a30 con 0x7f67d8107d50 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.964+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1529548484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d810b3f0 con 0x7f67d8107d50 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.964+0000 7f67bffff700 1 --2- 172.21.15.45:0/1529548484 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f67c0038a40 0x7f67c003af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.964+0000 7f67bffff700 1 -- 172.21.15.45:0/1529548484 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f67c804ce70 con 0x7f67d8107d50 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.965+0000 7f67d67fc700 1 --2- 172.21.15.45:0/1529548484 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f67c0038a40 0x7f67c003af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.965+0000 7f67d67fc700 1 --2- 172.21.15.45:0/1529548484 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f67c0038a40 0x7f67c003af00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f67cc006fd0 tx=0x7f67cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:38.969+0000 7f67bffff700 1 -- 172.21.15.45:0/1529548484 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f67c8018b80 con 0x7f67d8107d50 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.244+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1529548484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f67d81067a0 con 0x7f67d8107d50 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.589+0000 7f67bffff700 1 -- 172.21.15.45:0/1529548484 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f67c8023e60 con 0x7f67d8107d50 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.590+0000 7f67bffff700 1 -- 172.21.15.45:0/1529548484 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 9) v1 ==== 45654+0+0 (secure 0 0 0) 0x7f67c8019ce0 con 0x7f67d8107d50 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.592+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1529548484 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f67c0038a40 msgr2=0x7f67c003af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.592+0000 7f67dd7aa700 1 --2- 172.21.15.45:0/1529548484 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f67c0038a40 0x7f67c003af00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f67cc006fd0 tx=0x7f67cc006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:40.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.592+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1529548484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 msgr2=0x7f67d81075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:40.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.592+0000 7f67dd7aa700 1 --2- 172.21.15.45:0/1529548484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 0x7f67d81075d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f67c8011670 tx=0x7f67c8011a30 comp rx=0 tx=0).stop 2024-01-31T05:13:40.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.592+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1529548484 shutdown_connections 2024-01-31T05:13:40.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.592+0000 7f67dd7aa700 1 --2- 172.21.15.45:0/1529548484 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f67c0038a40 0x7f67c003af00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:40.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.592+0000 7f67dd7aa700 1 --2- 172.21.15.45:0/1529548484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d8107d50 0x7f67d81075d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:40.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.592+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1529548484 >> 172.21.15.45:0/1529548484 conn(0x7f67d81016e0 msgr2=0x7f67d8102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:40.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.592+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1529548484 shutdown_connections 2024-01-31T05:13:40.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:39.592+0000 7f67dd7aa700 1 -- 172.21.15.45:0/1529548484 wait complete. 2024-01-31T05:13:41.087 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:40 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:40.904+0000 7f4f6c31e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-31T05:13:41.087 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:40 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:40.998+0000 7f4f6c31e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-31T05:13:41.087 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:41.088+0000 7f4f6c31e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-31T05:13:41.392 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:41.246+0000 7f4f6c31e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-31T05:13:41.393 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:41.332+0000 7f4f6c31e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-31T05:13:41.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:41 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/665721734' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-01-31T05:13:41.700 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:41.514+0000 7f4f6c31e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-31T05:13:41.700 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:41.700+0000 7f4f6c31e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-31T05:13:41.977 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:41.790+0000 7f4f6c31e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-31T05:13:41.978 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:41.887+0000 7f4f6c31e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-31T05:13:42.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-01-31T05:13:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-01-31T05:13:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-01-31T05:13:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-01-31T05:13:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-01-31T05:13:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-01-31T05:13:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.237+0000 7f674d1cc700 1 Processor -- start 2024-01-31T05:13:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.237+0000 7f674d1cc700 1 -- start start 2024-01-31T05:13:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.237+0000 7f674d1cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 0x7f6748106550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.237+0000 7f674d1cc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6748106b20 con 0x7f6748106130 2024-01-31T05:13:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.237+0000 7f6746d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 0x7f6748106550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.237+0000 7f6746d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 0x7f6748106550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60428/0 (socket says 172.21.15.45:60428) 2024-01-31T05:13:42.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.237+0000 7f6746d9d700 1 -- 172.21.15.45:0/811817355 learned_addr learned my addr 172.21.15.45:0/811817355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.238+0000 7f6746d9d700 1 -- 172.21.15.45:0/811817355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6748107330 con 0x7f6748106130 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.238+0000 7f6746d9d700 1 --2- 172.21.15.45:0/811817355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 0x7f6748106550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f673c00bb40 tx=0x7f673c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac413b60ac404946 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.238+0000 7f6745d9b700 1 -- 172.21.15.45:0/811817355 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f673c00dbe0 con 0x7f6748106130 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.238+0000 7f6745d9b700 1 -- 172.21.15.45:0/811817355 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f673c0044f0 con 0x7f6748106130 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.238+0000 7f6745d9b700 1 -- 172.21.15.45:0/811817355 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f673c003e60 con 0x7f6748106130 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.238+0000 7f674d1cc700 1 -- 172.21.15.45:0/811817355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 msgr2=0x7f6748106550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.238+0000 7f674d1cc700 1 --2- 172.21.15.45:0/811817355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 0x7f6748106550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f673c00bb40 tx=0x7f673c00be50 comp rx=0 tx=0).stop 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.239+0000 7f674d1cc700 1 -- 172.21.15.45:0/811817355 shutdown_connections 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.239+0000 7f674d1cc700 1 --2- 172.21.15.45:0/811817355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 0x7f6748106550 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.239+0000 7f674d1cc700 1 -- 172.21.15.45:0/811817355 >> 172.21.15.45:0/811817355 conn(0x7f6748101710 msgr2=0x7f6748103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.239+0000 7f674d1cc700 1 -- 172.21.15.45:0/811817355 shutdown_connections 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.239+0000 7f674d1cc700 1 -- 172.21.15.45:0/811817355 wait complete. 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.239+0000 7f674d1cc700 1 Processor -- start 2024-01-31T05:13:42.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.240+0000 7f674d1cc700 1 -- start start 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.240+0000 7f674d1cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 0x7f6748198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.240+0000 7f674d1cc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67481992d0 con 0x7f6748106130 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.240+0000 7f6746d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 0x7f6748198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.240+0000 7f6746d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 0x7f6748198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60434/0 (socket says 172.21.15.45:60434) 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.240+0000 7f6746d9d700 1 -- 172.21.15.45:0/665721734 learned_addr learned my addr 172.21.15.45:0/665721734 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.240+0000 7f6746d9d700 1 -- 172.21.15.45:0/665721734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c00b7f0 con 0x7f6748106130 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.240+0000 7f6746d9d700 1 --2- 172.21.15.45:0/665721734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 0x7f6748198d90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f673c00b340 tx=0x7f673c003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.241+0000 7f672ffff700 1 -- 172.21.15.45:0/665721734 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f673c01e070 con 0x7f6748106130 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.241+0000 7f672ffff700 1 -- 172.21.15.45:0/665721734 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f673c010b40 con 0x7f6748106130 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.241+0000 7f674d1cc700 1 -- 172.21.15.45:0/665721734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67481994d0 con 0x7f6748106130 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.241+0000 7f672ffff700 1 -- 172.21.15.45:0/665721734 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f673c022ac0 con 0x7f6748106130 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.241+0000 7f674d1cc700 1 -- 172.21.15.45:0/665721734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6748199970 con 0x7f6748106130 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.242+0000 7f674d1cc700 1 -- 172.21.15.45:0/665721734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6748079fe0 con 0x7f6748106130 2024-01-31T05:13:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.242+0000 7f672ffff700 1 -- 172.21.15.45:0/665721734 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) v1 ==== 45654+0+0 (secure 0 0 0) 0x7f673c02c440 con 0x7f6748106130 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.242+0000 7f672ffff700 1 --2- 172.21.15.45:0/665721734 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f6730038aa0 0x7f673003af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.242+0000 7f672ffff700 1 -- 172.21.15.45:0/665721734 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f673c04f560 con 0x7f6748106130 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.242+0000 7f674659c700 1 -- 172.21.15.45:0/665721734 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f6730038aa0 msgr2=0x7f673003af60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4050317147 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.242+0000 7f674659c700 1 --2- 172.21.15.45:0/665721734 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f6730038aa0 0x7f673003af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.247+0000 7f672ffff700 1 -- 172.21.15.45:0/665721734 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f673c02a950 con 0x7f6748106130 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.443+0000 7f674659c700 1 -- 172.21.15.45:0/665721734 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f6730038aa0 msgr2=0x7f673003af60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4050317147 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.443+0000 7f674659c700 1 --2- 172.21.15.45:0/665721734 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f6730038aa0 0x7f673003af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.505+0000 7f674d1cc700 1 -- 172.21.15.45:0/665721734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f6748192450 con 0x7f6748106130 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.505+0000 7f672ffff700 1 -- 172.21.15.45:0/665721734 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f673c027030 con 0x7f6748106130 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.507+0000 7f674d1cc700 1 -- 172.21.15.45:0/665721734 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f6730038aa0 msgr2=0x7f673003af60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.507+0000 7f674d1cc700 1 --2- 172.21.15.45:0/665721734 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f6730038aa0 0x7f673003af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.507+0000 7f674d1cc700 1 -- 172.21.15.45:0/665721734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 msgr2=0x7f6748198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.507+0000 7f674d1cc700 1 --2- 172.21.15.45:0/665721734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 0x7f6748198d90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f673c00b340 tx=0x7f673c003fb0 comp rx=0 tx=0).stop 2024-01-31T05:13:42.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.507+0000 7f674d1cc700 1 -- 172.21.15.45:0/665721734 shutdown_connections 2024-01-31T05:13:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.507+0000 7f674d1cc700 1 --2- 172.21.15.45:0/665721734 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f6730038aa0 0x7f673003af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.507+0000 7f674d1cc700 1 --2- 172.21.15.45:0/665721734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6748106130 0x7f6748198d90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.507+0000 7f674d1cc700 1 -- 172.21.15.45:0/665721734 >> 172.21.15.45:0/665721734 conn(0x7f6748101710 msgr2=0x7f674818f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.508+0000 7f674d1cc700 1 -- 172.21.15.45:0/665721734 shutdown_connections 2024-01-31T05:13:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:41.508+0000 7f674d1cc700 1 -- 172.21.15.45:0/665721734 wait complete. 2024-01-31T05:13:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-01-31T05:13:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 9... 2024-01-31T05:13:42.287 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:41.978+0000 7f4f6c31e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-31T05:13:43.392 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:43 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:43.068+0000 7f4f6c31e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-31T05:13:43.393 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:43 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:43.202+0000 7f4f6c31e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-31T05:13:43.820 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:43 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:43.403+0000 7f4f6c31e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-31T05:13:43.820 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:43 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:43.635+0000 7f4f6c31e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-31T05:13:43.820 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:43 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:43.820+0000 7f4f6c31e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:13:44.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:44.011+0000 7f4f6c31e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-31T05:13:44.643 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:44.287+0000 7f4f6c31e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:13:44.643 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:44.377+0000 7f4f6c31e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-31T05:13:45.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:44.778+0000 7f4f6c31e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-31T05:13:45.892 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:45.608+0000 7f4f6c31e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-31T05:13:46.496 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:46.188+0000 7f4f6c31e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-31T05:13:46.496 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:46.289+0000 7f4f6c31e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-31T05:13:46.497 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:13:46.497+0000 7f4f6c31e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-31T05:13:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-mon[31585]: Active manager daemon x restarted 2024-01-31T05:13:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-mon[31585]: Activating manager daemon x 2024-01-31T05:13:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-mon[31585]: osdmap e3: 0 total, 0 up, 0 in 2024-01-31T05:13:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-mon[31585]: mgrmap e10: x(active, starting, since 0.00329817s) 2024-01-31T05:13:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:13:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-31T05:13:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-31T05:13:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-31T05:13:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-31T05:13:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-mon[31585]: Manager daemon x is now available 2024-01-31T05:13:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:13:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:13:47.774 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-31T05:13:47.774 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-31T05:13:47.774 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:47 smithi045 ceph-mon[31585]: mgrmap e11: x(active, since 1.00582s) 2024-01-31T05:13:48.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-01-31T05:13:48.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-01-31T05:13:48.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-01-31T05:13:48.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-01-31T05:13:48.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.516+0000 7f3429b7c700 1 Processor -- start 2024-01-31T05:13:48.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.516+0000 7f3429b7c700 1 -- start start 2024-01-31T05:13:48.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.516+0000 7f3429b7c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 0x7f342410a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.516+0000 7f3429b7c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342407ae80 con 0x7f3424107ce0 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.517+0000 7f34237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 0x7f342410a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.517+0000 7f34237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 0x7f342410a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60436/0 (socket says 172.21.15.45:60436) 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.517+0000 7f34237fe700 1 -- 172.21.15.45:0/818132976 learned_addr learned my addr 172.21.15.45:0/818132976 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.517+0000 7f34237fe700 1 -- 172.21.15.45:0/818132976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342410a610 con 0x7f3424107ce0 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.517+0000 7f34237fe700 1 --2- 172.21.15.45:0/818132976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 0x7f342410a0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3414009e80 tx=0x7f341400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6747a78505712da1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.517+0000 7f34227fc700 1 -- 172.21.15.45:0/818132976 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f341400e070 con 0x7f3424107ce0 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.517+0000 7f34227fc700 1 -- 172.21.15.45:0/818132976 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f341400b8c0 con 0x7f3424107ce0 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.517+0000 7f34227fc700 1 -- 172.21.15.45:0/818132976 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34140043e0 con 0x7f3424107ce0 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.518+0000 7f3429b7c700 1 -- 172.21.15.45:0/818132976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 msgr2=0x7f342410a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.518+0000 7f3429b7c700 1 --2- 172.21.15.45:0/818132976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 0x7f342410a0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3414009e80 tx=0x7f341400b330 comp rx=0 tx=0).stop 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.518+0000 7f3429b7c700 1 -- 172.21.15.45:0/818132976 shutdown_connections 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.518+0000 7f3429b7c700 1 --2- 172.21.15.45:0/818132976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 0x7f342410a0d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:48.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.518+0000 7f3429b7c700 1 -- 172.21.15.45:0/818132976 >> 172.21.15.45:0/818132976 conn(0x7f3424101710 msgr2=0x7f3424103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.518+0000 7f3429b7c700 1 -- 172.21.15.45:0/818132976 shutdown_connections 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.518+0000 7f3429b7c700 1 -- 172.21.15.45:0/818132976 wait complete. 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.519+0000 7f3429b7c700 1 Processor -- start 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.519+0000 7f3429b7c700 1 -- start start 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.519+0000 7f3429b7c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 0x7f342419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.519+0000 7f3429b7c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342419b490 con 0x7f3424107ce0 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.519+0000 7f34237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 0x7f342419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.519+0000 7f34237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 0x7f342419af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60442/0 (socket says 172.21.15.45:60442) 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.519+0000 7f34237fe700 1 -- 172.21.15.45:0/896898145 learned_addr learned my addr 172.21.15.45:0/896898145 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.520+0000 7f34237fe700 1 -- 172.21.15.45:0/896898145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3414009b30 con 0x7f3424107ce0 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.520+0000 7f34237fe700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 0x7f342419af50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3414004070 tx=0x7f3414003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.520+0000 7f3420ff9700 1 -- 172.21.15.45:0/896898145 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f341400e040 con 0x7f3424107ce0 2024-01-31T05:13:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.520+0000 7f3429b7c700 1 -- 172.21.15.45:0/896898145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342419b690 con 0x7f3424107ce0 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.520+0000 7f3420ff9700 1 -- 172.21.15.45:0/896898145 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f341400bdf0 con 0x7f3424107ce0 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.520+0000 7f3420ff9700 1 -- 172.21.15.45:0/896898145 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f341401c750 con 0x7f3424107ce0 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.520+0000 7f3429b7c700 1 -- 172.21.15.45:0/896898145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342419bb90 con 0x7f3424107ce0 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.521+0000 7f3420ff9700 1 -- 172.21.15.45:0/896898145 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) v1 ==== 45654+0+0 (secure 0 0 0) 0x7f3414012430 con 0x7f3424107ce0 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.521+0000 7f3420ff9700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 0x7f341003af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.521+0000 7f3420ff9700 1 -- 172.21.15.45:0/896898145 --> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f341003b610 con 0x7f3410038a40 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.521+0000 7f3420ff9700 1 -- 172.21.15.45:0/896898145 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f341404ca50 con 0x7f3424107ce0 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.521+0000 7f3422ffd700 1 -- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 msgr2=0x7f341003af00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4050317147 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.521+0000 7f3422ffd700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 0x7f341003af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.722+0000 7f3422ffd700 1 -- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 msgr2=0x7f341003af00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4050317147 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:42.722+0000 7f3422ffd700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 0x7f341003af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:43.123+0000 7f3422ffd700 1 -- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 msgr2=0x7f341003af00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4050317147 2024-01-31T05:13:48.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:43.123+0000 7f3422ffd700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 0x7f341003af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:43.924+0000 7f3422ffd700 1 -- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 msgr2=0x7f341003af00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4050317147 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:43.924+0000 7f3422ffd700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 0x7f341003af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:45.525+0000 7f3422ffd700 1 -- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 msgr2=0x7f341003af00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/4050317147 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:45.525+0000 7f3422ffd700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 0x7f341003af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:46.511+0000 7f3420ff9700 1 -- 172.21.15.45:0/896898145 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 10) v1 ==== 45421+0+0 (secure 0 0 0) 0x7f34140243b0 con 0x7f3424107ce0 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:46.512+0000 7f3420ff9700 1 -- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 msgr2=0x7f341003af00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:46.512+0000 7f3420ff9700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 0x7f341003af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.513+0000 7f3420ff9700 1 -- 172.21.15.45:0/896898145 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 11) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f341404e2e0 con 0x7f3424107ce0 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.513+0000 7f3420ff9700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f341003c4b0 0x7f341003e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.513+0000 7f3420ff9700 1 -- 172.21.15.45:0/896898145 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f341003b610 con 0x7f341003c4b0 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.513+0000 7f3422ffd700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f341003c4b0 0x7f341003e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.514+0000 7f3422ffd700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f341003c4b0 0x7f341003e8a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f340c003a10 tx=0x7f340c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.515+0000 7f3420ff9700 1 -- 172.21.15.45:0/896898145 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f341003b610 con 0x7f341003c4b0 2024-01-31T05:13:48.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.521+0000 7f3429b7c700 1 -- 172.21.15.45:0/896898145 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f342410b3f0 con 0x7f341003c4b0 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.522+0000 7f3420ff9700 1 -- 172.21.15.45:0/896898145 <== mgr.14152 v2:172.21.15.45:6800/2788245525 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f342410b3f0 con 0x7f341003c4b0 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.522+0000 7f3429b7c700 1 -- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f341003c4b0 msgr2=0x7f341003e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.522+0000 7f3429b7c700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f341003c4b0 0x7f341003e8a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f340c003a10 tx=0x7f340c0092b0 comp rx=0 tx=0).stop 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.522+0000 7f3429b7c700 1 -- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 msgr2=0x7f342419af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.522+0000 7f3429b7c700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 0x7f342419af50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3414004070 tx=0x7f3414003d00 comp rx=0 tx=0).stop 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.522+0000 7f3429b7c700 1 -- 172.21.15.45:0/896898145 shutdown_connections 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.522+0000 7f3429b7c700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f341003c4b0 0x7f341003e8a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.522+0000 7f3429b7c700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3424107ce0 0x7f342419af50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.522+0000 7f3429b7c700 1 --2- 172.21.15.45:0/896898145 >> [v2:172.21.15.45:6800/4050317147,v1:172.21.15.45:6801/4050317147] conn(0x7f3410038a40 0x7f341003af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.522+0000 7f3429b7c700 1 -- 172.21.15.45:0/896898145 >> 172.21.15.45:0/896898145 conn(0x7f3424101710 msgr2=0x7f34241022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.522+0000 7f3429b7c700 1 -- 172.21.15.45:0/896898145 shutdown_connections 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:47.522+0000 7f3429b7c700 1 -- 172.21.15.45:0/896898145 wait complete. 2024-01-31T05:13:48.446 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 9 is available 2024-01-31T05:13:48.447 INFO:teuthology.orchestra.run.smithi045.stdout:Generating a dashboard self-signed certificate... 2024-01-31T05:13:48.859 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:48 smithi045 ceph-mon[31585]: [31/Jan/2024:05:13:47] ENGINE Bus STARTING 2024-01-31T05:13:48.859 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:48 smithi045 ceph-mon[31585]: [31/Jan/2024:05:13:47] ENGINE Serving on https://172.21.15.45:7150 2024-01-31T05:13:48.859 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:48 smithi045 ceph-mon[31585]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-01-31T05:13:48.859 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:48 smithi045 ceph-mon[31585]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-01-31T05:13:48.859 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:48 smithi045 ceph-mon[31585]: [31/Jan/2024:05:13:47] ENGINE Serving on http://172.21.15.45:8765 2024-01-31T05:13:48.859 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:48 smithi045 ceph-mon[31585]: [31/Jan/2024:05:13:47] ENGINE Bus STARTED 2024-01-31T05:13:48.860 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:13:49.805 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-01-31T05:13:49.805 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.874+0000 7f80c4af5700 1 Processor -- start 2024-01-31T05:13:49.805 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.874+0000 7f80c4af5700 1 -- start start 2024-01-31T05:13:49.805 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.874+0000 7f80c4af5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 0x7f80c0105930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:49.805 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.874+0000 7f80c4af5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c0105f00 con 0x7f80c0105510 2024-01-31T05:13:49.805 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.875+0000 7f80be59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 0x7f80c0105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:49.805 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.875+0000 7f80be59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 0x7f80c0105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56108/0 (socket says 172.21.15.45:56108) 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.875+0000 7f80be59c700 1 -- 172.21.15.45:0/2478038774 learned_addr learned my addr 172.21.15.45:0/2478038774 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.875+0000 7f80be59c700 1 -- 172.21.15.45:0/2478038774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c01084a0 con 0x7f80c0105510 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.875+0000 7f80be59c700 1 --2- 172.21.15.45:0/2478038774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 0x7f80c0105930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f80b00098a0 tx=0x7f80b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3c4a70e387b30e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.875+0000 7f80bd59a700 1 -- 172.21.15.45:0/2478038774 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b0004160 con 0x7f80c0105510 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.875+0000 7f80bd59a700 1 -- 172.21.15.45:0/2478038774 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b00042c0 con 0x7f80c0105510 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.875+0000 7f80bd59a700 1 -- 172.21.15.45:0/2478038774 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b00119b0 con 0x7f80c0105510 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.876+0000 7f80c4af5700 1 -- 172.21.15.45:0/2478038774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 msgr2=0x7f80c0105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.876+0000 7f80c4af5700 1 --2- 172.21.15.45:0/2478038774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 0x7f80c0105930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f80b00098a0 tx=0x7f80b0009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.876+0000 7f80c4af5700 1 -- 172.21.15.45:0/2478038774 shutdown_connections 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.876+0000 7f80c4af5700 1 --2- 172.21.15.45:0/2478038774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 0x7f80c0105930 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.876+0000 7f80c4af5700 1 -- 172.21.15.45:0/2478038774 >> 172.21.15.45:0/2478038774 conn(0x7f80c0100ee0 msgr2=0x7f80c0103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.876+0000 7f80c4af5700 1 -- 172.21.15.45:0/2478038774 shutdown_connections 2024-01-31T05:13:49.806 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.876+0000 7f80c4af5700 1 -- 172.21.15.45:0/2478038774 wait complete. 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.877+0000 7f80c4af5700 1 Processor -- start 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.877+0000 7f80c4af5700 1 -- start start 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.877+0000 7f80c4af5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 0x7f80c0107d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.877+0000 7f80c4af5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c0108260 con 0x7f80c0105510 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.877+0000 7f80be59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 0x7f80c0107d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.877+0000 7f80be59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 0x7f80c0107d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56124/0 (socket says 172.21.15.45:56124) 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.877+0000 7f80be59c700 1 -- 172.21.15.45:0/4141200130 learned_addr learned my addr 172.21.15.45:0/4141200130 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.877+0000 7f80be59c700 1 -- 172.21.15.45:0/4141200130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b0009550 con 0x7f80c0105510 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.878+0000 7f80be59c700 1 --2- 172.21.15.45:0/4141200130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 0x7f80c0107d20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f80b0003e60 tx=0x7f80b0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.878+0000 7f80af7fe700 1 -- 172.21.15.45:0/4141200130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b00038b0 con 0x7f80c0105510 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.878+0000 7f80af7fe700 1 -- 172.21.15.45:0/4141200130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b0003a10 con 0x7f80c0105510 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.878+0000 7f80c4af5700 1 -- 172.21.15.45:0/4141200130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c0106450 con 0x7f80c0105510 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.878+0000 7f80af7fe700 1 -- 172.21.15.45:0/4141200130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b0019530 con 0x7f80c0105510 2024-01-31T05:13:49.807 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.878+0000 7f80c4af5700 1 -- 172.21.15.45:0/4141200130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c01068f0 con 0x7f80c0105510 2024-01-31T05:13:49.808 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.879+0000 7f80af7fe700 1 -- 172.21.15.45:0/4141200130 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 11) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f80b00197a0 con 0x7f80c0105510 2024-01-31T05:13:49.808 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.879+0000 7f80af7fe700 1 --2- 172.21.15.45:0/4141200130 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f80a8038980 0x7f80a803ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:49.808 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.879+0000 7f80c4af5700 1 -- 172.21.15.45:0/4141200130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c010b6d0 con 0x7f80c0105510 2024-01-31T05:13:49.808 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.879+0000 7f80af7fe700 1 -- 172.21.15.45:0/4141200130 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f80b004d9e0 con 0x7f80c0105510 2024-01-31T05:13:49.808 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.879+0000 7f80bdd9b700 1 --2- 172.21.15.45:0/4141200130 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f80a8038980 0x7f80a803ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:49.808 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.879+0000 7f80bdd9b700 1 --2- 172.21.15.45:0/4141200130 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f80a8038980 0x7f80a803ae40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80b4006fd0 tx=0x7f80b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:49.808 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:48.884+0000 7f80af7fe700 1 -- 172.21.15.45:0/4141200130 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f80b0019af0 con 0x7f80c0105510 2024-01-31T05:13:49.808 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.092+0000 7f80c4af5700 1 -- 172.21.15.45:0/4141200130 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f80c0106ea0 con 0x7f80a8038980 2024-01-31T05:13:49.808 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.131+0000 7f80af7fe700 1 -- 172.21.15.45:0/4141200130 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 12) v1 ==== 45654+0+0 (secure 0 0 0) 0x7f80b004d020 con 0x7f80c0105510 2024-01-31T05:13:49.808 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.216+0000 7f80af7fe700 1 -- 172.21.15.45:0/4141200130 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f80c0106ea0 con 0x7f80a8038980 2024-01-31T05:13:49.808 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.219+0000 7f80c4af5700 1 -- 172.21.15.45:0/4141200130 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f80a8038980 msgr2=0x7f80a803ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:49.808 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.219+0000 7f80c4af5700 1 --2- 172.21.15.45:0/4141200130 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f80a8038980 0x7f80a803ae40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80b4006fd0 tx=0x7f80b4006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:49.809 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.219+0000 7f80c4af5700 1 -- 172.21.15.45:0/4141200130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 msgr2=0x7f80c0107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:49.809 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.219+0000 7f80c4af5700 1 --2- 172.21.15.45:0/4141200130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 0x7f80c0107d20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f80b0003e60 tx=0x7f80b0011a30 comp rx=0 tx=0).stop 2024-01-31T05:13:49.809 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.219+0000 7f80c4af5700 1 -- 172.21.15.45:0/4141200130 shutdown_connections 2024-01-31T05:13:49.809 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.219+0000 7f80c4af5700 1 --2- 172.21.15.45:0/4141200130 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f80a8038980 0x7f80a803ae40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:49.809 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.219+0000 7f80c4af5700 1 --2- 172.21.15.45:0/4141200130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c0105510 0x7f80c0107d20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:49.809 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.219+0000 7f80c4af5700 1 -- 172.21.15.45:0/4141200130 >> 172.21.15.45:0/4141200130 conn(0x7f80c0100ee0 msgr2=0x7f80c0101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:49.809 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.219+0000 7f80c4af5700 1 -- 172.21.15.45:0/4141200130 shutdown_connections 2024-01-31T05:13:49.809 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:49.219+0000 7f80c4af5700 1 -- 172.21.15.45:0/4141200130 wait complete. 2024-01-31T05:13:49.809 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial admin user... 2024-01-31T05:13:50.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:50 smithi045 ceph-mon[31585]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:13:50.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:50 smithi045 ceph-mon[31585]: mgrmap e12: x(active, since 2s) 2024-01-31T05:13:50.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:13:50.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:13:50.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:13:51.298 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$vLQyaHKPo79rN2VJLJ8jRe45dQKElvUDncIgDkGBCwQU64Qq/l6bS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1706678030, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-01-31T05:13:51.299 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.191+0000 7fc16c41b700 1 Processor -- start 2024-01-31T05:13:51.299 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.191+0000 7fc16c41b700 1 -- start start 2024-01-31T05:13:51.299 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.191+0000 7fc16c41b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 0x7fc16410a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:51.299 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.191+0000 7fc16c41b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16407ae80 con 0x7fc164107d10 2024-01-31T05:13:51.299 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.191+0000 7fc16a1b7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 0x7fc16410a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:51.299 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.191+0000 7fc16a1b7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 0x7fc16410a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56140/0 (socket says 172.21.15.45:56140) 2024-01-31T05:13:51.299 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.191+0000 7fc16a1b7700 1 -- 172.21.15.45:0/107154635 learned_addr learned my addr 172.21.15.45:0/107154635 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:51.299 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.192+0000 7fc16a1b7700 1 -- 172.21.15.45:0/107154635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16410a640 con 0x7fc164107d10 2024-01-31T05:13:51.299 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.192+0000 7fc16a1b7700 1 --2- 172.21.15.45:0/107154635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 0x7fc16410a100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1600098a0 tx=0x7fc160009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594a6343604d9e33 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:51.299 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.192+0000 7fc1691b5700 1 -- 172.21.15.45:0/107154635 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc160004160 con 0x7fc164107d10 2024-01-31T05:13:51.299 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.192+0000 7fc1691b5700 1 -- 172.21.15.45:0/107154635 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1600042c0 con 0x7fc164107d10 2024-01-31T05:13:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.192+0000 7fc1691b5700 1 -- 172.21.15.45:0/107154635 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1600119b0 con 0x7fc164107d10 2024-01-31T05:13:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.193+0000 7fc16c41b700 1 -- 172.21.15.45:0/107154635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 msgr2=0x7fc16410a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.193+0000 7fc16c41b700 1 --2- 172.21.15.45:0/107154635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 0x7fc16410a100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1600098a0 tx=0x7fc160009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.193+0000 7fc16c41b700 1 -- 172.21.15.45:0/107154635 shutdown_connections 2024-01-31T05:13:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.193+0000 7fc16c41b700 1 --2- 172.21.15.45:0/107154635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 0x7fc16410a100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.193+0000 7fc16c41b700 1 -- 172.21.15.45:0/107154635 >> 172.21.15.45:0/107154635 conn(0x7fc164101740 msgr2=0x7fc164103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.193+0000 7fc16c41b700 1 -- 172.21.15.45:0/107154635 shutdown_connections 2024-01-31T05:13:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.193+0000 7fc16c41b700 1 -- 172.21.15.45:0/107154635 wait complete. 2024-01-31T05:13:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.194+0000 7fc16c41b700 1 Processor -- start 2024-01-31T05:13:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.194+0000 7fc16c41b700 1 -- start start 2024-01-31T05:13:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.194+0000 7fc16c41b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 0x7fc164107540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.194+0000 7fc16c41b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc164107a80 con 0x7fc164107d10 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.194+0000 7fc16a1b7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 0x7fc164107540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.194+0000 7fc16a1b7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 0x7fc164107540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56148/0 (socket says 172.21.15.45:56148) 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.194+0000 7fc16a1b7700 1 -- 172.21.15.45:0/1987263497 learned_addr learned my addr 172.21.15.45:0/1987263497 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.195+0000 7fc16a1b7700 1 -- 172.21.15.45:0/1987263497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc160009550 con 0x7fc164107d10 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.195+0000 7fc16a1b7700 1 --2- 172.21.15.45:0/1987263497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 0x7fc164107540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc160006af0 tx=0x7fc160011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.195+0000 7fc1577fe700 1 -- 172.21.15.45:0/1987263497 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc160003a40 con 0x7fc164107d10 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.195+0000 7fc1577fe700 1 -- 172.21.15.45:0/1987263497 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc160003ba0 con 0x7fc164107d10 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.195+0000 7fc1577fe700 1 -- 172.21.15.45:0/1987263497 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1600196c0 con 0x7fc164107d10 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.195+0000 7fc16c41b700 1 -- 172.21.15.45:0/1987263497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc164105cc0 con 0x7fc164107d10 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.195+0000 7fc16c41b700 1 -- 172.21.15.45:0/1987263497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc164106160 con 0x7fc164107d10 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.196+0000 7fc1577fe700 1 -- 172.21.15.45:0/1987263497 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) v1 ==== 45654+0+0 (secure 0 0 0) 0x7fc160019980 con 0x7fc164107d10 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.196+0000 7fc16c41b700 1 -- 172.21.15.45:0/1987263497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16410b420 con 0x7fc164107d10 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.196+0000 7fc1577fe700 1 --2- 172.21.15.45:0/1987263497 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc150038aa0 0x7fc15003af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:51.301 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.196+0000 7fc1577fe700 1 -- 172.21.15.45:0/1987263497 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc16004d0f0 con 0x7fc164107d10 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.197+0000 7fc1699b6700 1 --2- 172.21.15.45:0/1987263497 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc150038aa0 0x7fc15003af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.197+0000 7fc1699b6700 1 --2- 172.21.15.45:0/1987263497 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc150038aa0 0x7fc15003af60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc158006fd0 tx=0x7fc158006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.204+0000 7fc1577fe700 1 -- 172.21.15.45:0/1987263497 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fc160080600 con 0x7fc164107d10 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.423+0000 7fc16c41b700 1 -- 172.21.15.45:0/1987263497 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc164062c30 con 0x7fc150038aa0 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.670+0000 7fc1577fe700 1 -- 172.21.15.45:0/1987263497 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fc164062c30 con 0x7fc150038aa0 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.672+0000 7fc16c41b700 1 -- 172.21.15.45:0/1987263497 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc150038aa0 msgr2=0x7fc15003af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.672+0000 7fc16c41b700 1 --2- 172.21.15.45:0/1987263497 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc150038aa0 0x7fc15003af60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc158006fd0 tx=0x7fc158006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.672+0000 7fc16c41b700 1 -- 172.21.15.45:0/1987263497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 msgr2=0x7fc164107540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.672+0000 7fc16c41b700 1 --2- 172.21.15.45:0/1987263497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 0x7fc164107540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc160006af0 tx=0x7fc160011a10 comp rx=0 tx=0).stop 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.672+0000 7fc16c41b700 1 -- 172.21.15.45:0/1987263497 shutdown_connections 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.672+0000 7fc16c41b700 1 --2- 172.21.15.45:0/1987263497 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc150038aa0 0x7fc15003af60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.672+0000 7fc16c41b700 1 --2- 172.21.15.45:0/1987263497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc164107d10 0x7fc164107540 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:51.302 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.672+0000 7fc16c41b700 1 -- 172.21.15.45:0/1987263497 >> 172.21.15.45:0/1987263497 conn(0x7fc164101740 msgr2=0x7fc164102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:51.303 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.672+0000 7fc16c41b700 1 -- 172.21.15.45:0/1987263497 shutdown_connections 2024-01-31T05:13:51.303 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:50.672+0000 7fc16c41b700 1 -- 172.21.15.45:0/1987263497 wait complete. 2024-01-31T05:13:51.303 INFO:teuthology.orchestra.run.smithi045.stdout:Fetching dashboard port number... 2024-01-31T05:13:51.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:51 smithi045 ceph-mon[31585]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:13:51.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:51 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:13:52.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:52 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1151874740' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-01-31T05:13:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 8443 2024-01-31T05:13:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.723+0000 7ffa7a2d6700 1 Processor -- start 2024-01-31T05:13:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.723+0000 7ffa7a2d6700 1 -- start start 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.723+0000 7ffa7a2d6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 0x7ffa7410a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.723+0000 7ffa7a2d6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7407ae80 con 0x7ffa74107cb0 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.723+0000 7ffa73fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 0x7ffa7410a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.723+0000 7ffa73fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 0x7ffa7410a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56152/0 (socket says 172.21.15.45:56152) 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.723+0000 7ffa73fff700 1 -- 172.21.15.45:0/2238571396 learned_addr learned my addr 172.21.15.45:0/2238571396 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.723+0000 7ffa73fff700 1 -- 172.21.15.45:0/2238571396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7410a5e0 con 0x7ffa74107cb0 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.724+0000 7ffa73fff700 1 --2- 172.21.15.45:0/2238571396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 0x7ffa7410a0a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa640098a0 tx=0x7ffa64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4d56fe15e3380e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.724+0000 7ffa72ffd700 1 -- 172.21.15.45:0/2238571396 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa64004160 con 0x7ffa74107cb0 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.724+0000 7ffa72ffd700 1 -- 172.21.15.45:0/2238571396 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa640042c0 con 0x7ffa74107cb0 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.724+0000 7ffa72ffd700 1 -- 172.21.15.45:0/2238571396 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa640119b0 con 0x7ffa74107cb0 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.724+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/2238571396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 msgr2=0x7ffa7410a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.724+0000 7ffa7a2d6700 1 --2- 172.21.15.45:0/2238571396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 0x7ffa7410a0a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa640098a0 tx=0x7ffa64009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.724+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/2238571396 shutdown_connections 2024-01-31T05:13:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.724+0000 7ffa7a2d6700 1 --2- 172.21.15.45:0/2238571396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 0x7ffa7410a0a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.724+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/2238571396 >> 172.21.15.45:0/2238571396 conn(0x7ffa74101720 msgr2=0x7ffa74103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.724+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/2238571396 shutdown_connections 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.724+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/2238571396 wait complete. 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.725+0000 7ffa7a2d6700 1 Processor -- start 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.725+0000 7ffa7a2d6700 1 -- start start 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.725+0000 7ffa7a2d6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 0x7ffa74198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.725+0000 7ffa7a2d6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa74199230 con 0x7ffa74107cb0 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.725+0000 7ffa73fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 0x7ffa74198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.726+0000 7ffa73fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 0x7ffa74198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56166/0 (socket says 172.21.15.45:56166) 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.726+0000 7ffa73fff700 1 -- 172.21.15.45:0/1151874740 learned_addr learned my addr 172.21.15.45:0/1151874740 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.726+0000 7ffa73fff700 1 -- 172.21.15.45:0/1151874740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64009550 con 0x7ffa74107cb0 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.726+0000 7ffa73fff700 1 --2- 172.21.15.45:0/1151874740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 0x7ffa74198cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffa64011670 tx=0x7ffa64011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.726+0000 7ffa717fa700 1 -- 172.21.15.45:0/1151874740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa64003b20 con 0x7ffa74107cb0 2024-01-31T05:13:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.726+0000 7ffa717fa700 1 -- 172.21.15.45:0/1151874740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa64003c80 con 0x7ffa74107cb0 2024-01-31T05:13:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.726+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/1151874740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa74199430 con 0x7ffa74107cb0 2024-01-31T05:13:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.726+0000 7ffa717fa700 1 -- 172.21.15.45:0/1151874740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa64019770 con 0x7ffa74107cb0 2024-01-31T05:13:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.726+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/1151874740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa741998d0 con 0x7ffa74107cb0 2024-01-31T05:13:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.727+0000 7ffa717fa700 1 -- 172.21.15.45:0/1151874740 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) v1 ==== 45654+0+0 (secure 0 0 0) 0x7ffa64019a10 con 0x7ffa74107cb0 2024-01-31T05:13:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.727+0000 7ffa717fa700 1 --2- 172.21.15.45:0/1151874740 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ffa5c03cef0 0x7ffa5c03f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.727+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/1151874740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa74192c30 con 0x7ffa74107cb0 2024-01-31T05:13:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.727+0000 7ffa717fa700 1 -- 172.21.15.45:0/1151874740 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffa6404cdf0 con 0x7ffa74107cb0 2024-01-31T05:13:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.727+0000 7ffa737fe700 1 --2- 172.21.15.45:0/1151874740 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ffa5c03cef0 0x7ffa5c03f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.728+0000 7ffa737fe700 1 --2- 172.21.15.45:0/1151874740 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ffa5c03cef0 0x7ffa5c03f3b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffa68006fd0 tx=0x7ffa68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.732+0000 7ffa717fa700 1 -- 172.21.15.45:0/1151874740 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7ffa640189a0 con 0x7ffa74107cb0 2024-01-31T05:13:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.936+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/1151874740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7ffa7402d580 con 0x7ffa74107cb0 2024-01-31T05:13:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.936+0000 7ffa717fa700 1 -- 172.21.15.45:0/1151874740 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7ffa64023e80 con 0x7ffa74107cb0 2024-01-31T05:13:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.938+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/1151874740 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ffa5c03cef0 msgr2=0x7ffa5c03f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.938+0000 7ffa7a2d6700 1 --2- 172.21.15.45:0/1151874740 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ffa5c03cef0 0x7ffa5c03f3b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffa68006fd0 tx=0x7ffa68006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.938+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/1151874740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 msgr2=0x7ffa74198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.938+0000 7ffa7a2d6700 1 --2- 172.21.15.45:0/1151874740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 0x7ffa74198cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffa64011670 tx=0x7ffa64011a30 comp rx=0 tx=0).stop 2024-01-31T05:13:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.938+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/1151874740 shutdown_connections 2024-01-31T05:13:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.938+0000 7ffa7a2d6700 1 --2- 172.21.15.45:0/1151874740 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ffa5c03cef0 0x7ffa5c03f3b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.938+0000 7ffa7a2d6700 1 --2- 172.21.15.45:0/1151874740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa74107cb0 0x7ffa74198cf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.938+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/1151874740 >> 172.21.15.45:0/1151874740 conn(0x7ffa74101720 msgr2=0x7ffa74101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.938+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/1151874740 shutdown_connections 2024-01-31T05:13:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:51.938+0000 7ffa7a2d6700 1 -- 172.21.15.45:0/1151874740 wait complete. 2024-01-31T05:13:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-01-31T05:13:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-01-31T05:13:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-01-31T05:13:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-01-31T05:13:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-01-31T05:13:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-01-31T05:13:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph Dashboard is now available at: 2024-01-31T05:13:52.698 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:13:52.698 INFO:teuthology.orchestra.run.smithi045.stdout: URL: https://smithi045.front.sepia.ceph.com:8443/ 2024-01-31T05:13:52.698 INFO:teuthology.orchestra.run.smithi045.stdout: User: admin 2024-01-31T05:13:52.698 INFO:teuthology.orchestra.run.smithi045.stdout: Password: lplsyiye45 2024-01-31T05:13:52.698 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:13:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:Saving cluster configuration to /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config directory 2024-01-31T05:13:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling autotune for osd_memory_target 2024-01-31T05:13:53.891 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.151+0000 7f286a1e8700 1 Processor -- start 2024-01-31T05:13:53.891 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.151+0000 7f286a1e8700 1 -- start start 2024-01-31T05:13:53.891 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.151+0000 7f286a1e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 0x7f2864106520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:53.891 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.151+0000 7f286a1e8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2864106af0 con 0x7f2864106100 2024-01-31T05:13:53.891 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.152+0000 7f28637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 0x7f2864106520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:53.891 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.152+0000 7f28637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 0x7f2864106520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56168/0 (socket says 172.21.15.45:56168) 2024-01-31T05:13:53.891 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.152+0000 7f28637fe700 1 -- 172.21.15.45:0/2758197148 learned_addr learned my addr 172.21.15.45:0/2758197148 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.152+0000 7f28637fe700 1 -- 172.21.15.45:0/2758197148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864107300 con 0x7f2864106100 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.152+0000 7f28637fe700 1 --2- 172.21.15.45:0/2758197148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 0x7f2864106520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f285c00bb40 tx=0x7f285c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324c430310569ec2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.152+0000 7f28627fc700 1 -- 172.21.15.45:0/2758197148 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f285c00dbe0 con 0x7f2864106100 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.152+0000 7f28627fc700 1 -- 172.21.15.45:0/2758197148 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f285c0044f0 con 0x7f2864106100 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.152+0000 7f28627fc700 1 -- 172.21.15.45:0/2758197148 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f285c003e60 con 0x7f2864106100 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.153+0000 7f286a1e8700 1 -- 172.21.15.45:0/2758197148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 msgr2=0x7f2864106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.153+0000 7f286a1e8700 1 --2- 172.21.15.45:0/2758197148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 0x7f2864106520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f285c00bb40 tx=0x7f285c00be50 comp rx=0 tx=0).stop 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.153+0000 7f286a1e8700 1 -- 172.21.15.45:0/2758197148 shutdown_connections 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.153+0000 7f286a1e8700 1 --2- 172.21.15.45:0/2758197148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 0x7f2864106520 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.153+0000 7f286a1e8700 1 -- 172.21.15.45:0/2758197148 >> 172.21.15.45:0/2758197148 conn(0x7f2864101720 msgr2=0x7f2864103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.153+0000 7f286a1e8700 1 -- 172.21.15.45:0/2758197148 shutdown_connections 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.153+0000 7f286a1e8700 1 -- 172.21.15.45:0/2758197148 wait complete. 2024-01-31T05:13:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.153+0000 7f286a1e8700 1 Processor -- start 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f286a1e8700 1 -- start start 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f286a1e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 0x7f286419d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f286a1e8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286419d660 con 0x7f2864106100 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f28637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 0x7f286419d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f28637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 0x7f286419d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56174/0 (socket says 172.21.15.45:56174) 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f28637fe700 1 -- 172.21.15.45:0/864844262 learned_addr learned my addr 172.21.15.45:0/864844262 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f28637fe700 1 -- 172.21.15.45:0/864844262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c00b7f0 con 0x7f2864106100 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f28637fe700 1 --2- 172.21.15.45:0/864844262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 0x7f286419d120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f285c0104d0 tx=0x7f285c004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f2860ff9700 1 -- 172.21.15.45:0/864844262 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f285c010760 con 0x7f2864106100 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f2860ff9700 1 -- 172.21.15.45:0/864844262 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f285c010d80 con 0x7f2864106100 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f286a1e8700 1 -- 172.21.15.45:0/864844262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286419d860 con 0x7f2864106100 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f2860ff9700 1 -- 172.21.15.45:0/864844262 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f285c019ce0 con 0x7f2864106100 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.154+0000 7f286a1e8700 1 -- 172.21.15.45:0/864844262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286419dca0 con 0x7f2864106100 2024-01-31T05:13:53.893 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.155+0000 7f2860ff9700 1 -- 172.21.15.45:0/864844262 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) v1 ==== 45654+0+0 (secure 0 0 0) 0x7f285c02a450 con 0x7f2864106100 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.155+0000 7f286a1e8700 1 -- 172.21.15.45:0/864844262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2864079fe0 con 0x7f2864106100 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.155+0000 7f2860ff9700 1 --2- 172.21.15.45:0/864844262 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2850038a40 0x7f285003af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.156+0000 7f2860ff9700 1 -- 172.21.15.45:0/864844262 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f285c04cee0 con 0x7f2864106100 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.156+0000 7f2862ffd700 1 --2- 172.21.15.45:0/864844262 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2850038a40 0x7f285003af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.156+0000 7f2862ffd700 1 --2- 172.21.15.45:0/864844262 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2850038a40 0x7f285003af00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f284c006fd0 tx=0x7f284c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.160+0000 7f2860ff9700 1 -- 172.21.15.45:0/864844262 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f285c028710 con 0x7f2864106100 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.357+0000 7f286a1e8700 1 -- 172.21.15.45:0/864844262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f2864192420 con 0x7f2864106100 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.358+0000 7f2860ff9700 1 -- 172.21.15.45:0/864844262 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f285c028350 con 0x7f2864106100 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.360+0000 7f2860ff9700 1 -- 172.21.15.45:0/864844262 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f285c04e070 con 0x7f2864106100 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.361+0000 7f286a1e8700 1 -- 172.21.15.45:0/864844262 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2850038a40 msgr2=0x7f285003af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.361+0000 7f286a1e8700 1 --2- 172.21.15.45:0/864844262 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2850038a40 0x7f285003af00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f284c006fd0 tx=0x7f284c006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.361+0000 7f286a1e8700 1 -- 172.21.15.45:0/864844262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 msgr2=0x7f286419d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.361+0000 7f286a1e8700 1 --2- 172.21.15.45:0/864844262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 0x7f286419d120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f285c0104d0 tx=0x7f285c004020 comp rx=0 tx=0).stop 2024-01-31T05:13:53.895 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.361+0000 7f286a1e8700 1 -- 172.21.15.45:0/864844262 shutdown_connections 2024-01-31T05:13:53.895 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.361+0000 7f286a1e8700 1 --2- 172.21.15.45:0/864844262 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2850038a40 0x7f285003af00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:53.895 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.361+0000 7f286a1e8700 1 --2- 172.21.15.45:0/864844262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2864106100 0x7f286419d120 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:53.895 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.361+0000 7f286a1e8700 1 -- 172.21.15.45:0/864844262 >> 172.21.15.45:0/864844262 conn(0x7f2864101720 msgr2=0x7f286418f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:53.895 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.361+0000 7f286a1e8700 1 -- 172.21.15.45:0/864844262 shutdown_connections 2024-01-31T05:13:53.895 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:53.361+0000 7f286a1e8700 1 -- 172.21.15.45:0/864844262 wait complete. 2024-01-31T05:13:54.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:54 smithi045 ceph-mon[31585]: mgrmap e13: x(active, since 6s) 2024-01-31T05:13:55.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.316+0000 7f526fb01700 1 Processor -- start 2024-01-31T05:13:55.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.316+0000 7f526fb01700 1 -- start start 2024-01-31T05:13:55.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.316+0000 7f526fb01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 0x7f5268106520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:55.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.316+0000 7f526fb01700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5268106af0 con 0x7f5268106100 2024-01-31T05:13:55.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.316+0000 7f526d89d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 0x7f5268106520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:55.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.316+0000 7f526d89d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 0x7f5268106520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56180/0 (socket says 172.21.15.45:56180) 2024-01-31T05:13:55.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.316+0000 7f526d89d700 1 -- 172.21.15.45:0/3371934997 learned_addr learned my addr 172.21.15.45:0/3371934997 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:55.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.316+0000 7f526d89d700 1 -- 172.21.15.45:0/3371934997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5268107300 con 0x7f5268106100 2024-01-31T05:13:55.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.317+0000 7f526d89d700 1 --2- 172.21.15.45:0/3371934997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 0x7f5268106520 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f52640098a0 tx=0x7f5264009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cf441185c294d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:55.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.317+0000 7f526c89b700 1 -- 172.21.15.45:0/3371934997 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5264004160 con 0x7f5268106100 2024-01-31T05:13:55.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.317+0000 7f526c89b700 1 -- 172.21.15.45:0/3371934997 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52640042c0 con 0x7f5268106100 2024-01-31T05:13:55.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.317+0000 7f526c89b700 1 -- 172.21.15.45:0/3371934997 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52640119b0 con 0x7f5268106100 2024-01-31T05:13:55.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.317+0000 7f526fb01700 1 -- 172.21.15.45:0/3371934997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 msgr2=0x7f5268106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:55.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.317+0000 7f526fb01700 1 --2- 172.21.15.45:0/3371934997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 0x7f5268106520 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f52640098a0 tx=0x7f5264009bb0 comp rx=0 tx=0).stop 2024-01-31T05:13:55.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.317+0000 7f526fb01700 1 -- 172.21.15.45:0/3371934997 shutdown_connections 2024-01-31T05:13:55.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.317+0000 7f526fb01700 1 --2- 172.21.15.45:0/3371934997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 0x7f5268106520 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:55.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.317+0000 7f526fb01700 1 -- 172.21.15.45:0/3371934997 >> 172.21.15.45:0/3371934997 conn(0x7f52681016e0 msgr2=0x7f5268103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:55.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.318+0000 7f526fb01700 1 -- 172.21.15.45:0/3371934997 shutdown_connections 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.318+0000 7f526fb01700 1 -- 172.21.15.45:0/3371934997 wait complete. 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.318+0000 7f526fb01700 1 Processor -- start 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.318+0000 7f526fb01700 1 -- start start 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.318+0000 7f526fb01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 0x7f5268198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.318+0000 7f526fb01700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52681992c0 con 0x7f5268106100 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.318+0000 7f526d89d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 0x7f5268198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.318+0000 7f526d89d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 0x7f5268198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56184/0 (socket says 172.21.15.45:56184) 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.318+0000 7f526d89d700 1 -- 172.21.15.45:0/3210087690 learned_addr learned my addr 172.21.15.45:0/3210087690 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.319+0000 7f526d89d700 1 -- 172.21.15.45:0/3210087690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5264009550 con 0x7f5268106100 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.319+0000 7f526d89d700 1 --2- 172.21.15.45:0/3210087690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 0x7f5268198d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5264003e60 tx=0x7f5264011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.319+0000 7f525effd700 1 -- 172.21.15.45:0/3210087690 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5264003a60 con 0x7f5268106100 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.319+0000 7f526fb01700 1 -- 172.21.15.45:0/3210087690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52681994c0 con 0x7f5268106100 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.319+0000 7f525effd700 1 -- 172.21.15.45:0/3210087690 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5264003bc0 con 0x7f5268106100 2024-01-31T05:13:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.319+0000 7f525effd700 1 -- 172.21.15.45:0/3210087690 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52640196c0 con 0x7f5268106100 2024-01-31T05:13:55.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.319+0000 7f526fb01700 1 -- 172.21.15.45:0/3210087690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5268199960 con 0x7f5268106100 2024-01-31T05:13:55.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.320+0000 7f525effd700 1 -- 172.21.15.45:0/3210087690 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f5264019940 con 0x7f5268106100 2024-01-31T05:13:55.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.320+0000 7f525effd700 1 --2- 172.21.15.45:0/3210087690 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f525403ad80 0x7f525403d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:55.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.320+0000 7f525effd700 1 -- 172.21.15.45:0/3210087690 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f526404cf70 con 0x7f5268106100 2024-01-31T05:13:55.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.320+0000 7f526d09c700 1 --2- 172.21.15.45:0/3210087690 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f525403ad80 0x7f525403d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:55.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.320+0000 7f526fb01700 1 -- 172.21.15.45:0/3210087690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5268062e10 con 0x7f5268106100 2024-01-31T05:13:55.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.320+0000 7f526d09c700 1 --2- 172.21.15.45:0/3210087690 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f525403ad80 0x7f525403d240 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5258006fd0 tx=0x7f5258006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:55.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.325+0000 7f525effd700 1 -- 172.21.15.45:0/3210087690 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5264010560 con 0x7f5268106100 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.637+0000 7f526fb01700 1 -- 172.21.15.45:0/3210087690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f5268192460 con 0x7f5268106100 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.640+0000 7f525effd700 1 -- 172.21.15.45:0/3210087690 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f5264010380 con 0x7f5268106100 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.642+0000 7f526fb01700 1 -- 172.21.15.45:0/3210087690 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f525403ad80 msgr2=0x7f525403d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.642+0000 7f526fb01700 1 --2- 172.21.15.45:0/3210087690 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f525403ad80 0x7f525403d240 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5258006fd0 tx=0x7f5258006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.642+0000 7f526fb01700 1 -- 172.21.15.45:0/3210087690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 msgr2=0x7f5268198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.642+0000 7f526fb01700 1 --2- 172.21.15.45:0/3210087690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 0x7f5268198d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5264003e60 tx=0x7f5264011a30 comp rx=0 tx=0).stop 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.642+0000 7f526fb01700 1 -- 172.21.15.45:0/3210087690 shutdown_connections 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.642+0000 7f526fb01700 1 --2- 172.21.15.45:0/3210087690 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f525403ad80 0x7f525403d240 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.642+0000 7f526fb01700 1 --2- 172.21.15.45:0/3210087690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5268106100 0x7f5268198d80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.642+0000 7f526fb01700 1 -- 172.21.15.45:0/3210087690 >> 172.21.15.45:0/3210087690 conn(0x7f52681016e0 msgr2=0x7f526818f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.642+0000 7f526fb01700 1 -- 172.21.15.45:0/3210087690 shutdown_connections 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-01-31T05:13:54.642+0000 7f526fb01700 1 -- 172.21.15.45:0/3210087690 wait complete. 2024-01-31T05:13:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout:Or, if you are only running a single cluster on this host: 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout:Please consider enabling telemetry to help improve Ceph: 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout: ceph telemetry on 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout:For more information see: 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:13:55.153 INFO:teuthology.orchestra.run.smithi045.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-01-31T05:13:55.154 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:13:55.154 INFO:teuthology.orchestra.run.smithi045.stdout:Bootstrap complete. 2024-01-31T05:13:55.194 INFO:tasks.cephadm:Fetching config... 2024-01-31T05:13:55.195 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:13:55.195 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-01-31T05:13:55.212 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-01-31T05:13:55.212 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:13:55.212 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-01-31T05:13:55.267 INFO:tasks.cephadm:Fetching mon keyring... 2024-01-31T05:13:55.267 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:13:55.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/keyring of=/dev/stdout 2024-01-31T05:13:55.335 INFO:tasks.cephadm:Fetching pub ssh key... 2024-01-31T05:13:55.335 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:13:55.335 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-01-31T05:13:55.390 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-01-31T05:13:55.390 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpmPPAWI1o5+wek56GWFejkuztix0b1NWfEkMUdaYVM65tw//L10g+E7vGdDYh1szTf+n7rpQBO5/HS7VZtTMli74qr6aDeKSx/C8o2/JLrg8m+AM2zoAxnEOaulurhbdpdCMIuwLfVF5luB2O2jgH/aS2OnHn3OaGJ/mgD0E1PR/NNSoqI+2uFSR0a7bo9CBZrn3D2gR1MCleYDJihbmzErLTl8msViXnLo9jVBkfcxdNzZCdDqAQNetcsadD62EVR3XBqz+zHahR2GABR90k63GIogGmPoXcdjHCr9CUvbAR939v0RO6XxI5CnXCiupQWwobDz50dUwF8Bke4bqu/78iynQDRFpoLd2Th2Q8HMuoGrr14hrRHf1e1MXFhdFB7hybq3lWuuvgeJojS23vL0iIl5zzDdoP6hNTxnfv9/kT37TKTynARkseKx9Tao9wEJciwvJoGT6Dpj5C15q7RMYOYPYbmFXmaUZvw/9MBT5NmCj6XIdqZTKMuxkzdX0= ceph-3a1f4122-bff7-11ee-95b3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-01-31T05:13:55.468 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpmPPAWI1o5+wek56GWFejkuztix0b1NWfEkMUdaYVM65tw//L10g+E7vGdDYh1szTf+n7rpQBO5/HS7VZtTMli74qr6aDeKSx/C8o2/JLrg8m+AM2zoAxnEOaulurhbdpdCMIuwLfVF5luB2O2jgH/aS2OnHn3OaGJ/mgD0E1PR/NNSoqI+2uFSR0a7bo9CBZrn3D2gR1MCleYDJihbmzErLTl8msViXnLo9jVBkfcxdNzZCdDqAQNetcsadD62EVR3XBqz+zHahR2GABR90k63GIogGmPoXcdjHCr9CUvbAR939v0RO6XxI5CnXCiupQWwobDz50dUwF8Bke4bqu/78iynQDRFpoLd2Th2Q8HMuoGrr14hrRHf1e1MXFhdFB7hybq3lWuuvgeJojS23vL0iIl5zzDdoP6hNTxnfv9/kT37TKTynARkseKx9Tao9wEJciwvJoGT6Dpj5C15q7RMYOYPYbmFXmaUZvw/9MBT5NmCj6XIdqZTKMuxkzdX0= ceph-3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:13:55.483 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpmPPAWI1o5+wek56GWFejkuztix0b1NWfEkMUdaYVM65tw//L10g+E7vGdDYh1szTf+n7rpQBO5/HS7VZtTMli74qr6aDeKSx/C8o2/JLrg8m+AM2zoAxnEOaulurhbdpdCMIuwLfVF5luB2O2jgH/aS2OnHn3OaGJ/mgD0E1PR/NNSoqI+2uFSR0a7bo9CBZrn3D2gR1MCleYDJihbmzErLTl8msViXnLo9jVBkfcxdNzZCdDqAQNetcsadD62EVR3XBqz+zHahR2GABR90k63GIogGmPoXcdjHCr9CUvbAR939v0RO6XxI5CnXCiupQWwobDz50dUwF8Bke4bqu/78iynQDRFpoLd2Th2Q8HMuoGrr14hrRHf1e1MXFhdFB7hybq3lWuuvgeJojS23vL0iIl5zzDdoP6hNTxnfv9/kT37TKTynARkseKx9Tao9wEJciwvJoGT6Dpj5C15q7RMYOYPYbmFXmaUZvw/9MBT5NmCj6XIdqZTKMuxkzdX0= ceph-3a1f4122-bff7-11ee-95b3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-01-31T05:13:55.530 INFO:teuthology.orchestra.run.smithi156.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpmPPAWI1o5+wek56GWFejkuztix0b1NWfEkMUdaYVM65tw//L10g+E7vGdDYh1szTf+n7rpQBO5/HS7VZtTMli74qr6aDeKSx/C8o2/JLrg8m+AM2zoAxnEOaulurhbdpdCMIuwLfVF5luB2O2jgH/aS2OnHn3OaGJ/mgD0E1PR/NNSoqI+2uFSR0a7bo9CBZrn3D2gR1MCleYDJihbmzErLTl8msViXnLo9jVBkfcxdNzZCdDqAQNetcsadD62EVR3XBqz+zHahR2GABR90k63GIogGmPoXcdjHCr9CUvbAR939v0RO6XxI5CnXCiupQWwobDz50dUwF8Bke4bqu/78iynQDRFpoLd2Th2Q8HMuoGrr14hrRHf1e1MXFhdFB7hybq3lWuuvgeJojS23vL0iIl5zzDdoP6hNTxnfv9/kT37TKTynARkseKx9Tao9wEJciwvJoGT6Dpj5C15q7RMYOYPYbmFXmaUZvw/9MBT5NmCj6XIdqZTKMuxkzdX0= ceph-3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:13:55.545 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpmPPAWI1o5+wek56GWFejkuztix0b1NWfEkMUdaYVM65tw//L10g+E7vGdDYh1szTf+n7rpQBO5/HS7VZtTMli74qr6aDeKSx/C8o2/JLrg8m+AM2zoAxnEOaulurhbdpdCMIuwLfVF5luB2O2jgH/aS2OnHn3OaGJ/mgD0E1PR/NNSoqI+2uFSR0a7bo9CBZrn3D2gR1MCleYDJihbmzErLTl8msViXnLo9jVBkfcxdNzZCdDqAQNetcsadD62EVR3XBqz+zHahR2GABR90k63GIogGmPoXcdjHCr9CUvbAR939v0RO6XxI5CnXCiupQWwobDz50dUwF8Bke4bqu/78iynQDRFpoLd2Th2Q8HMuoGrr14hrRHf1e1MXFhdFB7hybq3lWuuvgeJojS23vL0iIl5zzDdoP6hNTxnfv9/kT37TKTynARkseKx9Tao9wEJciwvJoGT6Dpj5C15q7RMYOYPYbmFXmaUZvw/9MBT5NmCj6XIdqZTKMuxkzdX0= ceph-3a1f4122-bff7-11ee-95b3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-01-31T05:13:55.591 INFO:teuthology.orchestra.run.smithi186.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpmPPAWI1o5+wek56GWFejkuztix0b1NWfEkMUdaYVM65tw//L10g+E7vGdDYh1szTf+n7rpQBO5/HS7VZtTMli74qr6aDeKSx/C8o2/JLrg8m+AM2zoAxnEOaulurhbdpdCMIuwLfVF5luB2O2jgH/aS2OnHn3OaGJ/mgD0E1PR/NNSoqI+2uFSR0a7bo9CBZrn3D2gR1MCleYDJihbmzErLTl8msViXnLo9jVBkfcxdNzZCdDqAQNetcsadD62EVR3XBqz+zHahR2GABR90k63GIogGmPoXcdjHCr9CUvbAR939v0RO6XxI5CnXCiupQWwobDz50dUwF8Bke4bqu/78iynQDRFpoLd2Th2Q8HMuoGrr14hrRHf1e1MXFhdFB7hybq3lWuuvgeJojS23vL0iIl5zzDdoP6hNTxnfv9/kT37TKTynARkseKx9Tao9wEJciwvJoGT6Dpj5C15q7RMYOYPYbmFXmaUZvw/9MBT5NmCj6XIdqZTKMuxkzdX0= ceph-3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:13:55.607 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-01-31T05:13:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:13:55.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:55 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3210087690' entity='client.admin' 2024-01-31T05:13:58.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.325+0000 7f3549f63700 1 -- 172.21.15.45:0/864440163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35440ff4e0 msgr2=0x7f35440ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:58.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.325+0000 7f3549f63700 1 --2- 172.21.15.45:0/864440163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35440ff4e0 0x7f35440ff900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f352c009910 tx=0x7f352c009c20 comp rx=0 tx=0).stop 2024-01-31T05:13:58.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.325+0000 7f3549f63700 1 -- 172.21.15.45:0/864440163 shutdown_connections 2024-01-31T05:13:58.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.325+0000 7f3549f63700 1 --2- 172.21.15.45:0/864440163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35440ff4e0 0x7f35440ff900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:58.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.325+0000 7f3549f63700 1 -- 172.21.15.45:0/864440163 >> 172.21.15.45:0/864440163 conn(0x7f35440fae40 msgr2=0x7f35440fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.325+0000 7f3549f63700 1 -- 172.21.15.45:0/864440163 shutdown_connections 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.325+0000 7f3549f63700 1 -- 172.21.15.45:0/864440163 wait complete. 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.325+0000 7f3549f63700 1 Processor -- start 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.326+0000 7f3549f63700 1 -- start start 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.326+0000 7f3549f63700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35440ff4e0 0x7f3544101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.326+0000 7f3549f63700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3544102280 con 0x7f35440ff4e0 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.326+0000 7f35437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35440ff4e0 0x7f3544101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.326+0000 7f35437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35440ff4e0 0x7f3544101d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41130/0 (socket says 172.21.15.45:41130) 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.326+0000 7f35437fe700 1 -- 172.21.15.45:0/126857991 learned_addr learned my addr 172.21.15.45:0/126857991 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.326+0000 7f35437fe700 1 -- 172.21.15.45:0/126857991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0095f0 con 0x7f35440ff4e0 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.326+0000 7f35437fe700 1 --2- 172.21.15.45:0/126857991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35440ff4e0 0x7f3544101d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f352c004f70 tx=0x7f352c004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.327+0000 7f3540ff9700 1 -- 172.21.15.45:0/126857991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f352c01d070 con 0x7f35440ff4e0 2024-01-31T05:13:58.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.327+0000 7f3540ff9700 1 -- 172.21.15.45:0/126857991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f352c00be20 con 0x7f35440ff4e0 2024-01-31T05:13:58.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.327+0000 7f3540ff9700 1 -- 172.21.15.45:0/126857991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f352c022df0 con 0x7f35440ff4e0 2024-01-31T05:13:58.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.327+0000 7f3549f63700 1 -- 172.21.15.45:0/126857991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3544100470 con 0x7f35440ff4e0 2024-01-31T05:13:58.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.327+0000 7f3549f63700 1 -- 172.21.15.45:0/126857991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3544100910 con 0x7f35440ff4e0 2024-01-31T05:13:58.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.327+0000 7f3540ff9700 1 -- 172.21.15.45:0/126857991 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f352c00f460 con 0x7f35440ff4e0 2024-01-31T05:13:58.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.328+0000 7f3540ff9700 1 --2- 172.21.15.45:0/126857991 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3530038b40 0x7f353003b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:13:58.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.328+0000 7f3540ff9700 1 -- 172.21.15.45:0/126857991 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f352c00b960 con 0x7f35440ff4e0 2024-01-31T05:13:58.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.328+0000 7f3542ffd700 1 --2- 172.21.15.45:0/126857991 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3530038b40 0x7f353003b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:13:58.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.328+0000 7f3549f63700 1 -- 172.21.15.45:0/126857991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3544101320 con 0x7f35440ff4e0 2024-01-31T05:13:58.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.328+0000 7f3542ffd700 1 --2- 172.21.15.45:0/126857991 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3530038b40 0x7f353003b000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3534006fd0 tx=0x7f3534006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:13:58.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.333+0000 7f3540ff9700 1 -- 172.21.15.45:0/126857991 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f352c027070 con 0x7f35440ff4e0 2024-01-31T05:13:58.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.537+0000 7f3549f63700 1 -- 172.21.15.45:0/126857991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f3544107240 con 0x7f35440ff4e0 2024-01-31T05:13:58.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.539+0000 7f3540ff9700 1 -- 172.21.15.45:0/126857991 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f352c020780 con 0x7f35440ff4e0 2024-01-31T05:13:58.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.542+0000 7f3549f63700 1 -- 172.21.15.45:0/126857991 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3530038b40 msgr2=0x7f353003b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:58.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.542+0000 7f3549f63700 1 --2- 172.21.15.45:0/126857991 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3530038b40 0x7f353003b000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3534006fd0 tx=0x7f3534006e40 comp rx=0 tx=0).stop 2024-01-31T05:13:58.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.542+0000 7f3549f63700 1 -- 172.21.15.45:0/126857991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35440ff4e0 msgr2=0x7f3544101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:13:58.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.542+0000 7f3549f63700 1 --2- 172.21.15.45:0/126857991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35440ff4e0 0x7f3544101d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f352c004f70 tx=0x7f352c004fa0 comp rx=0 tx=0).stop 2024-01-31T05:13:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.542+0000 7f3549f63700 1 -- 172.21.15.45:0/126857991 shutdown_connections 2024-01-31T05:13:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.542+0000 7f3549f63700 1 --2- 172.21.15.45:0/126857991 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3530038b40 0x7f353003b000 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.542+0000 7f3549f63700 1 --2- 172.21.15.45:0/126857991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35440ff4e0 0x7f3544101d40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:13:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.542+0000 7f3549f63700 1 -- 172.21.15.45:0/126857991 >> 172.21.15.45:0/126857991 conn(0x7f35440fae40 msgr2=0x7f35440fb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:13:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.542+0000 7f3549f63700 1 -- 172.21.15.45:0/126857991 shutdown_connections 2024-01-31T05:13:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:13:58.542+0000 7f3549f63700 1 -- 172.21.15.45:0/126857991 wait complete. 2024-01-31T05:13:59.171 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-01-31T05:13:59.172 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-01-31T05:13:59.797 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:13:59 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/126857991' entity='client.admin' 2024-01-31T05:13:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:14:01.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:00 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:01.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:00 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:01.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:00 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:14:01.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:00 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:01.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:00 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:01.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:00 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:01.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.422+0000 7fab42411700 1 -- 172.21.15.45:0/1814209258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab3c101ca0 msgr2=0x7fab3c104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:01.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.422+0000 7fab42411700 1 --2- 172.21.15.45:0/1814209258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab3c101ca0 0x7fab3c104090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009910 tx=0x7fab2c009c20 comp rx=0 tx=0).stop 2024-01-31T05:14:01.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.422+0000 7fab42411700 1 -- 172.21.15.45:0/1814209258 shutdown_connections 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.422+0000 7fab42411700 1 --2- 172.21.15.45:0/1814209258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab3c101ca0 0x7fab3c104090 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.422+0000 7fab42411700 1 -- 172.21.15.45:0/1814209258 >> 172.21.15.45:0/1814209258 conn(0x7fab3c0fb6b0 msgr2=0x7fab3c0fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.422+0000 7fab42411700 1 -- 172.21.15.45:0/1814209258 shutdown_connections 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.422+0000 7fab42411700 1 -- 172.21.15.45:0/1814209258 wait complete. 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.422+0000 7fab42411700 1 Processor -- start 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.422+0000 7fab42411700 1 -- start start 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.423+0000 7fab42411700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab3c101ca0 0x7fab3c196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.423+0000 7fab42411700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c196b50 con 0x7fab3c101ca0 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.423+0000 7fab3bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab3c101ca0 0x7fab3c196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.423+0000 7fab3bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab3c101ca0 0x7fab3c196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41150/0 (socket says 172.21.15.45:41150) 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.423+0000 7fab3bfff700 1 -- 172.21.15.45:0/2881552942 learned_addr learned my addr 172.21.15.45:0/2881552942 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.423+0000 7fab3bfff700 1 -- 172.21.15.45:0/2881552942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c0095f0 con 0x7fab3c101ca0 2024-01-31T05:14:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.423+0000 7fab3bfff700 1 --2- 172.21.15.45:0/2881552942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab3c101ca0 0x7fab3c196610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab2c005b40 tx=0x7fab2c0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:01.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.423+0000 7fab397fa700 1 -- 172.21.15.45:0/2881552942 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab2c01d070 con 0x7fab3c101ca0 2024-01-31T05:14:01.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.424+0000 7fab42411700 1 -- 172.21.15.45:0/2881552942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c196d50 con 0x7fab3c101ca0 2024-01-31T05:14:01.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.424+0000 7fab397fa700 1 -- 172.21.15.45:0/2881552942 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab2c00be20 con 0x7fab3c101ca0 2024-01-31T05:14:01.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.424+0000 7fab397fa700 1 -- 172.21.15.45:0/2881552942 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab2c022d80 con 0x7fab3c101ca0 2024-01-31T05:14:01.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.424+0000 7fab42411700 1 -- 172.21.15.45:0/2881552942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c1971f0 con 0x7fab3c101ca0 2024-01-31T05:14:01.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.424+0000 7fab397fa700 1 -- 172.21.15.45:0/2881552942 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fab2c00f460 con 0x7fab3c101ca0 2024-01-31T05:14:01.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.425+0000 7fab42411700 1 -- 172.21.15.45:0/2881552942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3c18ff30 con 0x7fab3c101ca0 2024-01-31T05:14:01.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.425+0000 7fab397fa700 1 --2- 172.21.15.45:0/2881552942 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fab24038ae0 0x7fab2403afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:01.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.425+0000 7fab397fa700 1 -- 172.21.15.45:0/2881552942 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fab2c04cce0 con 0x7fab3c101ca0 2024-01-31T05:14:01.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.425+0000 7fab3b7fe700 1 --2- 172.21.15.45:0/2881552942 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fab24038ae0 0x7fab2403afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:01.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.425+0000 7fab3b7fe700 1 --2- 172.21.15.45:0/2881552942 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fab24038ae0 0x7fab2403afa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fab30006fd0 tx=0x7fab30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.430+0000 7fab397fa700 1 -- 172.21.15.45:0/2881552942 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fab2c00b960 con 0x7fab3c101ca0 2024-01-31T05:14:01.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.634+0000 7fab42411700 1 -- 172.21.15.45:0/2881552942 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fab3c02d4a0 con 0x7fab24038ae0 2024-01-31T05:14:01.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.637+0000 7fab397fa700 1 -- 172.21.15.45:0/2881552942 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fab3c02d4a0 con 0x7fab24038ae0 2024-01-31T05:14:01.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.639+0000 7fab42411700 1 -- 172.21.15.45:0/2881552942 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fab24038ae0 msgr2=0x7fab2403afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:01.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.639+0000 7fab42411700 1 --2- 172.21.15.45:0/2881552942 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fab24038ae0 0x7fab2403afa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fab30006fd0 tx=0x7fab30006e40 comp rx=0 tx=0).stop 2024-01-31T05:14:01.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.639+0000 7fab42411700 1 -- 172.21.15.45:0/2881552942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab3c101ca0 msgr2=0x7fab3c196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:01.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.639+0000 7fab42411700 1 --2- 172.21.15.45:0/2881552942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab3c101ca0 0x7fab3c196610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab2c005b40 tx=0x7fab2c0057e0 comp rx=0 tx=0).stop 2024-01-31T05:14:01.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.639+0000 7fab42411700 1 -- 172.21.15.45:0/2881552942 shutdown_connections 2024-01-31T05:14:01.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.639+0000 7fab42411700 1 --2- 172.21.15.45:0/2881552942 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fab24038ae0 0x7fab2403afa0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:01.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.639+0000 7fab42411700 1 --2- 172.21.15.45:0/2881552942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab3c101ca0 0x7fab3c196610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:01.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.639+0000 7fab42411700 1 -- 172.21.15.45:0/2881552942 >> 172.21.15.45:0/2881552942 conn(0x7fab3c0fb6b0 msgr2=0x7fab3c0fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:01.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.639+0000 7fab42411700 1 -- 172.21.15.45:0/2881552942 shutdown_connections 2024-01-31T05:14:01.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:01.639+0000 7fab42411700 1 -- 172.21.15.45:0/2881552942 wait complete. 2024-01-31T05:14:02.423 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi156 2024-01-31T05:14:02.423 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-01-31T05:14:02.423 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.conf 2024-01-31T05:14:02.441 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-01-31T05:14:02.441 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:14:02.499 INFO:tasks.cephadm:Adding host smithi156 to orchestrator... 2024-01-31T05:14:02.499 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch host add smithi156 2024-01-31T05:14:02.736 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:14:02.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:02 smithi045 ceph-mon[31585]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:14:02.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:02.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:02.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:02.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:02.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:02.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:02.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:03.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:03 smithi045 ceph-mon[31585]: Updating smithi045:/etc/ceph/ceph.conf 2024-01-31T05:14:03.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:03 smithi045 ceph-mon[31585]: Updating smithi045:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:03.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:03 smithi045 ceph-mon[31585]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:14:03.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:03 smithi045 ceph-mon[31585]: Updating smithi045:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:14:04.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.121+0000 7f8302353700 1 -- 172.21.15.45:0/2730013864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82fc101420 msgr2=0x7f82fc101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:04.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.121+0000 7f8302353700 1 --2- 172.21.15.45:0/2730013864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82fc101420 0x7f82fc101800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009910 tx=0x7f82ec009c20 comp rx=0 tx=0).stop 2024-01-31T05:14:04.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.121+0000 7f8302353700 1 -- 172.21.15.45:0/2730013864 shutdown_connections 2024-01-31T05:14:04.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.121+0000 7f8302353700 1 --2- 172.21.15.45:0/2730013864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82fc101420 0x7f82fc101800 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:04.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.121+0000 7f8302353700 1 -- 172.21.15.45:0/2730013864 >> 172.21.15.45:0/2730013864 conn(0x7f82fc0fccf0 msgr2=0x7f82fc0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.121+0000 7f8302353700 1 -- 172.21.15.45:0/2730013864 shutdown_connections 2024-01-31T05:14:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.121+0000 7f8302353700 1 -- 172.21.15.45:0/2730013864 wait complete. 2024-01-31T05:14:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.122+0000 7f8302353700 1 Processor -- start 2024-01-31T05:14:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.122+0000 7f8302353700 1 -- start start 2024-01-31T05:14:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.122+0000 7f8302353700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82fc101420 0x7f82fc19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.122+0000 7f8302353700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc19cbf0 con 0x7f82fc101420 2024-01-31T05:14:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.122+0000 7f82fbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82fc101420 0x7f82fc19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.122+0000 7f82fbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82fc101420 0x7f82fc19c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41166/0 (socket says 172.21.15.45:41166) 2024-01-31T05:14:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.122+0000 7f82fbfff700 1 -- 172.21.15.45:0/3755034932 learned_addr learned my addr 172.21.15.45:0/3755034932 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:14:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.122+0000 7f82fbfff700 1 -- 172.21.15.45:0/3755034932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec0095f0 con 0x7f82fc101420 2024-01-31T05:14:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.123+0000 7f82fbfff700 1 --2- 172.21.15.45:0/3755034932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82fc101420 0x7f82fc19c6b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82ec004f40 tx=0x7f82ec005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.123+0000 7f82f97fa700 1 -- 172.21.15.45:0/3755034932 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82ec01d070 con 0x7f82fc101420 2024-01-31T05:14:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.123+0000 7f8302353700 1 -- 172.21.15.45:0/3755034932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc19cdf0 con 0x7f82fc101420 2024-01-31T05:14:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.123+0000 7f82f97fa700 1 -- 172.21.15.45:0/3755034932 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82ec00bc10 con 0x7f82fc101420 2024-01-31T05:14:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.123+0000 7f82f97fa700 1 -- 172.21.15.45:0/3755034932 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82ec0037d0 con 0x7f82fc101420 2024-01-31T05:14:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.123+0000 7f8302353700 1 -- 172.21.15.45:0/3755034932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc19d210 con 0x7f82fc101420 2024-01-31T05:14:04.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.124+0000 7f82f97fa700 1 -- 172.21.15.45:0/3755034932 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f82ec003a90 con 0x7f82fc101420 2024-01-31T05:14:04.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.124+0000 7f8302353700 1 -- 172.21.15.45:0/3755034932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc062e10 con 0x7f82fc101420 2024-01-31T05:14:04.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.124+0000 7f82f97fa700 1 --2- 172.21.15.45:0/3755034932 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f82e4038ae0 0x7f82e403afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:04.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.124+0000 7f82f97fa700 1 -- 172.21.15.45:0/3755034932 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f82ec04c980 con 0x7f82fc101420 2024-01-31T05:14:04.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.124+0000 7f82fb7fe700 1 --2- 172.21.15.45:0/3755034932 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f82e4038ae0 0x7f82e403afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:04.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.124+0000 7f82fb7fe700 1 --2- 172.21.15.45:0/3755034932 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f82e4038ae0 0x7f82e403afa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f82f0006fd0 tx=0x7f82f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:04.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.129+0000 7f82f97fa700 1 -- 172.21.15.45:0/3755034932 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f82ec029360 con 0x7f82fc101420 2024-01-31T05:14:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:04.334+0000 7f8302353700 1 -- 172.21.15.45:0/3755034932 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}) v1 -- 0x7f82fc10b620 con 0x7f82e4038ae0 2024-01-31T05:14:05.098 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:04 smithi045 ceph-mon[31585]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:14:06.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:05 smithi045 ceph-mon[31585]: Deploying cephadm binary to smithi156 2024-01-31T05:14:07.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:07 smithi045 ceph-mon[31585]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:08.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:08.643+0000 7f82f97fa700 1 -- 172.21.15.45:0/3755034932 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f82fc10b620 con 0x7f82e4038ae0 2024-01-31T05:14:08.643 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi156' with addr '172.21.15.156' 2024-01-31T05:14:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:08.645+0000 7f8302353700 1 -- 172.21.15.45:0/3755034932 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f82e4038ae0 msgr2=0x7f82e403afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:08.645+0000 7f8302353700 1 --2- 172.21.15.45:0/3755034932 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f82e4038ae0 0x7f82e403afa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f82f0006fd0 tx=0x7f82f0006e40 comp rx=0 tx=0).stop 2024-01-31T05:14:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:08.645+0000 7f8302353700 1 -- 172.21.15.45:0/3755034932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82fc101420 msgr2=0x7f82fc19c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:08.645+0000 7f8302353700 1 --2- 172.21.15.45:0/3755034932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82fc101420 0x7f82fc19c6b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82ec004f40 tx=0x7f82ec005020 comp rx=0 tx=0).stop 2024-01-31T05:14:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:08.646+0000 7f8302353700 1 -- 172.21.15.45:0/3755034932 shutdown_connections 2024-01-31T05:14:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:08.646+0000 7f8302353700 1 --2- 172.21.15.45:0/3755034932 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f82e4038ae0 0x7f82e403afa0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:08.646+0000 7f8302353700 1 --2- 172.21.15.45:0/3755034932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82fc101420 0x7f82fc19c6b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:08.646+0000 7f8302353700 1 -- 172.21.15.45:0/3755034932 >> 172.21.15.45:0/3755034932 conn(0x7f82fc0fccf0 msgr2=0x7f82fc102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:08.646+0000 7f8302353700 1 -- 172.21.15.45:0/3755034932 shutdown_connections 2024-01-31T05:14:08.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:08.646+0000 7f8302353700 1 -- 172.21.15.45:0/3755034932 wait complete. 2024-01-31T05:14:09.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch host ls --format=json 2024-01-31T05:14:09.473 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:14:09.885 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:09 smithi045 ceph-mon[31585]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:09.885 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:09.885 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:09 smithi045 ceph-mon[31585]: Added host smithi156 2024-01-31T05:14:09.885 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:09.885 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:10.849 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:10 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:10.849 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:10 smithi045 ceph-mon[31585]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.878+0000 7fb40e17c700 1 -- 172.21.15.45:0/2622704773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4081023b0 msgr2=0x7fb408102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.878+0000 7fb40e17c700 1 --2- 172.21.15.45:0/2622704773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4081023b0 0x7fb408102790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009910 tx=0x7fb3f8009c20 comp rx=0 tx=0).stop 2024-01-31T05:14:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.878+0000 7fb40e17c700 1 -- 172.21.15.45:0/2622704773 shutdown_connections 2024-01-31T05:14:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.878+0000 7fb40e17c700 1 --2- 172.21.15.45:0/2622704773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4081023b0 0x7fb408102790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.878+0000 7fb40e17c700 1 -- 172.21.15.45:0/2622704773 >> 172.21.15.45:0/2622704773 conn(0x7fb4080fbc20 msgr2=0x7fb4080fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.878+0000 7fb40e17c700 1 -- 172.21.15.45:0/2622704773 shutdown_connections 2024-01-31T05:14:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.878+0000 7fb40e17c700 1 -- 172.21.15.45:0/2622704773 wait complete. 2024-01-31T05:14:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.878+0000 7fb40e17c700 1 Processor -- start 2024-01-31T05:14:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.878+0000 7fb40e17c700 1 -- start start 2024-01-31T05:14:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.879+0000 7fb40e17c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4081023b0 0x7fb40819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.879+0000 7fb40e17c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40819cc00 con 0x7fb4081023b0 2024-01-31T05:14:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.879+0000 7fb4077fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4081023b0 0x7fb40819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.879+0000 7fb4077fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4081023b0 0x7fb40819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35148/0 (socket says 172.21.15.45:35148) 2024-01-31T05:14:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.879+0000 7fb4077fe700 1 -- 172.21.15.45:0/2153271537 learned_addr learned my addr 172.21.15.45:0/2153271537 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:14:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.879+0000 7fb4077fe700 1 -- 172.21.15.45:0/2153271537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f80095f0 con 0x7fb4081023b0 2024-01-31T05:14:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.879+0000 7fb4077fe700 1 --2- 172.21.15.45:0/2153271537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4081023b0 0x7fb40819c6c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8004f40 tx=0x7fb3f8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.879+0000 7fb404ff9700 1 -- 172.21.15.45:0/2153271537 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3f801d070 con 0x7fb4081023b0 2024-01-31T05:14:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.879+0000 7fb404ff9700 1 -- 172.21.15.45:0/2153271537 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3f8004370 con 0x7fb4081023b0 2024-01-31T05:14:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.879+0000 7fb40e17c700 1 -- 172.21.15.45:0/2153271537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40819ce00 con 0x7fb4081023b0 2024-01-31T05:14:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.879+0000 7fb404ff9700 1 -- 172.21.15.45:0/2153271537 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3f8003970 con 0x7fb4081023b0 2024-01-31T05:14:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.879+0000 7fb40e17c700 1 -- 172.21.15.45:0/2153271537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40819d220 con 0x7fb4081023b0 2024-01-31T05:14:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.880+0000 7fb404ff9700 1 -- 172.21.15.45:0/2153271537 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fb3f8003c30 con 0x7fb4081023b0 2024-01-31T05:14:10.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.880+0000 7fb40e17c700 1 -- 172.21.15.45:0/2153271537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb408062e10 con 0x7fb4081023b0 2024-01-31T05:14:10.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.880+0000 7fb404ff9700 1 --2- 172.21.15.45:0/2153271537 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb3f0038ae0 0x7fb3f003afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:10.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.880+0000 7fb404ff9700 1 -- 172.21.15.45:0/2153271537 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb3f804c800 con 0x7fb4081023b0 2024-01-31T05:14:10.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.880+0000 7fb406ffd700 1 --2- 172.21.15.45:0/2153271537 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb3f0038ae0 0x7fb3f003afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:10.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.881+0000 7fb406ffd700 1 --2- 172.21.15.45:0/2153271537 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb3f0038ae0 0x7fb3f003afa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc006fd0 tx=0x7fb3fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:10.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:10.885+0000 7fb404ff9700 1 -- 172.21.15.45:0/2153271537 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb3f8029950 con 0x7fb4081023b0 2024-01-31T05:14:11.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:11.090+0000 7fb40e17c700 1 -- 172.21.15.45:0/2153271537 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb40810b5b0 con 0x7fb3f0038ae0 2024-01-31T05:14:11.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:11.091+0000 7fb404ff9700 1 -- 172.21.15.45:0/2153271537 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fb40810b5b0 con 0x7fb3f0038ae0 2024-01-31T05:14:11.091 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:14:11.091 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}] 2024-01-31T05:14:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:11.093+0000 7fb40e17c700 1 -- 172.21.15.45:0/2153271537 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb3f0038ae0 msgr2=0x7fb3f003afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:11.093+0000 7fb40e17c700 1 --2- 172.21.15.45:0/2153271537 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb3f0038ae0 0x7fb3f003afa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc006fd0 tx=0x7fb3fc006e40 comp rx=0 tx=0).stop 2024-01-31T05:14:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:11.093+0000 7fb40e17c700 1 -- 172.21.15.45:0/2153271537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4081023b0 msgr2=0x7fb40819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:11.093+0000 7fb40e17c700 1 --2- 172.21.15.45:0/2153271537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4081023b0 0x7fb40819c6c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8004f40 tx=0x7fb3f8005020 comp rx=0 tx=0).stop 2024-01-31T05:14:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:11.093+0000 7fb40e17c700 1 -- 172.21.15.45:0/2153271537 shutdown_connections 2024-01-31T05:14:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:11.093+0000 7fb40e17c700 1 --2- 172.21.15.45:0/2153271537 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb3f0038ae0 0x7fb3f003afa0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:11.093+0000 7fb40e17c700 1 --2- 172.21.15.45:0/2153271537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4081023b0 0x7fb40819c6c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:11.093+0000 7fb40e17c700 1 -- 172.21.15.45:0/2153271537 >> 172.21.15.45:0/2153271537 conn(0x7fb4080fbc20 msgr2=0x7fb408103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:11.094+0000 7fb40e17c700 1 -- 172.21.15.45:0/2153271537 shutdown_connections 2024-01-31T05:14:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:11.094+0000 7fb40e17c700 1 -- 172.21.15.45:0/2153271537 wait complete. 2024-01-31T05:14:11.713 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi186 2024-01-31T05:14:11.713 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-01-31T05:14:11.714 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.conf 2024-01-31T05:14:11.733 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-01-31T05:14:11.733 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:14:11.790 INFO:tasks.cephadm:Adding host smithi186 to orchestrator... 2024-01-31T05:14:11.790 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch host add smithi186 2024-01-31T05:14:12.027 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:14:12.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:12 smithi045 ceph-mon[31585]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:14:12.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:12 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.448+0000 7fdac6c1c700 1 -- 172.21.15.45:0/857317409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdac0102410 msgr2=0x7fdac01027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.448+0000 7fdac6c1c700 1 --2- 172.21.15.45:0/857317409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdac0102410 0x7fdac01027f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdab800b0a0 tx=0x7fdab800b3b0 comp rx=0 tx=0).stop 2024-01-31T05:14:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.448+0000 7fdac6c1c700 1 -- 172.21.15.45:0/857317409 shutdown_connections 2024-01-31T05:14:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.448+0000 7fdac6c1c700 1 --2- 172.21.15.45:0/857317409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdac0102410 0x7fdac01027f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.448+0000 7fdac6c1c700 1 -- 172.21.15.45:0/857317409 >> 172.21.15.45:0/857317409 conn(0x7fdac0076d20 msgr2=0x7fdac00fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.448+0000 7fdac6c1c700 1 -- 172.21.15.45:0/857317409 shutdown_connections 2024-01-31T05:14:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.448+0000 7fdac6c1c700 1 -- 172.21.15.45:0/857317409 wait complete. 2024-01-31T05:14:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.448+0000 7fdac6c1c700 1 Processor -- start 2024-01-31T05:14:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.448+0000 7fdac6c1c700 1 -- start start 2024-01-31T05:14:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.449+0000 7fdac6c1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdac0102410 0x7fdac0100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.449+0000 7fdac6c1c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac0101f60 con 0x7fdac0102410 2024-01-31T05:14:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.449+0000 7fdac49b8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdac0102410 0x7fdac0100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.449+0000 7fdac49b8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdac0102410 0x7fdac0100070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35170/0 (socket says 172.21.15.45:35170) 2024-01-31T05:14:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.449+0000 7fdac49b8700 1 -- 172.21.15.45:0/3689053012 learned_addr learned my addr 172.21.15.45:0/3689053012 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.449+0000 7fdac49b8700 1 -- 172.21.15.45:0/3689053012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8009d00 con 0x7fdac0102410 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.449+0000 7fdac49b8700 1 --2- 172.21.15.45:0/3689053012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdac0102410 0x7fdac0100070 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdab8015040 tx=0x7fdab80096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.449+0000 7fdabdffb700 1 -- 172.21.15.45:0/3689053012 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdab800e040 con 0x7fdac0102410 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.449+0000 7fdabdffb700 1 -- 172.21.15.45:0/3689053012 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdab8004040 con 0x7fdac0102410 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.449+0000 7fdac6c1c700 1 -- 172.21.15.45:0/3689053012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac01005b0 con 0x7fdac0102410 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.449+0000 7fdabdffb700 1 -- 172.21.15.45:0/3689053012 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdab801d610 con 0x7fdac0102410 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.450+0000 7fdac6c1c700 1 -- 172.21.15.45:0/3689053012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac01009d0 con 0x7fdac0102410 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.450+0000 7fdabdffb700 1 -- 172.21.15.45:0/3689053012 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fdab801d880 con 0x7fdac0102410 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.450+0000 7fdac6c1c700 1 -- 172.21.15.45:0/3689053012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac0062e10 con 0x7fdac0102410 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.451+0000 7fdabdffb700 1 --2- 172.21.15.45:0/3689053012 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdaa8038b40 0x7fdaa803b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.451+0000 7fdabdffb700 1 -- 172.21.15.45:0/3689053012 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdab804cdb0 con 0x7fdac0102410 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.451+0000 7fdabffff700 1 --2- 172.21.15.45:0/3689053012 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdaa8038b40 0x7fdaa803b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.451+0000 7fdabffff700 1 --2- 172.21.15.45:0/3689053012 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdaa8038b40 0x7fdaa803b000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdab4006fd0 tx=0x7fdab4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.455+0000 7fdabdffb700 1 -- 172.21.15.45:0/3689053012 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fdab8025360 con 0x7fdac0102410 2024-01-31T05:14:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:13 smithi045 ceph-mon[31585]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:13.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:13.662+0000 7fdac6c1c700 1 -- 172.21.15.45:0/3689053012 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}) v1 -- 0x7fdac0100c80 con 0x7fdaa8038b40 2024-01-31T05:14:15.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:15 smithi045 ceph-mon[31585]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:14:15.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:15 smithi045 ceph-mon[31585]: Deploying cephadm binary to smithi186 2024-01-31T05:14:15.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:15.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:15.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:15.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:15.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:14:15.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:15.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:15.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:15 smithi045 ceph-mon[31585]: Updating smithi156:/etc/ceph/ceph.conf 2024-01-31T05:14:15.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:15 smithi045 ceph-mon[31585]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:15.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:15 smithi045 ceph-mon[31585]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:16.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:16 smithi045 ceph-mon[31585]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:14:17.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:17 smithi045 ceph-mon[31585]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:17.991+0000 7fdabdffb700 1 -- 172.21.15.45:0/3689053012 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fdac0100c80 con 0x7fdaa8038b40 2024-01-31T05:14:17.991 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi186' with addr '172.21.15.186' 2024-01-31T05:14:17.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:17.994+0000 7fdac6c1c700 1 -- 172.21.15.45:0/3689053012 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdaa8038b40 msgr2=0x7fdaa803b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:17.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:17.994+0000 7fdac6c1c700 1 --2- 172.21.15.45:0/3689053012 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdaa8038b40 0x7fdaa803b000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdab4006fd0 tx=0x7fdab4006e40 comp rx=0 tx=0).stop 2024-01-31T05:14:17.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:17.994+0000 7fdac6c1c700 1 -- 172.21.15.45:0/3689053012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdac0102410 msgr2=0x7fdac0100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:17.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:17.994+0000 7fdac6c1c700 1 --2- 172.21.15.45:0/3689053012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdac0102410 0x7fdac0100070 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdab8015040 tx=0x7fdab80096e0 comp rx=0 tx=0).stop 2024-01-31T05:14:17.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:17.994+0000 7fdac6c1c700 1 -- 172.21.15.45:0/3689053012 shutdown_connections 2024-01-31T05:14:17.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:17.994+0000 7fdac6c1c700 1 --2- 172.21.15.45:0/3689053012 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdaa8038b40 0x7fdaa803b000 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:17.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:17.994+0000 7fdac6c1c700 1 --2- 172.21.15.45:0/3689053012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdac0102410 0x7fdac0100070 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:17.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:17.994+0000 7fdac6c1c700 1 -- 172.21.15.45:0/3689053012 >> 172.21.15.45:0/3689053012 conn(0x7fdac0076d20 msgr2=0x7fdac01031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:17.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:17.994+0000 7fdac6c1c700 1 -- 172.21.15.45:0/3689053012 shutdown_connections 2024-01-31T05:14:17.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:17.994+0000 7fdac6c1c700 1 -- 172.21.15.45:0/3689053012 wait complete. 2024-01-31T05:14:18.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:18 smithi045 ceph-mon[31585]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:14:18.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:18 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:18.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:18 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:18.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:18 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:18.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:18 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:18.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:18 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:18.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch host ls --format=json 2024-01-31T05:14:18.813 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:14:19.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:19 smithi045 ceph-mon[31585]: Added host smithi186 2024-01-31T05:14:19.819 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:19.819 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:19 smithi045 ceph-mon[31585]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:19.819 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.305+0000 7f587f906700 1 -- 172.21.15.45:0/564694156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5878100a60 msgr2=0x7f5878100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.305+0000 7f587f906700 1 --2- 172.21.15.45:0/564694156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5878100a60 0x7f5878100e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5874009910 tx=0x7f5874009c20 comp rx=0 tx=0).stop 2024-01-31T05:14:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.305+0000 7f587f906700 1 -- 172.21.15.45:0/564694156 shutdown_connections 2024-01-31T05:14:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.305+0000 7f587f906700 1 --2- 172.21.15.45:0/564694156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5878100a60 0x7f5878100e40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.305+0000 7f587f906700 1 -- 172.21.15.45:0/564694156 >> 172.21.15.45:0/564694156 conn(0x7f58780fb330 msgr2=0x7f58780fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.305+0000 7f587f906700 1 -- 172.21.15.45:0/564694156 shutdown_connections 2024-01-31T05:14:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.305+0000 7f587f906700 1 -- 172.21.15.45:0/564694156 wait complete. 2024-01-31T05:14:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.305+0000 7f587f906700 1 Processor -- start 2024-01-31T05:14:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.305+0000 7f587f906700 1 -- start start 2024-01-31T05:14:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.306+0000 7f587f906700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5878100a60 0x7f5878101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.306+0000 7f587f906700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5878103ed0 con 0x7f5878100a60 2024-01-31T05:14:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.306+0000 7f587d6a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5878100a60 0x7f5878101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.306+0000 7f587d6a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5878100a60 0x7f5878101fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53962/0 (socket says 172.21.15.45:53962) 2024-01-31T05:14:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.306+0000 7f587d6a2700 1 -- 172.21.15.45:0/597672828 learned_addr learned my addr 172.21.15.45:0/597672828 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:14:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.306+0000 7f587d6a2700 1 -- 172.21.15.45:0/597672828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740095f0 con 0x7f5878100a60 2024-01-31T05:14:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.306+0000 7f587d6a2700 1 --2- 172.21.15.45:0/597672828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5878100a60 0x7f5878101fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5874006010 tx=0x7f5874004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.306+0000 7f586a7fc700 1 -- 172.21.15.45:0/597672828 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f587401d070 con 0x7f5878100a60 2024-01-31T05:14:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.306+0000 7f587f906700 1 -- 172.21.15.45:0/597672828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5878102520 con 0x7f5878100a60 2024-01-31T05:14:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.306+0000 7f586a7fc700 1 -- 172.21.15.45:0/597672828 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f587400bc10 con 0x7f5878100a60 2024-01-31T05:14:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.306+0000 7f586a7fc700 1 -- 172.21.15.45:0/597672828 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58740038a0 con 0x7f5878100a60 2024-01-31T05:14:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.306+0000 7f587f906700 1 -- 172.21.15.45:0/597672828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5878102940 con 0x7f5878100a60 2024-01-31T05:14:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.307+0000 7f586a7fc700 1 -- 172.21.15.45:0/597672828 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f5874003ad0 con 0x7f5878100a60 2024-01-31T05:14:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.307+0000 7f586a7fc700 1 --2- 172.21.15.45:0/597672828 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5864038ae0 0x7f586403afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.307+0000 7f586a7fc700 1 -- 172.21.15.45:0/597672828 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f587404c7a0 con 0x7f5878100a60 2024-01-31T05:14:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.307+0000 7f587f906700 1 -- 172.21.15.45:0/597672828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5878062e10 con 0x7f5878100a60 2024-01-31T05:14:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.307+0000 7f587cea1700 1 --2- 172.21.15.45:0/597672828 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5864038ae0 0x7f586403afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.308+0000 7f587cea1700 1 --2- 172.21.15.45:0/597672828 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5864038ae0 0x7f586403afa0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f586c006fd0 tx=0x7f586c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.312+0000 7f586a7fc700 1 -- 172.21.15.45:0/597672828 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5874026080 con 0x7f5878100a60 2024-01-31T05:14:20.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.518+0000 7f587f906700 1 -- 172.21.15.45:0/597672828 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58780689b0 con 0x7f5864038ae0 2024-01-31T05:14:20.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.519+0000 7f586a7fc700 1 -- 172.21.15.45:0/597672828 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f58780689b0 con 0x7f5864038ae0 2024-01-31T05:14:20.518 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:14:20.518 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}] 2024-01-31T05:14:20.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.521+0000 7f587f906700 1 -- 172.21.15.45:0/597672828 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5864038ae0 msgr2=0x7f586403afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:20.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.521+0000 7f587f906700 1 --2- 172.21.15.45:0/597672828 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5864038ae0 0x7f586403afa0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f586c006fd0 tx=0x7f586c006e40 comp rx=0 tx=0).stop 2024-01-31T05:14:20.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.521+0000 7f587f906700 1 -- 172.21.15.45:0/597672828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5878100a60 msgr2=0x7f5878101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:20.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.521+0000 7f587f906700 1 --2- 172.21.15.45:0/597672828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5878100a60 0x7f5878101fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5874006010 tx=0x7f5874004ff0 comp rx=0 tx=0).stop 2024-01-31T05:14:20.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.521+0000 7f587f906700 1 -- 172.21.15.45:0/597672828 shutdown_connections 2024-01-31T05:14:20.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.521+0000 7f587f906700 1 --2- 172.21.15.45:0/597672828 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5864038ae0 0x7f586403afa0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:20.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.521+0000 7f587f906700 1 --2- 172.21.15.45:0/597672828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5878100a60 0x7f5878101fe0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:20.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.521+0000 7f587f906700 1 -- 172.21.15.45:0/597672828 >> 172.21.15.45:0/597672828 conn(0x7f58780fb330 msgr2=0x7f58780ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:20.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.521+0000 7f587f906700 1 -- 172.21.15.45:0/597672828 shutdown_connections 2024-01-31T05:14:20.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:20.521+0000 7f587f906700 1 -- 172.21.15.45:0/597672828 wait complete. 2024-01-31T05:14:21.097 INFO:tasks.cephadm:Setting crush tunables to default 2024-01-31T05:14:21.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd crush tunables default 2024-01-31T05:14:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:14:21.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:21.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:21 smithi045 ceph-mon[31585]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:14:21.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:21 smithi045 ceph-mon[31585]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:22.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.868+0000 7feccbaed700 1 -- 172.21.15.45:0/3412078357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc4105840 msgr2=0x7fecc4105c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:22.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.868+0000 7feccbaed700 1 --2- 172.21.15.45:0/3412078357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc4105840 0x7fecc4105c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009910 tx=0x7fecb8009c20 comp rx=0 tx=0).stop 2024-01-31T05:14:22.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.868+0000 7feccbaed700 1 -- 172.21.15.45:0/3412078357 shutdown_connections 2024-01-31T05:14:22.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.868+0000 7feccbaed700 1 --2- 172.21.15.45:0/3412078357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc4105840 0x7fecc4105c20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:22.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.868+0000 7feccbaed700 1 -- 172.21.15.45:0/3412078357 >> 172.21.15.45:0/3412078357 conn(0x7fecc40fb330 msgr2=0x7fecc40fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:22.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.869+0000 7feccbaed700 1 -- 172.21.15.45:0/3412078357 shutdown_connections 2024-01-31T05:14:22.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.869+0000 7feccbaed700 1 -- 172.21.15.45:0/3412078357 wait complete. 2024-01-31T05:14:22.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.869+0000 7feccbaed700 1 Processor -- start 2024-01-31T05:14:22.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.869+0000 7feccbaed700 1 -- start start 2024-01-31T05:14:22.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.870+0000 7feccbaed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc4105840 0x7fecc419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:22.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.870+0000 7feccbaed700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc419bb50 con 0x7fecc4105840 2024-01-31T05:14:22.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.870+0000 7fecc9889700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc4105840 0x7fecc419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:22.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.870+0000 7fecc9889700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc4105840 0x7fecc419b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53974/0 (socket says 172.21.15.45:53974) 2024-01-31T05:14:22.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.870+0000 7fecc9889700 1 -- 172.21.15.45:0/3931195139 learned_addr learned my addr 172.21.15.45:0/3931195139 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:14:22.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.870+0000 7fecc9889700 1 -- 172.21.15.45:0/3931195139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb80095f0 con 0x7fecc4105840 2024-01-31T05:14:22.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.870+0000 7fecc9889700 1 --2- 172.21.15.45:0/3931195139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc4105840 0x7fecc419b610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fecb8004f40 tx=0x7fecb8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:22.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.870+0000 7fecb6ffd700 1 -- 172.21.15.45:0/3931195139 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecb801d070 con 0x7fecc4105840 2024-01-31T05:14:22.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.870+0000 7fecb6ffd700 1 -- 172.21.15.45:0/3931195139 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecb800bbb0 con 0x7fecc4105840 2024-01-31T05:14:22.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.870+0000 7feccbaed700 1 -- 172.21.15.45:0/3931195139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc419bdb0 con 0x7fecc4105840 2024-01-31T05:14:22.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.870+0000 7fecb6ffd700 1 -- 172.21.15.45:0/3931195139 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecb8003810 con 0x7fecc4105840 2024-01-31T05:14:22.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.870+0000 7feccbaed700 1 -- 172.21.15.45:0/3931195139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc41956e0 con 0x7fecc4105840 2024-01-31T05:14:22.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.871+0000 7fecb6ffd700 1 -- 172.21.15.45:0/3931195139 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fecb800b6f0 con 0x7fecc4105840 2024-01-31T05:14:22.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.871+0000 7feccbaed700 1 -- 172.21.15.45:0/3931195139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc4100f80 con 0x7fecc4105840 2024-01-31T05:14:22.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.871+0000 7fecb6ffd700 1 --2- 172.21.15.45:0/3931195139 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fecb0038ae0 0x7fecb003afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:22.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.872+0000 7fecb6ffd700 1 -- 172.21.15.45:0/3931195139 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fecb804c750 con 0x7fecc4105840 2024-01-31T05:14:22.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.872+0000 7fecc9088700 1 --2- 172.21.15.45:0/3931195139 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fecb0038ae0 0x7fecb003afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:22.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.872+0000 7fecc9088700 1 --2- 172.21.15.45:0/3931195139 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fecb0038ae0 0x7fecb003afa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fecc0006fd0 tx=0x7fecc0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:22.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:22.876+0000 7fecb6ffd700 1 -- 172.21.15.45:0/3931195139 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fecb8029360 con 0x7fecc4105840 2024-01-31T05:14:23.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:23.082+0000 7feccbaed700 1 -- 172.21.15.45:0/3931195139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fecc4109b40 con 0x7fecc4105840 2024-01-31T05:14:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:23.364+0000 7fecb6ffd700 1 -- 172.21.15.45:0/3931195139 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fecb8026020 con 0x7fecc4105840 2024-01-31T05:14:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:23.368+0000 7feccbaed700 1 -- 172.21.15.45:0/3931195139 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fecb0038ae0 msgr2=0x7fecb003afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:23.368+0000 7feccbaed700 1 --2- 172.21.15.45:0/3931195139 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fecb0038ae0 0x7fecb003afa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fecc0006fd0 tx=0x7fecc0006e40 comp rx=0 tx=0).stop 2024-01-31T05:14:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:23.368+0000 7feccbaed700 1 -- 172.21.15.45:0/3931195139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc4105840 msgr2=0x7fecc419b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:23.368+0000 7feccbaed700 1 --2- 172.21.15.45:0/3931195139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc4105840 0x7fecc419b610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fecb8004f40 tx=0x7fecb8005020 comp rx=0 tx=0).stop 2024-01-31T05:14:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:23.368+0000 7feccbaed700 1 -- 172.21.15.45:0/3931195139 shutdown_connections 2024-01-31T05:14:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:23.368+0000 7feccbaed700 1 --2- 172.21.15.45:0/3931195139 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fecb0038ae0 0x7fecb003afa0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:23.368+0000 7feccbaed700 1 --2- 172.21.15.45:0/3931195139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc4105840 0x7fecc419b610 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:23.368+0000 7feccbaed700 1 -- 172.21.15.45:0/3931195139 >> 172.21.15.45:0/3931195139 conn(0x7fecc40fb330 msgr2=0x7fecc40fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:23.368+0000 7feccbaed700 1 -- 172.21.15.45:0/3931195139 shutdown_connections 2024-01-31T05:14:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:23.368+0000 7feccbaed700 1 -- 172.21.15.45:0/3931195139 wait complete. 2024-01-31T05:14:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-01-31T05:14:23.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:23 smithi045 ceph-mon[31585]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:23.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:23 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3931195139' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-01-31T05:14:23.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:23.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:23.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:23.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:23.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:14:23.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:23.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:24.032 INFO:tasks.cephadm:Adding mon.a on smithi045 2024-01-31T05:14:24.032 INFO:tasks.cephadm:Adding mon.b on smithi156 2024-01-31T05:14:24.032 INFO:tasks.cephadm:Adding mon.c on smithi186 2024-01-31T05:14:24.032 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch apply mon '3;smithi045:172.21.15.45=a;smithi156:172.21.15.156=b;smithi186:172.21.15.186=c' 2024-01-31T05:14:24.265 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:24.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:24 smithi045 ceph-mon[31585]: Updating smithi186:/etc/ceph/ceph.conf 2024-01-31T05:14:24.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:24 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3931195139' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-01-31T05:14:24.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:24 smithi045 ceph-mon[31585]: osdmap e4: 0 total, 0 up, 0 in 2024-01-31T05:14:24.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:24 smithi045 ceph-mon[31585]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:24.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:24 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:24.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:24 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:24.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:24 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:25.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:25 smithi045 ceph-mon[31585]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:14:25.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:25 smithi045 ceph-mon[31585]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:14:25.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:25 smithi045 ceph-mon[31585]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:25.731 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.730+0000 7f7619372700 1 -- 172.21.15.186:0/1243039117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7614102390 msgr2=0x7f7614102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:25.732 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.730+0000 7f7619372700 1 --2- 172.21.15.186:0/1243039117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7614102390 0x7f7614102770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7604009910 tx=0x7f7604009c20 comp rx=0 tx=0).stop 2024-01-31T05:14:25.732 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.730+0000 7f7619372700 1 -- 172.21.15.186:0/1243039117 shutdown_connections 2024-01-31T05:14:25.732 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.730+0000 7f7619372700 1 --2- 172.21.15.186:0/1243039117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7614102390 0x7f7614102770 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:25.732 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.730+0000 7f7619372700 1 -- 172.21.15.186:0/1243039117 >> 172.21.15.186:0/1243039117 conn(0x7f76140fbbe0 msgr2=0x7f76140fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:25.732 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.730+0000 7f7619372700 1 -- 172.21.15.186:0/1243039117 shutdown_connections 2024-01-31T05:14:25.732 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.730+0000 7f7619372700 1 -- 172.21.15.186:0/1243039117 wait complete. 2024-01-31T05:14:25.732 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.731+0000 7f7619372700 1 Processor -- start 2024-01-31T05:14:25.732 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.731+0000 7f7619372700 1 -- start start 2024-01-31T05:14:25.732 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.731+0000 7f7619372700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7614102390 0x7f761419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:25.732 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.731+0000 7f7619372700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761419cc50 con 0x7f7614102390 2024-01-31T05:14:25.732 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.731+0000 7f7612ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7614102390 0x7f761419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:25.732 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.732+0000 7f7612ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7614102390 0x7f761419c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.186:52342/0 (socket says 172.21.15.186:52342) 2024-01-31T05:14:25.733 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.732+0000 7f7612ffd700 1 -- 172.21.15.186:0/4274789455 learned_addr learned my addr 172.21.15.186:0/4274789455 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-01-31T05:14:25.733 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.732+0000 7f7612ffd700 1 -- 172.21.15.186:0/4274789455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76040095f0 con 0x7f7614102390 2024-01-31T05:14:25.733 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.732+0000 7f7612ffd700 1 --2- 172.21.15.186:0/4274789455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7614102390 0x7f761419c710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7604004f40 tx=0x7f7604005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:25.733 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.732+0000 7f75fbfff700 1 -- 172.21.15.186:0/4274789455 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f760401d070 con 0x7f7614102390 2024-01-31T05:14:25.733 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.732+0000 7f7619372700 1 -- 172.21.15.186:0/4274789455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761419ce50 con 0x7f7614102390 2024-01-31T05:14:25.733 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.732+0000 7f75fbfff700 1 -- 172.21.15.186:0/4274789455 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7604004370 con 0x7f7614102390 2024-01-31T05:14:25.733 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.732+0000 7f75fbfff700 1 -- 172.21.15.186:0/4274789455 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7604003970 con 0x7f7614102390 2024-01-31T05:14:25.733 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.732+0000 7f7619372700 1 -- 172.21.15.186:0/4274789455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761419d270 con 0x7f7614102390 2024-01-31T05:14:25.734 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.733+0000 7f7619372700 1 -- 172.21.15.186:0/4274789455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7614062e10 con 0x7f7614102390 2024-01-31T05:14:25.734 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.733+0000 7f75fbfff700 1 -- 172.21.15.186:0/4274789455 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f7604003c30 con 0x7f7614102390 2024-01-31T05:14:25.734 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.733+0000 7f75fbfff700 1 --2- 172.21.15.186:0/4274789455 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f75fc038ae0 0x7f75fc03afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:25.734 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.733+0000 7f75fbfff700 1 -- 172.21.15.186:0/4274789455 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f760404c8a0 con 0x7f7614102390 2024-01-31T05:14:25.734 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.734+0000 7f76127fc700 1 --2- 172.21.15.186:0/4274789455 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f75fc038ae0 0x7f75fc03afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:25.734 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.734+0000 7f76127fc700 1 --2- 172.21.15.186:0/4274789455 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f75fc038ae0 0x7f75fc03afa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7608006fd0 tx=0x7f7608006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:25.738 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.739+0000 7f75fbfff700 1 -- 172.21.15.186:0/4274789455 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f760402a3b0 con 0x7f7614102390 2024-01-31T05:14:25.938 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.938+0000 7f7619372700 1 -- 172.21.15.186:0/4274789455 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi156:172.21.15.156=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f761410b640 con 0x7f75fc038ae0 2024-01-31T05:14:25.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.945+0000 7f75fbfff700 1 -- 172.21.15.186:0/4274789455 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f761410b640 con 0x7f75fc038ae0 2024-01-31T05:14:25.944 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mon update... 2024-01-31T05:14:25.946 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.947+0000 7f7619372700 1 -- 172.21.15.186:0/4274789455 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f75fc038ae0 msgr2=0x7f75fc03afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:25.946 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.947+0000 7f7619372700 1 --2- 172.21.15.186:0/4274789455 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f75fc038ae0 0x7f75fc03afa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7608006fd0 tx=0x7f7608006e40 comp rx=0 tx=0).stop 2024-01-31T05:14:25.946 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.947+0000 7f7619372700 1 -- 172.21.15.186:0/4274789455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7614102390 msgr2=0x7f761419c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:25.946 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.947+0000 7f7619372700 1 --2- 172.21.15.186:0/4274789455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7614102390 0x7f761419c710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7604004f40 tx=0x7f7604005020 comp rx=0 tx=0).stop 2024-01-31T05:14:25.946 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.947+0000 7f7619372700 1 -- 172.21.15.186:0/4274789455 shutdown_connections 2024-01-31T05:14:25.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.947+0000 7f7619372700 1 --2- 172.21.15.186:0/4274789455 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f75fc038ae0 0x7f75fc03afa0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:25.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.947+0000 7f7619372700 1 --2- 172.21.15.186:0/4274789455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7614102390 0x7f761419c710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:25.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.947+0000 7f7619372700 1 -- 172.21.15.186:0/4274789455 >> 172.21.15.186:0/4274789455 conn(0x7f76140fbbe0 msgr2=0x7f7614103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:25.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.947+0000 7f7619372700 1 -- 172.21.15.186:0/4274789455 shutdown_connections 2024-01-31T05:14:25.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:25.947+0000 7f7619372700 1 -- 172.21.15.186:0/4274789455 wait complete. 2024-01-31T05:14:26.569 DEBUG:teuthology.orchestra.run.smithi156:mon.b> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.b.service 2024-01-31T05:14:26.572 DEBUG:teuthology.orchestra.run.smithi186:mon.c> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.c.service 2024-01-31T05:14:26.574 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-01-31T05:14:26.575 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph mon dump -f json 2024-01-31T05:14:27.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:26 smithi045 ceph-mon[31585]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi156:172.21.15.156=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:14:27.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:26 smithi045 ceph-mon[31585]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi156:172.21.15.156=b;smithi186:172.21.15.186=c;count:3 2024-01-31T05:14:27.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:26 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:27.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:26 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:27.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:26 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:27.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:26 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:27.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:26 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:27.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:26 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:27.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:26 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:27.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:26 smithi045 ceph-mon[31585]: Deploying daemon mon.c on smithi186 2024-01-31T05:14:27.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:26 smithi045 ceph-mon[31585]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:27.597 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:29.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:29 smithi045 ceph-mon[31585]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:30.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.138+0000 7f152817c700 1 -- 172.21.15.186:0/2384542560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1520100310 msgr2=0x7f15201006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:30.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.138+0000 7f152817c700 1 --2- 172.21.15.186:0/2384542560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1520100310 0x7f15201006f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f151c009910 tx=0x7f151c009c20 comp rx=0 tx=0).stop 2024-01-31T05:14:30.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.138+0000 7f152817c700 1 -- 172.21.15.186:0/2384542560 shutdown_connections 2024-01-31T05:14:30.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.138+0000 7f152817c700 1 --2- 172.21.15.186:0/2384542560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1520100310 0x7f15201006f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:30.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.138+0000 7f152817c700 1 -- 172.21.15.186:0/2384542560 >> 172.21.15.186:0/2384542560 conn(0x7f1520076d20 msgr2=0x7f15200fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:30.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.138+0000 7f152817c700 1 -- 172.21.15.186:0/2384542560 shutdown_connections 2024-01-31T05:14:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.138+0000 7f152817c700 1 -- 172.21.15.186:0/2384542560 wait complete. 2024-01-31T05:14:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.139+0000 7f152817c700 1 Processor -- start 2024-01-31T05:14:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.139+0000 7f152817c700 1 -- start start 2024-01-31T05:14:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.139+0000 7f152817c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1520100310 0x7f1520101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.139+0000 7f152817c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1520103ca0 con 0x7f1520100310 2024-01-31T05:14:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.139+0000 7f1525f18700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1520100310 0x7f1520101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.139+0000 7f1525f18700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1520100310 0x7f1520101db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.186:36022/0 (socket says 172.21.15.186:36022) 2024-01-31T05:14:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.139+0000 7f1525f18700 1 -- 172.21.15.186:0/974036539 learned_addr learned my addr 172.21.15.186:0/974036539 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-01-31T05:14:30.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.140+0000 7f1525f18700 1 -- 172.21.15.186:0/974036539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c0095f0 con 0x7f1520100310 2024-01-31T05:14:30.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.140+0000 7f1525f18700 1 --2- 172.21.15.186:0/974036539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1520100310 0x7f1520101db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f151c004f40 tx=0x7f151c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:30.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.140+0000 7f1516ffd700 1 -- 172.21.15.186:0/974036539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f151c01d070 con 0x7f1520100310 2024-01-31T05:14:30.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.140+0000 7f1516ffd700 1 -- 172.21.15.186:0/974036539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151c00bc10 con 0x7f1520100310 2024-01-31T05:14:30.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.140+0000 7f152817c700 1 -- 172.21.15.186:0/974036539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15201022f0 con 0x7f1520100310 2024-01-31T05:14:30.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.140+0000 7f152817c700 1 -- 172.21.15.186:0/974036539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1520102710 con 0x7f1520100310 2024-01-31T05:14:30.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.141+0000 7f1516ffd700 1 -- 172.21.15.186:0/974036539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f151c0037d0 con 0x7f1520100310 2024-01-31T05:14:30.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.141+0000 7f1516ffd700 1 -- 172.21.15.186:0/974036539 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f151c003ac0 con 0x7f1520100310 2024-01-31T05:14:30.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.141+0000 7f152817c700 1 -- 172.21.15.186:0/974036539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1520062e10 con 0x7f1520100310 2024-01-31T05:14:30.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.142+0000 7f1516ffd700 1 --2- 172.21.15.186:0/974036539 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f150c038b40 0x7f150c03b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:30.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.142+0000 7f1516ffd700 1 -- 172.21.15.186:0/974036539 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f151c04c810 con 0x7f1520100310 2024-01-31T05:14:30.143 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.142+0000 7f1525717700 1 --2- 172.21.15.186:0/974036539 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f150c038b40 0x7f150c03b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:30.143 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.143+0000 7f1525717700 1 --2- 172.21.15.186:0/974036539 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f150c038b40 0x7f150c03b000 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1510006fd0 tx=0x7f1510006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:30.146 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.147+0000 7f1516ffd700 1 -- 172.21.15.186:0/974036539 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f151c029770 con 0x7f1520100310 2024-01-31T05:14:30.422 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.423+0000 7f152817c700 1 -- 172.21.15.186:0/974036539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f152010a400 con 0x7f1520100310 2024-01-31T05:14:30.423 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.424+0000 7f1516ffd700 1 -- 172.21.15.186:0/974036539 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f151c026070 con 0x7f1520100310 2024-01-31T05:14:30.423 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:14:30.423 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","modified":"2024-01-31T05:12:48.541203Z","created":"2024-01-31T05:12:48.541203Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-01-31T05:14:30.425 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.426+0000 7f152817c700 1 -- 172.21.15.186:0/974036539 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f150c038b40 msgr2=0x7f150c03b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:30.425 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.426+0000 7f152817c700 1 --2- 172.21.15.186:0/974036539 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f150c038b40 0x7f150c03b000 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1510006fd0 tx=0x7f1510006e40 comp rx=0 tx=0).stop 2024-01-31T05:14:30.425 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.426+0000 7f152817c700 1 -- 172.21.15.186:0/974036539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1520100310 msgr2=0x7f1520101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:30.425 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.426+0000 7f152817c700 1 --2- 172.21.15.186:0/974036539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1520100310 0x7f1520101db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f151c004f40 tx=0x7f151c005020 comp rx=0 tx=0).stop 2024-01-31T05:14:30.425 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.426+0000 7f152817c700 1 -- 172.21.15.186:0/974036539 shutdown_connections 2024-01-31T05:14:30.426 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.426+0000 7f152817c700 1 --2- 172.21.15.186:0/974036539 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f150c038b40 0x7f150c03b000 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:30.426 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.426+0000 7f152817c700 1 --2- 172.21.15.186:0/974036539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1520100310 0x7f1520101db0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:30.426 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.426+0000 7f152817c700 1 -- 172.21.15.186:0/974036539 >> 172.21.15.186:0/974036539 conn(0x7f1520076d20 msgr2=0x7f1520068bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:30.426 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.426+0000 7f152817c700 1 -- 172.21.15.186:0/974036539 shutdown_connections 2024-01-31T05:14:30.426 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:30.426+0000 7f152817c700 1 -- 172.21.15.186:0/974036539 wait complete. 2024-01-31T05:14:30.426 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-01-31T05:14:32.157 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-01-31T05:14:32.158 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph mon dump -f json 2024-01-31T05:14:32.404 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.c/config 2024-01-31T05:14:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:14:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: mon.a calling monitor election 2024-01-31T05:14:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: mon.c calling monitor election 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: monmap e2: 2 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: fsmap 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: osdmap e4: 0 total, 0 up, 0 in 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: mgrmap e13: x(active, since 49s) 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: overall HEALTH_OK 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:36.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:36.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:36 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:36.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:36 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:36.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:36 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:36.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:36 smithi045 ceph-mon[31585]: Deploying daemon mon.b on smithi156 2024-01-31T05:14:36.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:36 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:37.082 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.083+0000 7f445e19f700 1 -- 172.21.15.186:0/3579122588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44400035c0 msgr2=0x7f4440005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:37.083 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.083+0000 7f445e19f700 1 --2- 172.21.15.186:0/3579122588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44400035c0 0x7f4440005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4458100a80 tx=0x7f444c00fe60 comp rx=0 tx=0).stop 2024-01-31T05:14:37.083 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.083+0000 7f445e19f700 1 -- 172.21.15.186:0/3579122588 shutdown_connections 2024-01-31T05:14:37.083 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.083+0000 7f445e19f700 1 --2- 172.21.15.186:0/3579122588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44400035c0 0x7f4440005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:37.083 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.083+0000 7f445e19f700 1 -- 172.21.15.186:0/3579122588 >> 172.21.15.186:0/3579122588 conn(0x7f44580767d0 msgr2=0x7f4458076be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:37.083 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.083+0000 7f445e19f700 1 -- 172.21.15.186:0/3579122588 shutdown_connections 2024-01-31T05:14:37.083 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.083+0000 7f445e19f700 1 -- 172.21.15.186:0/3579122588 wait complete. 2024-01-31T05:14:37.083 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:36 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:37.083 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:36 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:37.083 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:36 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:37.083 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:36 smithi186 ceph-mon[35659]: Deploying daemon mon.b on smithi156 2024-01-31T05:14:37.084 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:36 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:37.086 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f445e19f700 1 Processor -- start 2024-01-31T05:14:37.086 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f445e19f700 1 -- start start 2024-01-31T05:14:37.086 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f445e19f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44400035c0 0x7f445819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:37.086 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f445e19f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f445819cd70 0x7f44581969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:37.086 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f445e19f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445819d300 con 0x7f445819cd70 2024-01-31T05:14:37.086 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f445e19f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445819d470 con 0x7f44400035c0 2024-01-31T05:14:37.086 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f44577fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44400035c0 0x7f445819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:37.086 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f4456ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f445819cd70 0x7f44581969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f44577fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44400035c0 0x7f445819c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:41032/0 (socket says 172.21.15.186:41032) 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f4456ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f445819cd70 0x7f44581969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.186:36084/0 (socket says 172.21.15.186:36084) 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f44577fe700 1 -- 172.21.15.186:0/3948970882 learned_addr learned my addr 172.21.15.186:0/3948970882 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f44577fe700 1 -- 172.21.15.186:0/3948970882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44400035c0 msgr2=0x7f445819c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f44577fe700 1 -- 172.21.15.186:0/3948970882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44400035c0 msgr2=0x7f445819c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f44577fe700 1 --2- 172.21.15.186:0/3948970882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44400035c0 0x7f445819c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.084+0000 7f44577fe700 1 --2- 172.21.15.186:0/3948970882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44400035c0 0x7f445819c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.085+0000 7f4456ffd700 1 -- 172.21.15.186:0/3948970882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44400035c0 msgr2=0x7f445819c830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.085+0000 7f4456ffd700 1 --2- 172.21.15.186:0/3948970882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44400035c0 0x7f445819c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.085+0000 7f4456ffd700 1 -- 172.21.15.186:0/3948970882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c00f8b0 con 0x7f445819cd70 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.085+0000 7f4456ffd700 1 --2- 172.21.15.186:0/3948970882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f445819cd70 0x7f44581969f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f444800b4d0 tx=0x7f444800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.085+0000 7f4454ff9700 1 -- 172.21.15.186:0/3948970882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4448011630 con 0x7f445819cd70 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.085+0000 7f4454ff9700 1 -- 172.21.15.186:0/3948970882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4448011c50 con 0x7f445819cd70 2024-01-31T05:14:37.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.085+0000 7f445e19f700 1 -- 172.21.15.186:0/3948970882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4458196ff0 con 0x7f445819cd70 2024-01-31T05:14:37.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.085+0000 7f4454ff9700 1 -- 172.21.15.186:0/3948970882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4448010d90 con 0x7f445819cd70 2024-01-31T05:14:37.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.085+0000 7f445e19f700 1 -- 172.21.15.186:0/3948970882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4458197540 con 0x7f445819cd70 2024-01-31T05:14:37.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.086+0000 7f4454ff9700 1 -- 172.21.15.186:0/3948970882 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f444801a400 con 0x7f445819cd70 2024-01-31T05:14:37.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.086+0000 7f445e19f700 1 -- 172.21.15.186:0/3948970882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44581909d0 con 0x7f445819cd70 2024-01-31T05:14:37.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.086+0000 7f4454ff9700 1 --2- 172.21.15.186:0/3948970882 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f44440388a0 0x7f444403ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:37.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.086+0000 7f4454ff9700 1 -- 172.21.15.186:0/3948970882 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f444804b220 con 0x7f445819cd70 2024-01-31T05:14:37.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.086+0000 7f44577fe700 1 --2- 172.21.15.186:0/3948970882 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f44440388a0 0x7f444403ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:37.088 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.087+0000 7f44577fe700 1 --2- 172.21.15.186:0/3948970882 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f44440388a0 0x7f444403ad60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f444c001eb0 tx=0x7f444c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:37.090 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.091+0000 7f4454ff9700 1 -- 172.21.15.186:0/3948970882 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f44480176d0 con 0x7f445819cd70 2024-01-31T05:14:37.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.380+0000 7f445e19f700 1 -- 172.21.15.186:0/3948970882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4458197f20 con 0x7f445819cd70 2024-01-31T05:14:37.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.380+0000 7f4454ff9700 1 -- 172.21.15.186:0/3948970882 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f44480172c0 con 0x7f445819cd70 2024-01-31T05:14:37.380 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:14:37.380 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","modified":"2024-01-31T05:14:30.516290Z","created":"2024-01-31T05:12:48.541203Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-01-31T05:14:37.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.382+0000 7f445e19f700 1 -- 172.21.15.186:0/3948970882 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f44440388a0 msgr2=0x7f444403ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:37.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.382+0000 7f445e19f700 1 --2- 172.21.15.186:0/3948970882 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f44440388a0 0x7f444403ad60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f444c001eb0 tx=0x7f444c005ab0 comp rx=0 tx=0).stop 2024-01-31T05:14:37.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.382+0000 7f445e19f700 1 -- 172.21.15.186:0/3948970882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f445819cd70 msgr2=0x7f44581969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:37.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.382+0000 7f445e19f700 1 --2- 172.21.15.186:0/3948970882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f445819cd70 0x7f44581969f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f444800b4d0 tx=0x7f444800b890 comp rx=0 tx=0).stop 2024-01-31T05:14:37.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.382+0000 7f445e19f700 1 -- 172.21.15.186:0/3948970882 shutdown_connections 2024-01-31T05:14:37.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.382+0000 7f445e19f700 1 --2- 172.21.15.186:0/3948970882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44400035c0 0x7f445819c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:37.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.382+0000 7f445e19f700 1 --2- 172.21.15.186:0/3948970882 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f44440388a0 0x7f444403ad60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:37.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.382+0000 7f445e19f700 1 --2- 172.21.15.186:0/3948970882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f445819cd70 0x7f44581969f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:37.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.382+0000 7f445e19f700 1 -- 172.21.15.186:0/3948970882 >> 172.21.15.186:0/3948970882 conn(0x7f44580767d0 msgr2=0x7f44580fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:37.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.383+0000 7f445e19f700 1 -- 172.21.15.186:0/3948970882 shutdown_connections 2024-01-31T05:14:37.383 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:37.383+0000 7f445e19f700 1 -- 172.21.15.186:0/3948970882 wait complete. 2024-01-31T05:14:37.383 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2024-01-31T05:14:37.731 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:37 smithi186 ceph-mon[35659]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:37.731 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:37 smithi186 ceph-mon[35659]: from='client.? 172.21.15.186:0/3948970882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-31T05:14:37.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:37 smithi045 ceph-mon[31585]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:37.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:37 smithi045 ceph-mon[31585]: from='client.? 172.21.15.186:0/3948970882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-31T05:14:39.017 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-01-31T05:14:39.018 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph mon dump -f json 2024-01-31T05:14:39.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 podman[35702]: 2024-01-31 05:14:38.98370433 +0000 UTC m=+0.017894067 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:14:39.260 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.c/config 2024-01-31T05:14:39.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 podman[35702]: 2024-01-31 05:14:39.413792597 +0000 UTC m=+0.447982333 container init 5ab2d0d9b7697d4027e447fb39e95c3764485d773a059e459a044a5299bc2d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux ) 2024-01-31T05:14:39.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 podman[35702]: 2024-01-31 05:14:39.416704847 +0000 UTC m=+0.450894584 container start 5ab2d0d9b7697d4027e447fb39e95c3764485d773a059e459a044a5299bc2d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-b, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:14:39.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: set uid:gid to 167:167 (ceph:ceph) 2024-01-31T05:14:39.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev), process ceph-mon, pid 2 2024-01-31T05:14:39.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pidfile_write: ignore empty --pid-file 2024-01-31T05:14:39.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: load: jerasure load: lrc 2024-01-31T05:14:39.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: RocksDB version: 7.9.2 2024-01-31T05:14:39.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Git sha 0 2024-01-31T05:14:39.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Compile date 2024-01-25 05:03:27 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: DB SUMMARY 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: DB Session ID: B09HIMDPB9HFOQY758FP 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: CURRENT file: CURRENT 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: IDENTITY file: IDENTITY 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.error_if_exists: 0 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.create_if_missing: 0 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.paranoid_checks: 1 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.flush_verify_memtable_count: 1 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-01-31T05:14:39.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.env: 0x561272373ae0 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.fs: PosixFileSystem 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.info_log: 0x56127419b780 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_file_opening_threads: 16 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.statistics: (nil) 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.use_fsync: 0 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_log_file_size: 0 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.log_file_time_to_roll: 0 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.keep_log_file_num: 1000 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.recycle_log_file_num: 0 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.allow_fallocate: 1 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.allow_mmap_reads: 0 2024-01-31T05:14:39.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.allow_mmap_writes: 0 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.use_direct_reads: 0 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.create_missing_column_families: 0 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.db_log_dir: 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.wal_dir: 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.table_cache_numshardbits: 6 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.WAL_ttl_seconds: 0 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.WAL_size_limit_MB: 0 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.is_fd_close_on_exec: 1 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.advise_random_on_open: 1 2024-01-31T05:14:39.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.db_write_buffer_size: 0 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.write_buffer_manager: 0x56127416f400 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.use_adaptive_mutex: 0 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.rate_limiter: (nil) 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.wal_recovery_mode: 2 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.enable_thread_tracking: 0 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.enable_pipelined_write: 0 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.unordered_write: 0 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-01-31T05:14:39.731 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.row_cache: None 2024-01-31T05:14:39.732 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.wal_filter: None 2024-01-31T05:14:39.733 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-01-31T05:14:39.733 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.allow_ingest_behind: 0 2024-01-31T05:14:39.733 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.two_write_queues: 0 2024-01-31T05:14:39.733 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.manual_wal_flush: 0 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.wal_compression: 0 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.atomic_flush: 0 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.persist_stats_to_disk: 0 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.write_dbid_to_manifest: 0 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.log_readahead_size: 0 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.best_efforts_recovery: 0 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.allow_data_in_errors: 0 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.db_host_id: __hostname__ 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.enforce_single_del_contracts: true 2024-01-31T05:14:39.734 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_background_jobs: 2 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_background_compactions: -1 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_subcompactions: 1 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.delayed_write_rate : 16777216 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_total_wal_size: 0 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.stats_dump_period_sec: 600 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.stats_persist_period_sec: 600 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_open_files: -1 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bytes_per_sync: 0 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.wal_bytes_per_sync: 0 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.strict_bytes_per_sync: 0 2024-01-31T05:14:39.735 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_readahead_size: 0 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_background_flushes: -1 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Compression algorithms supported: 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: kZSTD supported: 0 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: kXpressCompression supported: 0 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: kLZ4HCCompression supported: 1 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: kZlibCompression supported: 1 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: kSnappyCompression supported: 1 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: kLZ4Compression supported: 1 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: kBZip2Compression supported: 0 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Fast CRC32 supported: Supported on x86 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: DMutex implementation: pthread_mutex_t 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-01-31T05:14:39.736 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.merge_operator: 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_filter: None 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_filter_factory: None 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.sst_partitioner_factory: None 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.memtable_factory: SkipListFactory 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.table_factory: BlockBasedTable 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56127419b8a0) 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout: cache_index_and_filter_blocks: 1 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout: pin_top_level_index_and_filter: 1 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout: index_type: 0 2024-01-31T05:14:39.737 INFO:journalctl@ceph.mon.b.smithi156.stdout: data_block_index_type: 0 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: index_shortening: 1 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: data_block_hash_table_util_ratio: 0.750000 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: checksum: 4 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: no_block_cache: 0 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_cache: 0x56127419f090 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_cache_name: BinnedLRUCache 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_cache_options: 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: capacity : 536870912 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: num_shard_bits : 4 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: strict_capacity_limit : 0 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: high_pri_pool_ratio: 0.000 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_cache_compressed: (nil) 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: persistent_cache: (nil) 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_size: 4096 2024-01-31T05:14:39.738 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_size_deviation: 10 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_restart_interval: 16 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: index_block_restart_interval: 1 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: metadata_block_size: 4096 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: partition_filters: 0 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: use_delta_encoding: 1 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: filter_policy: bloomfilter 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: whole_key_filtering: 1 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: verify_compression: 0 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: read_amp_bytes_per_bit: 0 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: format_version: 5 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: enable_index_compression: 1 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_align: 0 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: max_auto_readahead_size: 262144 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: prepopulate_block_cache: 0 2024-01-31T05:14:39.739 INFO:journalctl@ceph.mon.b.smithi156.stdout: initial_auto_readahead_size: 8192 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout: num_file_reads_for_auto_readahead: 2 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.write_buffer_size: 33554432 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_write_buffer_number: 2 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compression: NoCompression 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bottommost_compression: Disabled 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.prefix_extractor: nullptr 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.num_levels: 7 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-01-31T05:14:39.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compression_opts.window_bits: -14 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compression_opts.level: 32767 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compression_opts.strategy: 0 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-01-31T05:14:39.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compression_opts.enabled: false 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.target_file_size_base: 67108864 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.target_file_size_multiplier: 1 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-01-31T05:14:39.742 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.arena_block_size: 1048576 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.disable_auto_compactions: 0 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-01-31T05:14:39.743 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.table_properties_collectors: 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.inplace_update_support: 0 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.inplace_update_num_locks: 10000 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.memtable_huge_page_size: 0 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.bloom_locality: 0 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.max_successive_merges: 0 2024-01-31T05:14:39.744 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.optimize_filters_for_hits: 0 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.paranoid_file_checks: 0 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.force_consistency_checks: 1 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.report_bg_io_stats: 0 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.ttl: 2592000 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.periodic_compaction_seconds: 0 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.enable_blob_files: false 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.min_blob_size: 0 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.blob_file_size: 268435456 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.blob_compression_type: NoCompression 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.enable_blob_garbage_collection: false 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-01-31T05:14:39.745 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.blob_file_starting_level: 0 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2ad65f31-c392-45ff-b5b3-7142573b5974 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706678079446279, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706678079446711, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1706678079, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2ad65f31-c392-45ff-b5b3-7142573b5974", "db_session_id": "B09HIMDPB9HFOQY758FP", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706678079446800, "job": 1, "event": "recovery_finished"} 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56127428c000 2024-01-31T05:14:39.746 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: DB pointer 0x56127427a000 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout: ** DB Stats ** 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout: 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout: ** Compaction Stats [default] ** 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-01-31T05:14:39.747 INFO:journalctl@ceph.mon.b.smithi156.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: ** Compaction Stats [default] ** 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-01-31T05:14:39.748 INFO:journalctl@ceph.mon.b.smithi156.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout: AddFile(Keys): cumulative 0, interval 0 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout: Block cache BinnedLRUCache@0x56127419f090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout: 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout: ** File Read Latency Histogram By Level [default] ** 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-01-31T05:14:39.749 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: using public_addr v2:172.21.15.156:0/0 -> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: starting mon.b rank -1 at public addrs [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] at bind addrs [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(???) e0 preinit fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).mds e1 new map 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).mds e1 print_map 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout: e1 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout: legacy client fscid: -1 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout: 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout: No filesystems configured 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-01-31T05:14:39.750 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Deploying cephadm binary to smithi186 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:14:39.751 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Updating smithi156:/etc/ceph/ceph.conf 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Added host smithi186 2024-01-31T05:14:39.752 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3931195139' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Updating smithi186:/etc/ceph/ceph.conf 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3931195139' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: osdmap e4: 0 total, 0 up, 0 in 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi156:172.21.15.156=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi156:172.21.15.156=b;smithi186:172.21.15.186=c;count:3 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Deploying daemon mon.c on smithi186 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.a calling monitor election 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:39.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.c calling monitor election 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: monmap e2: 2 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: fsmap 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: osdmap e4: 0 total, 0 up, 0 in 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mgrmap e13: x(active, since 49s) 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: overall HEALTH_OK 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:39.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:39.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:39.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: Deploying daemon mon.b on smithi156 2024-01-31T05:14:39.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:39.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:39.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: from='client.? 172.21.15.186:0/3948970882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-31T05:14:39.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 ceph-mon[35720]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-01-31T05:14:39.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 bash[35702]: 5ab2d0d9b7697d4027e447fb39e95c3764485d773a059e459a044a5299bc2d3f 2024-01-31T05:14:39.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:39 smithi156 systemd[1]: Started Ceph mon.b for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:14:44.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:14:44.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: mon.a calling monitor election 2024-01-31T05:14:44.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: mon.c calling monitor election 2024-01-31T05:14:44.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:44.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: mon.b calling monitor election 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: fsmap 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: osdmap e4: 0 total, 0 up, 0 in 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: mgrmap e13: x(active, since 58s) 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: overall HEALTH_OK 2024-01-31T05:14:44.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:44.844 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:44.844 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:14:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: mon.a calling monitor election 2024-01-31T05:14:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: mon.c calling monitor election 2024-01-31T05:14:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: mon.b calling monitor election 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: fsmap 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: osdmap e4: 0 total, 0 up, 0 in 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: mgrmap e13: x(active, since 58s) 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: overall HEALTH_OK 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:44.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:44.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:14:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: mon.a calling monitor election 2024-01-31T05:14:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: mon.c calling monitor election 2024-01-31T05:14:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: mon.b calling monitor election 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: fsmap 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: osdmap e4: 0 total, 0 up, 0 in 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: mgrmap e13: x(active, since 58s) 2024-01-31T05:14:44.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: overall HEALTH_OK 2024-01-31T05:14:44.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:44.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:44.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:45.156 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.154+0000 7f0615e66700 1 -- 172.21.15.186:0/2456578638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0035c0 msgr2=0x7f05fc005a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:45.156 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.154+0000 7f0615e66700 1 --2- 172.21.15.186:0/2456578638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0035c0 0x7f05fc005a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f06101011f0 tx=0x7f060000fe60 comp rx=0 tx=0).stop 2024-01-31T05:14:45.156 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.154+0000 7f0615e66700 1 -- 172.21.15.186:0/2456578638 shutdown_connections 2024-01-31T05:14:45.156 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.154+0000 7f0615e66700 1 --2- 172.21.15.186:0/2456578638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0035c0 0x7f05fc005a30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:45.156 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.154+0000 7f0615e66700 1 -- 172.21.15.186:0/2456578638 >> 172.21.15.186:0/2456578638 conn(0x7f06100fcdb0 msgr2=0x7f06100ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:45.156 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.154+0000 7f0615e66700 1 -- 172.21.15.186:0/2456578638 shutdown_connections 2024-01-31T05:14:45.156 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.154+0000 7f0615e66700 1 -- 172.21.15.186:0/2456578638 wait complete. 2024-01-31T05:14:45.156 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0615e66700 1 Processor -- start 2024-01-31T05:14:45.156 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0615e66700 1 -- start start 2024-01-31T05:14:45.156 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0615e66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0035c0 0x7f061019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:45.156 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0615e66700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f061019ce70 0x7f0610196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:45.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0615e66700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0610197030 0x7f06101974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:45.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0615e66700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061019d530 con 0x7f05fc0035c0 2024-01-31T05:14:45.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0615e66700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06101a3800 con 0x7f0610197030 2024-01-31T05:14:45.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0615e66700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06101a3940 con 0x7f061019ce70 2024-01-31T05:14:45.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0606ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f061019ce70 0x7f0610196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:45.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0606ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f061019ce70 0x7f0610196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:55542/0 (socket says 172.21.15.186:55542) 2024-01-31T05:14:45.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0606ffd700 1 -- 172.21.15.186:0/1995755478 learned_addr learned my addr 172.21.15.186:0/1995755478 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-01-31T05:14:45.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0606ffd700 1 -- 172.21.15.186:0/1995755478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f061019ce70 msgr2=0x7f0610196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-31T05:14:45.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0606ffd700 1 -- 172.21.15.186:0/1995755478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f061019ce70 msgr2=0x7f0610196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-31T05:14:45.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0606ffd700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f061019ce70 0x7f0610196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-31T05:14:45.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f060ffff700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0610197030 0x7f06101974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:45.157 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f0606ffd700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f061019ce70 0x7f0610196af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f060f7fe700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0035c0 0x7f061019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f060ffff700 1 -- 172.21.15.186:0/1995755478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0610197030 msgr2=0x7f06101974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f060ffff700 1 -- 172.21.15.186:0/1995755478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0610197030 msgr2=0x7f06101974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f060ffff700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0610197030 0x7f06101974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.155+0000 7f060ffff700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0610197030 0x7f06101974e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.156+0000 7f060f7fe700 1 -- 172.21.15.186:0/1995755478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f061019ce70 msgr2=0x7f0610196af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.156+0000 7f060f7fe700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f061019ce70 0x7f0610196af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.156+0000 7f060f7fe700 1 -- 172.21.15.186:0/1995755478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0610197030 msgr2=0x7f06101974e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.156+0000 7f060f7fe700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0610197030 0x7f06101974e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.156+0000 7f060f7fe700 1 -- 172.21.15.186:0/1995755478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060000f8b0 con 0x7f05fc0035c0 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.156+0000 7f060f7fe700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0035c0 0x7f061019c930 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f06101011f0 tx=0x7f0600012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.156+0000 7f060d7fa700 1 -- 172.21.15.186:0/1995755478 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600003fb0 con 0x7f05fc0035c0 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.156+0000 7f060d7fa700 1 -- 172.21.15.186:0/1995755478 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0600004110 con 0x7f05fc0035c0 2024-01-31T05:14:45.158 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.156+0000 7f0615e66700 1 -- 172.21.15.186:0/1995755478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06101a3bc0 con 0x7f05fc0035c0 2024-01-31T05:14:45.159 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.156+0000 7f060d7fa700 1 -- 172.21.15.186:0/1995755478 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600027650 con 0x7f05fc0035c0 2024-01-31T05:14:45.159 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.156+0000 7f0615e66700 1 -- 172.21.15.186:0/1995755478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06101a40c0 con 0x7f05fc0035c0 2024-01-31T05:14:45.159 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.157+0000 7f0615e66700 1 -- 172.21.15.186:0/1995755478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0610190930 con 0x7f05fc0035c0 2024-01-31T05:14:45.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.157+0000 7f060d7fa700 1 -- 172.21.15.186:0/1995755478 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f0600024050 con 0x7f05fc0035c0 2024-01-31T05:14:45.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.158+0000 7f060d7fa700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f05f0038f70 0x7f05f003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:45.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.158+0000 7f060d7fa700 1 -- 172.21.15.186:0/1995755478 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f060004e290 con 0x7f05fc0035c0 2024-01-31T05:14:45.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.158+0000 7f0606ffd700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f05f0038f70 0x7f05f003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:45.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.158+0000 7f0606ffd700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f05f0038f70 0x7f05f003b430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009180 tx=0x7f05f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:45.162 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.163+0000 7f060d7fa700 1 -- 172.21.15.186:0/1995755478 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f060002f7c0 con 0x7f05fc0035c0 2024-01-31T05:14:45.454 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.454+0000 7f0615e66700 1 -- 172.21.15.186:0/1995755478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0610109010 con 0x7f05fc0035c0 2024-01-31T05:14:45.454 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.454+0000 7f060d7fa700 1 -- 172.21.15.186:0/1995755478 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1285 (secure 0 0 0) 0x7f060002b070 con 0x7f05fc0035c0 2024-01-31T05:14:45.454 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-01-31T05:14:45.454 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":3,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","modified":"2024-01-31T05:14:39.468498Z","created":"2024-01-31T05:12:48.541203Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-01-31T05:14:45.456 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.456+0000 7f0615e66700 1 -- 172.21.15.186:0/1995755478 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f05f0038f70 msgr2=0x7f05f003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:45.456 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.456+0000 7f0615e66700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f05f0038f70 0x7f05f003b430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009180 tx=0x7f05f8008040 comp rx=0 tx=0).stop 2024-01-31T05:14:45.456 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.456+0000 7f0615e66700 1 -- 172.21.15.186:0/1995755478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0035c0 msgr2=0x7f061019c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:45.456 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.456+0000 7f0615e66700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0035c0 0x7f061019c930 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f06101011f0 tx=0x7f0600012d70 comp rx=0 tx=0).stop 2024-01-31T05:14:45.456 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.457+0000 7f0615e66700 1 -- 172.21.15.186:0/1995755478 shutdown_connections 2024-01-31T05:14:45.456 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.457+0000 7f0615e66700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0610197030 0x7f06101974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:45.456 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.457+0000 7f0615e66700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f05f0038f70 0x7f05f003b430 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:45.457 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.457+0000 7f0615e66700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0035c0 0x7f061019c930 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:45.457 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.457+0000 7f0615e66700 1 --2- 172.21.15.186:0/1995755478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f061019ce70 0x7f0610196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:45.457 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.457+0000 7f0615e66700 1 -- 172.21.15.186:0/1995755478 >> 172.21.15.186:0/1995755478 conn(0x7f06100fcdb0 msgr2=0x7f06100fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:45.457 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.457+0000 7f0615e66700 1 -- 172.21.15.186:0/1995755478 shutdown_connections 2024-01-31T05:14:45.457 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:45.457+0000 7f0615e66700 1 -- 172.21.15.186:0/1995755478 wait complete. 2024-01-31T05:14:45.457 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 3 2024-01-31T05:14:45.827 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:45 smithi186 ceph-mon[35659]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:45.827 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:45.827 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:45.828 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:45 smithi186 ceph-mon[35659]: from='client.? 172.21.15.186:0/1995755478' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-31T05:14:45.828 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:45.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:45 smithi045 ceph-mon[31585]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:45.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:45.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:45.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:45 smithi045 ceph-mon[31585]: from='client.? 172.21.15.186:0/1995755478' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-31T05:14:45.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:45 smithi156 ceph-mon[35720]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:45 smithi156 ceph-mon[35720]: from='client.? 172.21.15.186:0/1995755478' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-31T05:14:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:14:46.204 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-01-31T05:14:46.204 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph config generate-minimal-conf 2024-01-31T05:14:46.453 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:14:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:47 smithi045 ceph-mon[31585]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:47 smithi045 ceph-mon[31585]: Updating smithi045:/etc/ceph/ceph.conf 2024-01-31T05:14:47.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:47 smithi045 ceph-mon[31585]: Updating smithi156:/etc/ceph/ceph.conf 2024-01-31T05:14:47.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:47 smithi045 ceph-mon[31585]: Updating smithi186:/etc/ceph/ceph.conf 2024-01-31T05:14:47.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:47 smithi186 ceph-mon[35659]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:47.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:47.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:47.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:47 smithi186 ceph-mon[35659]: Updating smithi045:/etc/ceph/ceph.conf 2024-01-31T05:14:47.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:47 smithi186 ceph-mon[35659]: Updating smithi156:/etc/ceph/ceph.conf 2024-01-31T05:14:47.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:47 smithi186 ceph-mon[35659]: Updating smithi186:/etc/ceph/ceph.conf 2024-01-31T05:14:47.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:47 smithi156 ceph-mon[35720]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:47.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:47.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:47.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:47.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:47 smithi156 ceph-mon[35720]: Updating smithi045:/etc/ceph/ceph.conf 2024-01-31T05:14:47.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:47 smithi156 ceph-mon[35720]: Updating smithi156:/etc/ceph/ceph.conf 2024-01-31T05:14:47.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:47 smithi156 ceph-mon[35720]: Updating smithi186:/etc/ceph/ceph.conf 2024-01-31T05:14:48.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.048+0000 7effd2c59700 1 -- 172.21.15.45:0/2253978559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effcc101430 msgr2=0x7effcc101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.048+0000 7effd2c59700 1 --2- 172.21.15.45:0/2253978559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effcc101430 0x7effcc101810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7effc0009910 tx=0x7effc0009c20 comp rx=0 tx=0).stop 2024-01-31T05:14:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.048+0000 7effd2c59700 1 -- 172.21.15.45:0/2253978559 shutdown_connections 2024-01-31T05:14:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.048+0000 7effd2c59700 1 --2- 172.21.15.45:0/2253978559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effcc101430 0x7effcc101810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.048+0000 7effd2c59700 1 -- 172.21.15.45:0/2253978559 >> 172.21.15.45:0/2253978559 conn(0x7effcc0fcce0 msgr2=0x7effcc0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.048+0000 7effd2c59700 1 -- 172.21.15.45:0/2253978559 shutdown_connections 2024-01-31T05:14:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.048+0000 7effd2c59700 1 -- 172.21.15.45:0/2253978559 wait complete. 2024-01-31T05:14:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effd2c59700 1 Processor -- start 2024-01-31T05:14:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effd2c59700 1 -- start start 2024-01-31T05:14:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effd2c59700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effcc101430 0x7effcc19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effd2c59700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effcc19ce10 0x7effcc196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effd2c59700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7effcc196fd0 0x7effcc197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effd2c59700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc19d4d0 con 0x7effcc19ce10 2024-01-31T05:14:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effd2c59700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc1a37a0 con 0x7effcc101430 2024-01-31T05:14:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effd2c59700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc1a38e0 con 0x7effcc196fd0 2024-01-31T05:14:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effcbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effcc19ce10 0x7effcc196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effcbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effcc19ce10 0x7effcc196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34968/0 (socket says 172.21.15.45:34968) 2024-01-31T05:14:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effcbfff700 1 -- 172.21.15.45:0/4256813323 learned_addr learned my addr 172.21.15.45:0/4256813323 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:14:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effd09f5700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effcc101430 0x7effcc19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.049+0000 7effd11f6700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7effcc196fd0 0x7effcc197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.050+0000 7effcbfff700 1 -- 172.21.15.45:0/4256813323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7effcc196fd0 msgr2=0x7effcc197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.050+0000 7effcbfff700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7effcc196fd0 0x7effcc197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.050+0000 7effcbfff700 1 -- 172.21.15.45:0/4256813323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effcc101430 msgr2=0x7effcc19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.050+0000 7effcbfff700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effcc101430 0x7effcc19c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.050+0000 7effcbfff700 1 -- 172.21.15.45:0/4256813323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc00095f0 con 0x7effcc19ce10 2024-01-31T05:14:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.050+0000 7effd11f6700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7effcc196fd0 0x7effcc197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:14:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.050+0000 7effcbfff700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effcc19ce10 0x7effcc196a90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7effb800b780 tx=0x7effb800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.050+0000 7effc9ffb700 1 -- 172.21.15.45:0/4256813323 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8004230 con 0x7effcc19ce10 2024-01-31T05:14:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.050+0000 7effc9ffb700 1 -- 172.21.15.45:0/4256813323 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effb8004390 con 0x7effcc19ce10 2024-01-31T05:14:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.050+0000 7effc9ffb700 1 -- 172.21.15.45:0/4256813323 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8011ad0 con 0x7effcc19ce10 2024-01-31T05:14:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.050+0000 7effd2c59700 1 -- 172.21.15.45:0/4256813323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effcc1a3bc0 con 0x7effcc19ce10 2024-01-31T05:14:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.050+0000 7effd2c59700 1 -- 172.21.15.45:0/4256813323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc1a4110 con 0x7effcc19ce10 2024-01-31T05:14:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.051+0000 7effc9ffb700 1 -- 172.21.15.45:0/4256813323 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7effb8011c30 con 0x7effcc19ce10 2024-01-31T05:14:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.051+0000 7effc9ffb700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7effbc038f70 0x7effbc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.051+0000 7effc9ffb700 1 -- 172.21.15.45:0/4256813323 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7effb804b160 con 0x7effcc19ce10 2024-01-31T05:14:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.051+0000 7effd2c59700 1 -- 172.21.15.45:0/4256813323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effcc062e10 con 0x7effcc19ce10 2024-01-31T05:14:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.051+0000 7effd09f5700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7effbc038f70 0x7effbc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.052+0000 7effd09f5700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7effbc038f70 0x7effbc03b430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7effc0006010 tx=0x7effc000b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.056+0000 7effc9ffb700 1 -- 172.21.15.45:0/4256813323 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7effb801a400 con 0x7effcc19ce10 2024-01-31T05:14:48.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.258+0000 7effd2c59700 1 -- 172.21.15.45:0/4256813323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7effcc10b4d0 con 0x7effcc19ce10 2024-01-31T05:14:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.258+0000 7effc9ffb700 1 -- 172.21.15.45:0/4256813323 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7effb8004570 con 0x7effcc19ce10 2024-01-31T05:14:48.258 INFO:teuthology.orchestra.run.smithi045.stdout:# minimal ceph.conf for 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:14:48.258 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-01-31T05:14:48.258 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:14:48.258 INFO:teuthology.orchestra.run.smithi045.stdout: mon_host = [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2024-01-31T05:14:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.260+0000 7effd2c59700 1 -- 172.21.15.45:0/4256813323 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7effbc038f70 msgr2=0x7effbc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.260+0000 7effd2c59700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7effbc038f70 0x7effbc03b430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7effc0006010 tx=0x7effc000b560 comp rx=0 tx=0).stop 2024-01-31T05:14:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.260+0000 7effd2c59700 1 -- 172.21.15.45:0/4256813323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effcc19ce10 msgr2=0x7effcc196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.260+0000 7effd2c59700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effcc19ce10 0x7effcc196a90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7effb800b780 tx=0x7effb800ba90 comp rx=0 tx=0).stop 2024-01-31T05:14:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.260+0000 7effd2c59700 1 -- 172.21.15.45:0/4256813323 shutdown_connections 2024-01-31T05:14:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.260+0000 7effd2c59700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7effcc196fd0 0x7effcc197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.260+0000 7effd2c59700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effcc101430 0x7effcc19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.260+0000 7effd2c59700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7effbc038f70 0x7effbc03b430 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.260+0000 7effd2c59700 1 --2- 172.21.15.45:0/4256813323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effcc19ce10 0x7effcc196a90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.260+0000 7effd2c59700 1 -- 172.21.15.45:0/4256813323 >> 172.21.15.45:0/4256813323 conn(0x7effcc0fcce0 msgr2=0x7effcc102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.260+0000 7effd2c59700 1 -- 172.21.15.45:0/4256813323 shutdown_connections 2024-01-31T05:14:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:48.261+0000 7effd2c59700 1 -- 172.21.15.45:0/4256813323 wait complete. 2024-01-31T05:14:48.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: Updating smithi045:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:48.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:48.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:48.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: Reconfiguring mon.a (unknown last config time)... 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:48.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:48 smithi156 ceph-mon[35720]: Reconfiguring daemon mon.a on smithi045 2024-01-31T05:14:48.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: Updating smithi045:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:48.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:48.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: Reconfiguring mon.a (unknown last config time)... 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:48.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-31T05:14:48.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:48.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:48 smithi045 ceph-mon[31585]: Reconfiguring daemon mon.a on smithi045 2024-01-31T05:14:48.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: Updating smithi045:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:48.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:48.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:14:48.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:48.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: Reconfiguring mon.a (unknown last config time)... 2024-01-31T05:14:48.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:48.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-31T05:14:48.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:48.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:48 smithi186 ceph-mon[35659]: Reconfiguring daemon mon.a on smithi045 2024-01-31T05:14:48.910 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-01-31T05:14:48.910 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:14:48.910 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2024-01-31T05:14:48.940 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:14:48.940 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:14:49.007 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-01-31T05:14:49.007 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.conf 2024-01-31T05:14:49.042 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-01-31T05:14:49.043 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:14:49.114 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-01-31T05:14:49.114 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.conf 2024-01-31T05:14:49.149 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-01-31T05:14:49.149 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:14:49.219 INFO:tasks.cephadm:Adding mgr.x on smithi045 2024-01-31T05:14:49.219 INFO:tasks.cephadm:Adding mgr.y on smithi156 2024-01-31T05:14:49.219 INFO:tasks.cephadm:Adding mgr.z on smithi186 2024-01-31T05:14:49.219 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch apply mgr '3;smithi045=x;smithi156=y;smithi186=z' 2024-01-31T05:14:49.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:49 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/4256813323' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:49.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:49 smithi156 ceph-mon[35720]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:49.490 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.c/config 2024-01-31T05:14:49.526 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:49 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/4256813323' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:49.526 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:49 smithi045 ceph-mon[31585]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:49.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:49 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/4256813323' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:49.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:49 smithi186 ceph-mon[35659]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:50.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.834+0000 7fe96dab9700 1 -- 172.21.15.186:0/4148308420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe968100250 msgr2=0x7fe968100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:50.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.834+0000 7fe96dab9700 1 --2- 172.21.15.186:0/4148308420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe968100250 0x7fe968100670 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe958009910 tx=0x7fe958009c20 comp rx=0 tx=0).stop 2024-01-31T05:14:50.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.834+0000 7fe96dab9700 1 -- 172.21.15.186:0/4148308420 shutdown_connections 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.834+0000 7fe96dab9700 1 --2- 172.21.15.186:0/4148308420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe968100250 0x7fe968100670 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.834+0000 7fe96dab9700 1 -- 172.21.15.186:0/4148308420 >> 172.21.15.186:0/4148308420 conn(0x7fe9680fb7f0 msgr2=0x7fe9680fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.834+0000 7fe96dab9700 1 -- 172.21.15.186:0/4148308420 shutdown_connections 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.834+0000 7fe96dab9700 1 -- 172.21.15.186:0/4148308420 wait complete. 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe96dab9700 1 Processor -- start 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe96dab9700 1 -- start start 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe96dab9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe968100250 0x7fe9681989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe96dab9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe968198f10 0x7fe96819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe96dab9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe968199390 0x7fe9681a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe96dab9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9681a0cb0 con 0x7fe968100250 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe96dab9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9681a0e20 con 0x7fe968198f10 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe96dab9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9681a0fe0 con 0x7fe968199390 2024-01-31T05:14:50.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe967fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe968199390 0x7fe9681a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe967fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe968199390 0x7fe9681a0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:33002/0 (socket says 172.21.15.186:33002) 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe967fff700 1 -- 172.21.15.186:0/4132594752 learned_addr learned my addr 172.21.15.186:0/4132594752 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe9677fe700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe968100250 0x7fe9681989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.835+0000 7fe966ffd700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe968198f10 0x7fe96819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.836+0000 7fe967fff700 1 -- 172.21.15.186:0/4132594752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe968198f10 msgr2=0x7fe96819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.836+0000 7fe967fff700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe968198f10 0x7fe96819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.836+0000 7fe967fff700 1 -- 172.21.15.186:0/4132594752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe968100250 msgr2=0x7fe9681989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.836+0000 7fe967fff700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe968100250 0x7fe9681989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.836+0000 7fe967fff700 1 -- 172.21.15.186:0/4132594752 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9580095f0 con 0x7fe968199390 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.836+0000 7fe967fff700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe968199390 0x7fe9681a0510 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe96000ba50 tx=0x7fe96000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.836+0000 7fe964ff9700 1 -- 172.21.15.186:0/4132594752 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96000c930 con 0x7fe968199390 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.836+0000 7fe964ff9700 1 -- 172.21.15.186:0/4132594752 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe960014440 con 0x7fe968199390 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.836+0000 7fe96dab9700 1 -- 172.21.15.186:0/4132594752 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9681a12c0 con 0x7fe968199390 2024-01-31T05:14:50.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.836+0000 7fe964ff9700 1 -- 172.21.15.186:0/4132594752 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600075b0 con 0x7fe968199390 2024-01-31T05:14:50.838 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.836+0000 7fe96dab9700 1 -- 172.21.15.186:0/4132594752 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9681a1840 con 0x7fe968199390 2024-01-31T05:14:50.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.837+0000 7fe96dab9700 1 -- 172.21.15.186:0/4132594752 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe968192a40 con 0x7fe968199390 2024-01-31T05:14:50.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.837+0000 7fe964ff9700 1 -- 172.21.15.186:0/4132594752 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fe960007710 con 0x7fe968199390 2024-01-31T05:14:50.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.837+0000 7fe964ff9700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe950038f70 0x7fe95003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:50.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.838+0000 7fe9677fe700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe950038f70 0x7fe95003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:50.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.838+0000 7fe9677fe700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe950038f70 0x7fe95003b430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe958009fd0 tx=0x7fe958005200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:50.839 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.839+0000 7fe964ff9700 1 -- 172.21.15.186:0/4132594752 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe9600148b0 con 0x7fe968199390 2024-01-31T05:14:50.843 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:50.844+0000 7fe964ff9700 1 -- 172.21.15.186:0/4132594752 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe96001b8b0 con 0x7fe968199390 2024-01-31T05:14:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:50 smithi156 ceph-mon[35720]: Reconfiguring mon.b (monmap changed)... 2024-01-31T05:14:50.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:50.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-31T05:14:50.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:50.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:50 smithi156 ceph-mon[35720]: Reconfiguring daemon mon.b on smithi156 2024-01-31T05:14:50.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:50 smithi156 ceph-mon[35720]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:51.043 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.043+0000 7fe96dab9700 1 -- 172.21.15.186:0/4132594752 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi156=y;smithi186=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fe96802d540 con 0x7fe950038f70 2024-01-31T05:14:51.043 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:51.043 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:51.043 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:50 smithi186 ceph-mon[35659]: Reconfiguring mon.b (monmap changed)... 2024-01-31T05:14:51.043 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:51.043 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-31T05:14:51.043 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:51.044 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:50 smithi186 ceph-mon[35659]: Reconfiguring daemon mon.b on smithi156 2024-01-31T05:14:51.044 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:50 smithi186 ceph-mon[35659]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:51.048 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.049+0000 7fe964ff9700 1 -- 172.21.15.186:0/4132594752 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe96802d540 con 0x7fe950038f70 2024-01-31T05:14:51.048 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mgr update... 2024-01-31T05:14:51.051 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.051+0000 7fe96dab9700 1 -- 172.21.15.186:0/4132594752 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe950038f70 msgr2=0x7fe95003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:51.051 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.051+0000 7fe96dab9700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe950038f70 0x7fe95003b430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe958009fd0 tx=0x7fe958005200 comp rx=0 tx=0).stop 2024-01-31T05:14:51.051 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.051+0000 7fe96dab9700 1 -- 172.21.15.186:0/4132594752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe968199390 msgr2=0x7fe9681a0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:51.051 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.051+0000 7fe96dab9700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe968199390 0x7fe9681a0510 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe96000ba50 tx=0x7fe96000bd60 comp rx=0 tx=0).stop 2024-01-31T05:14:51.051 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.051+0000 7fe96dab9700 1 -- 172.21.15.186:0/4132594752 shutdown_connections 2024-01-31T05:14:51.051 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.051+0000 7fe96dab9700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe968199390 0x7fe9681a0510 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:51.051 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.051+0000 7fe96dab9700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe950038f70 0x7fe95003b430 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:51.051 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.051+0000 7fe96dab9700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe968100250 0x7fe9681989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:51.051 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.051+0000 7fe96dab9700 1 --2- 172.21.15.186:0/4132594752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe968198f10 0x7fe96819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:51.052 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.051+0000 7fe96dab9700 1 -- 172.21.15.186:0/4132594752 >> 172.21.15.186:0/4132594752 conn(0x7fe9680fb7f0 msgr2=0x7fe96818fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:51.052 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.051+0000 7fe96dab9700 1 -- 172.21.15.186:0/4132594752 shutdown_connections 2024-01-31T05:14:51.052 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:14:51.052+0000 7fe96dab9700 1 -- 172.21.15.186:0/4132594752 wait complete. 2024-01-31T05:14:51.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:51.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:51.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:50 smithi045 ceph-mon[31585]: Reconfiguring mon.b (monmap changed)... 2024-01-31T05:14:51.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:51.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-31T05:14:51.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:51.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:50 smithi045 ceph-mon[31585]: Reconfiguring daemon mon.b on smithi156 2024-01-31T05:14:51.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:50 smithi045 ceph-mon[31585]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:51.587 DEBUG:teuthology.orchestra.run.smithi156:mgr.y> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.y.service 2024-01-31T05:14:51.590 DEBUG:teuthology.orchestra.run.smithi186:mgr.z> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.z.service 2024-01-31T05:14:51.592 INFO:tasks.cephadm:Deploying OSDs... 2024-01-31T05:14:51.592 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:14:51.592 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-01-31T05:14:51.611 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-31T05:14:51.611 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-01-31T05:14:51.669 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-31T05:14:51.669 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T05:14:51.669 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-01-31T05:14:51.669 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:14:51.669 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T05:14:51.669 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-01-31 05:13:58.519368872 +0000 2024-01-31T05:14:51.669 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-01-31 05:08:49.624913509 +0000 2024-01-31T05:14:51.669 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-01-31 05:08:49.624913509 +0000 2024-01-31T05:14:51.669 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-01-31 05:08:49.624913509 +0000 2024-01-31T05:14:51.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-31T05:14:51.739 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-01-31T05:14:51.739 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-01-31T05:14:51.739 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000123306 s, 4.2 MB/s 2024-01-31T05:14:51.740 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-31T05:14:51.798 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-01-31T05:14:51.856 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-31T05:14:51.856 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T05:14:51.856 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-01-31T05:14:51.856 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:14:51.856 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T05:14:51.856 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-01-31 05:13:58.560368241 +0000 2024-01-31T05:14:51.856 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-01-31 05:08:49.939908605 +0000 2024-01-31T05:14:51.856 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-01-31 05:08:49.939908605 +0000 2024-01-31T05:14:51.856 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-01-31 05:08:49.939908605 +0000 2024-01-31T05:14:51.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-31T05:14:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-01-31T05:14:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-01-31T05:14:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.3662e-05 s, 5.5 MB/s 2024-01-31T05:14:51.925 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-31T05:14:51.982 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-01-31T05:14:52.039 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-31T05:14:52.039 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T05:14:52.039 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-01-31T05:14:52.039 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:14:52.039 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T05:14:52.039 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-01-31 05:13:58.598367657 +0000 2024-01-31T05:14:52.039 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-01-31 05:08:50.260903607 +0000 2024-01-31T05:14:52.039 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-01-31 05:08:50.260903607 +0000 2024-01-31T05:14:52.039 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-01-31 05:08:50.260903607 +0000 2024-01-31T05:14:52.040 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-31T05:14:52.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-01-31T05:14:52.108 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-01-31T05:14:52.108 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.0001411 s, 3.6 MB/s 2024-01-31T05:14:52.108 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-31T05:14:52.164 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-01-31T05:14:52.221 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-31T05:14:52.221 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T05:14:52.221 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-01-31T05:14:52.222 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:14:52.222 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T05:14:52.222 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-01-31 05:13:58.632367134 +0000 2024-01-31T05:14:52.222 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-01-31 05:08:50.596898376 +0000 2024-01-31T05:14:52.222 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-01-31 05:08:50.596898376 +0000 2024-01-31T05:14:52.222 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-01-31 05:08:50.596898376 +0000 2024-01-31T05:14:52.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-31T05:14:52.287 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-01-31T05:14:52.288 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-01-31T05:14:52.288 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000112085 s, 4.6 MB/s 2024-01-31T05:14:52.289 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-31T05:14:52.303 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:52 smithi186 ceph-mon[35659]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi156=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:14:52.304 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:52 smithi186 ceph-mon[35659]: Saving service mgr spec with placement smithi045=x;smithi156=y;smithi186=z;count:3 2024-01-31T05:14:52.304 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:52 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:52.304 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:52 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:52.304 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:52 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:52.304 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:52 smithi186 ceph-mon[35659]: Reconfiguring mon.c (monmap changed)... 2024-01-31T05:14:52.304 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:52 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:52.304 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:52 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-31T05:14:52.304 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:52 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:52.304 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:52 smithi186 ceph-mon[35659]: Reconfiguring daemon mon.c on smithi186 2024-01-31T05:14:52.345 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-01-31T05:14:52.345 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2024-01-31T05:14:52.364 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-31T05:14:52.364 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2024-01-31T05:14:52.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:52 smithi045 ceph-mon[31585]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi156=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:14:52.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:52 smithi045 ceph-mon[31585]: Saving service mgr spec with placement smithi045=x;smithi156=y;smithi186=z;count:3 2024-01-31T05:14:52.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:52 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:52.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:52 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:52.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:52 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:52.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:52 smithi045 ceph-mon[31585]: Reconfiguring mon.c (monmap changed)... 2024-01-31T05:14:52.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:52 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:52.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:52 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-31T05:14:52.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:52 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:52.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:52 smithi045 ceph-mon[31585]: Reconfiguring daemon mon.c on smithi186 2024-01-31T05:14:52.422 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-31T05:14:52.422 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T05:14:52.422 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-01-31T05:14:52.422 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:14:52.422 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T05:14:52.422 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-01-31 05:14:38.795646444 +0000 2024-01-31T05:14:52.422 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-01-31 05:09:04.317006564 +0000 2024-01-31T05:14:52.422 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-01-31 05:09:04.317006564 +0000 2024-01-31T05:14:52.422 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-01-31 05:09:04.317006564 +0000 2024-01-31T05:14:52.423 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-31T05:14:52.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:52 smithi156 ceph-mon[35720]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi156=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:14:52.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:52 smithi156 ceph-mon[35720]: Saving service mgr spec with placement smithi045=x;smithi156=y;smithi186=z;count:3 2024-01-31T05:14:52.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:52 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:52.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:52 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:52.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:52 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:52.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:52 smithi156 ceph-mon[35720]: Reconfiguring mon.c (monmap changed)... 2024-01-31T05:14:52.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:52 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-31T05:14:52.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:52 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-31T05:14:52.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:52 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:52.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:52 smithi156 ceph-mon[35720]: Reconfiguring daemon mon.c on smithi186 2024-01-31T05:14:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-01-31T05:14:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-01-31T05:14:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000102885 s, 5.0 MB/s 2024-01-31T05:14:52.491 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-31T05:14:52.549 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2024-01-31T05:14:52.605 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-31T05:14:52.606 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T05:14:52.606 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-01-31T05:14:52.606 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:14:52.606 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T05:14:52.606 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-01-31 05:14:38.795646444 +0000 2024-01-31T05:14:52.606 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-01-31 05:09:04.643001412 +0000 2024-01-31T05:14:52.606 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-01-31 05:09:04.643001412 +0000 2024-01-31T05:14:52.606 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-01-31 05:09:04.643001412 +0000 2024-01-31T05:14:52.606 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-31T05:14:52.672 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-01-31T05:14:52.672 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-01-31T05:14:52.672 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000122355 s, 4.2 MB/s 2024-01-31T05:14:52.673 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-31T05:14:52.730 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2024-01-31T05:14:52.785 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-31T05:14:52.785 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T05:14:52.785 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-01-31T05:14:52.785 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:14:52.785 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T05:14:52.785 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-01-31 05:14:38.796646429 +0000 2024-01-31T05:14:52.785 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-01-31 05:09:04.994995849 +0000 2024-01-31T05:14:52.785 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-01-31 05:09:04.994995849 +0000 2024-01-31T05:14:52.785 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-01-31 05:09:04.994995849 +0000 2024-01-31T05:14:52.785 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-31T05:14:52.852 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-01-31T05:14:52.852 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-01-31T05:14:52.852 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000107028 s, 4.8 MB/s 2024-01-31T05:14:52.852 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-31T05:14:52.907 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2024-01-31T05:14:52.962 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-31T05:14:52.963 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T05:14:52.963 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-01-31T05:14:52.963 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:14:52.963 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T05:14:52.963 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-01-31 05:14:38.796646429 +0000 2024-01-31T05:14:52.963 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-01-31 05:09:05.314990792 +0000 2024-01-31T05:14:52.963 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-01-31 05:09:05.314990792 +0000 2024-01-31T05:14:52.964 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-01-31 05:09:05.314990792 +0000 2024-01-31T05:14:52.964 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-31T05:14:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-01-31T05:14:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-01-31T05:14:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000114773 s, 4.5 MB/s 2024-01-31T05:14:53.029 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-31T05:14:53.085 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-01-31T05:14:53.085 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2024-01-31T05:14:53.103 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-31T05:14:53.103 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2024-01-31T05:14:53.159 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-31T05:14:53.159 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T05:14:53.159 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-01-31T05:14:53.160 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:14:53.160 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T05:14:53.160 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-01-31 05:14:29.505332910 +0000 2024-01-31T05:14:53.160 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-01-31 05:09:10.817823421 +0000 2024-01-31T05:14:53.160 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-01-31 05:09:10.817823421 +0000 2024-01-31T05:14:53.160 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-01-31 05:09:10.817823421 +0000 2024-01-31T05:14:53.160 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-31T05:14:53.229 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-01-31T05:14:53.229 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-01-31T05:14:53.230 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000204377 s, 2.5 MB/s 2024-01-31T05:14:53.231 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-31T05:14:53.288 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2024-01-31T05:14:53.343 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-31T05:14:53.343 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T05:14:53.343 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-01-31T05:14:53.343 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:14:53.344 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T05:14:53.344 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-01-31 05:14:29.506332891 +0000 2024-01-31T05:14:53.344 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-01-31 05:09:11.156816937 +0000 2024-01-31T05:14:53.344 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-01-31 05:09:11.156816937 +0000 2024-01-31T05:14:53.344 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-01-31 05:09:11.156816937 +0000 2024-01-31T05:14:53.344 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-31T05:14:53.411 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-01-31T05:14:53.411 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-01-31T05:14:53.411 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000201291 s, 2.5 MB/s 2024-01-31T05:14:53.412 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-31T05:14:53.468 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2024-01-31T05:14:53.523 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-31T05:14:53.523 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T05:14:53.523 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-01-31T05:14:53.523 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:14:53.523 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T05:14:53.523 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-01-31 05:14:29.506332891 +0000 2024-01-31T05:14:53.523 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-01-31 05:09:11.516810051 +0000 2024-01-31T05:14:53.524 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-01-31 05:09:11.516810051 +0000 2024-01-31T05:14:53.524 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-01-31 05:09:11.516810051 +0000 2024-01-31T05:14:53.524 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-31T05:14:53.586 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:53 smithi186 ceph-mon[35659]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:53.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:53.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:53.587 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:53.589 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-01-31T05:14:53.589 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-01-31T05:14:53.589 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000196587 s, 2.6 MB/s 2024-01-31T05:14:53.590 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-31T05:14:53.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:53 smithi045 ceph-mon[31585]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:53.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:53.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:53.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:53.646 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2024-01-31T05:14:53.702 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-31T05:14:53.702 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T05:14:53.702 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-01-31T05:14:53.702 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:14:53.702 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T05:14:53.703 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-01-31 05:14:29.506332891 +0000 2024-01-31T05:14:53.703 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-01-31 05:09:11.842803816 +0000 2024-01-31T05:14:53.703 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-01-31 05:09:11.842803816 +0000 2024-01-31T05:14:53.703 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-01-31 05:09:11.842803816 +0000 2024-01-31T05:14:53.703 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-31T05:14:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:53 smithi156 ceph-mon[35720]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:53.768 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-01-31T05:14:53.768 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-01-31T05:14:53.768 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000184206 s, 2.8 MB/s 2024-01-31T05:14:53.769 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-31T05:14:53.824 INFO:tasks.cephadm:Deploying osd.0 on smithi045 with /dev/vg_nvme/lv_4... 2024-01-31T05:14:53.824 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-01-31T05:14:54.059 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:14:54.457 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:54 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:54.457 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:54 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:54.458 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:54 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:54.458 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:54 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-31T05:14:54.458 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:54 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-31T05:14:54.458 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:54 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-31T05:14:54.458 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:54 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:54.458 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:54 smithi045 ceph-mon[31585]: Deploying daemon mgr.z on smithi186 2024-01-31T05:14:54.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:54 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:54.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:54 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:54.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:54 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:54.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:54 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-31T05:14:54.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:54 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-31T05:14:54.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:54 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-31T05:14:54.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:54 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:54.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:54 smithi186 ceph-mon[35659]: Deploying daemon mgr.z on smithi186 2024-01-31T05:14:54.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:54 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:54.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:54 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:14:54.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:54 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:54.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:54 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-31T05:14:54.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:54 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-31T05:14:54.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:54 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-31T05:14:54.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:54 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:54.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:54 smithi156 ceph-mon[35720]: Deploying daemon mgr.z on smithi186 2024-01-31T05:14:55.588 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:55 smithi045 ceph-mon[31585]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:55.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:55 smithi156 ceph-mon[35720]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:55.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:55 smithi186 ceph-mon[35659]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:55.742 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:55 smithi186 systemd[1]: Starting Ceph mgr.z for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:14:55.922 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-01-31T05:14:55.922 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-01-31T05:14:55.922 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-01-31T05:14:55.922 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-01-31T05:14:55.922 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00852811 s, 1.2 GB/s 2024-01-31T05:14:55.922 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-01-31T05:14:56.017 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:55 smithi186 podman[37502]: 2024-01-31 05:14:55.646097973 +0000 UTC m=+0.017861682 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:14:56.018 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:55 smithi186 podman[37502]: 2024-01-31 05:14:55.749550989 +0000 UTC m=+0.121314718 container create 371604f44df0544fdde8f52139062031aa5e4b8784c77357d1cb15ecf6f70fb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123) 2024-01-31T05:14:56.018 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:55 smithi186 podman[37502]: 2024-01-31 05:14:55.960570173 +0000 UTC m=+0.332333882 container init 371604f44df0544fdde8f52139062031aa5e4b8784c77357d1cb15ecf6f70fb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3) 2024-01-31T05:14:56.018 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:55 smithi186 podman[37502]: 2024-01-31 05:14:55.963241666 +0000 UTC m=+0.335005376 container start 371604f44df0544fdde8f52139062031aa5e4b8784c77357d1cb15ecf6f70fb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z, io.buildah.version=1.31.3, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2) 2024-01-31T05:14:56.018 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:55 smithi186 ceph-mgr[37521]: -- 172.21.15.186:0/3583050394 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b28367b520 con 0x55b283650c00 2024-01-31T05:14:56.018 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:56 smithi186 bash[37502]: 371604f44df0544fdde8f52139062031aa5e4b8784c77357d1cb15ecf6f70fb9 2024-01-31T05:14:56.341 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:56 smithi186 systemd[1]: Started Ceph mgr.z for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:14:56.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2024-01-31T05:14:56.842 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:56 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:56.490+0000 7ff0bfd4c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-31T05:14:56.842 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:56 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:56.583+0000 7ff0bfd4c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-31T05:14:56.850 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:14:57.278 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:57.278 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:57.278 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:57.278 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-31T05:14:57.278 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-31T05:14:57.278 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-31T05:14:57.279 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:57.279 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:57 smithi045 ceph-mon[31585]: Deploying daemon mgr.y on smithi156 2024-01-31T05:14:57.279 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:57 smithi045 ceph-mon[31585]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:57.341 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:57.225+0000 7ff0bfd4c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-31T05:14:57.341 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:57.322+0000 7ff0bfd4c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-31T05:14:57.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:57.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:57.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:57.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-31T05:14:57.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-31T05:14:57.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-31T05:14:57.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:57.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-mon[35659]: Deploying daemon mgr.y on smithi156 2024-01-31T05:14:57.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-mon[35659]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:57.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:57.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:57.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:57.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-31T05:14:57.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-31T05:14:57.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-31T05:14:57.454 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:57.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:57 smithi156 ceph-mon[35720]: Deploying daemon mgr.y on smithi156 2024-01-31T05:14:57.455 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:57 smithi156 ceph-mon[35720]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:57.657 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:57.413+0000 7ff0bfd4c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-31T05:14:57.657 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:57.572+0000 7ff0bfd4c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-31T05:14:57.658 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:57.658+0000 7ff0bfd4c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-31T05:14:58.024 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:57 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:57.839+0000 7ff0bfd4c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-31T05:14:58.025 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:58 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:58.025+0000 7ff0bfd4c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-31T05:14:58.155 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:14:58 smithi156 systemd[1]: Starting Ceph mgr.y for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:14:58.298 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:58 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:58.113+0000 7ff0bfd4c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-31T05:14:58.298 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:58 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:58.209+0000 7ff0bfd4c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-31T05:14:58.298 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:58 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:58.298+0000 7ff0bfd4c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-31T05:14:58.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.402+0000 7f2b69786700 1 -- 172.21.15.45:0/2642712817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b641022b0 msgr2=0x7f2b64108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:58.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.402+0000 7f2b69786700 1 --2- 172.21.15.45:0/2642712817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b641022b0 0x7f2b64108b60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00b0d0 tx=0x7f2b5c00b3e0 comp rx=0 tx=0).stop 2024-01-31T05:14:58.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.402+0000 7f2b69786700 1 -- 172.21.15.45:0/2642712817 shutdown_connections 2024-01-31T05:14:58.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.402+0000 7f2b69786700 1 --2- 172.21.15.45:0/2642712817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b641022b0 0x7f2b64108b60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:58.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.402+0000 7f2b69786700 1 --2- 172.21.15.45:0/2642712817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b64100710 0x7f2b64100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:58.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.402+0000 7f2b69786700 1 --2- 172.21.15.45:0/2642712817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b64101910 0x7f2b64101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:58.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.402+0000 7f2b69786700 1 -- 172.21.15.45:0/2642712817 >> 172.21.15.45:0/2642712817 conn(0x7f2b640fbcd0 msgr2=0x7f2b640fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.403+0000 7f2b69786700 1 -- 172.21.15.45:0/2642712817 shutdown_connections 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.403+0000 7f2b69786700 1 -- 172.21.15.45:0/2642712817 wait complete. 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.403+0000 7f2b69786700 1 Processor -- start 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.403+0000 7f2b69786700 1 -- start start 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.403+0000 7f2b69786700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b64100710 0x7f2b6406aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b69786700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b64101910 0x7f2b6419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b69786700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b641022b0 0x7f2b6419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b69786700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6419f470 con 0x7f2b641022b0 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b69786700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6419f5e0 con 0x7f2b64100710 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b69786700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6419f7a0 con 0x7f2b64101910 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b641022b0 0x7f2b6419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b641022b0 0x7f2b6419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39520/0 (socket says 172.21.15.45:39520) 2024-01-31T05:14:58.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b62ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b64100710 0x7f2b6406aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b627fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b64101910 0x7f2b6419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b637fe700 1 -- 172.21.15.45:0/2773288135 learned_addr learned my addr 172.21.15.45:0/2773288135 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b637fe700 1 -- 172.21.15.45:0/2773288135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b64101910 msgr2=0x7f2b6419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b637fe700 1 --2- 172.21.15.45:0/2773288135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b64101910 0x7f2b6419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b637fe700 1 -- 172.21.15.45:0/2773288135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b64100710 msgr2=0x7f2b6406aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b637fe700 1 --2- 172.21.15.45:0/2773288135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b64100710 0x7f2b6406aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b637fe700 1 -- 172.21.15.45:0/2773288135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c009d00 con 0x7f2b641022b0 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b637fe700 1 --2- 172.21.15.45:0/2773288135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b641022b0 0x7f2b6419ecd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c007bd0 tx=0x7f2b5c0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b4bfff700 1 -- 172.21.15.45:0/2773288135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00e040 con 0x7f2b641022b0 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b4bfff700 1 -- 172.21.15.45:0/2773288135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b5c004e60 con 0x7f2b641022b0 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b69786700 1 -- 172.21.15.45:0/2773288135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6419fa20 con 0x7f2b641022b0 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b4bfff700 1 -- 172.21.15.45:0/2773288135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c01da20 con 0x7f2b641022b0 2024-01-31T05:14:58.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.404+0000 7f2b69786700 1 -- 172.21.15.45:0/2773288135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6419ff10 con 0x7f2b641022b0 2024-01-31T05:14:58.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.405+0000 7f2b4bfff700 1 -- 172.21.15.45:0/2773288135 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f2b5c005710 con 0x7f2b641022b0 2024-01-31T05:14:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.405+0000 7f2b69786700 1 -- 172.21.15.45:0/2773288135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b64107ad0 con 0x7f2b641022b0 2024-01-31T05:14:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.406+0000 7f2b4bfff700 1 --2- 172.21.15.45:0/2773288135 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2b4c038d20 0x7f2b4c03b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:14:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.406+0000 7f2b4bfff700 1 -- 172.21.15.45:0/2773288135 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2b5c04cb20 con 0x7f2b641022b0 2024-01-31T05:14:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.406+0000 7f2b62ffd700 1 --2- 172.21.15.45:0/2773288135 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2b4c038d20 0x7f2b4c03b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:14:58.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.406+0000 7f2b62ffd700 1 --2- 172.21.15.45:0/2773288135 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2b4c038d20 0x7f2b4c03b1e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2b64101770 tx=0x7f2b54009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:14:58.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.411+0000 7f2b4bfff700 1 -- 172.21.15.45:0/2773288135 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f2b5c011b80 con 0x7f2b641022b0 2024-01-31T05:14:58.476 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:14:58 smithi156 podman[37085]: 2024-01-31 05:14:58.273525399 +0000 UTC m=+0.017442133 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:14:58.476 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:14:58 smithi156 podman[37085]: 2024-01-31 05:14:58.382535157 +0000 UTC m=+0.126451875 container create d98f0bf733a7088667588fb847b956aa616377726a9eb48f7a4e10532c081e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-01-31T05:14:58.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:14:58.620+0000 7f2b69786700 1 -- 172.21.15.45:0/2773288135 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b6402d5a0 con 0x7f2b4c038d20 2024-01-31T05:14:58.976 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:14:58 smithi156 podman[37085]: 2024-01-31 05:14:58.61822893 +0000 UTC m=+0.362145648 container init d98f0bf733a7088667588fb847b956aa616377726a9eb48f7a4e10532c081e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, GIT_CLEAN=True) 2024-01-31T05:14:58.976 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:14:58 smithi156 podman[37085]: 2024-01-31 05:14:58.620892139 +0000 UTC m=+0.364808857 container start d98f0bf733a7088667588fb847b956aa616377726a9eb48f7a4e10532c081e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-01-31T05:14:58.976 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:14:58 smithi156 ceph-mgr[37104]: -- 172.21.15.156:0/177455024 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558bf605b520 con 0x558bf6030c00 2024-01-31T05:14:58.976 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:14:58 smithi156 bash[37085]: d98f0bf733a7088667588fb847b956aa616377726a9eb48f7a4e10532c081e24 2024-01-31T05:14:58.977 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:14:58 smithi156 systemd[1]: Started Ceph mgr.y for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:14:59.476 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:14:59.166+0000 7f4d9699c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-31T05:14:59.476 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:14:59.261+0000 7f4d9699c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-31T05:14:59.698 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:59.403+0000 7ff0bfd4c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-31T05:14:59.699 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:59.548+0000 7ff0bfd4c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-31T05:14:59.699 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-mon[35659]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:59.699 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-mon[35659]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:14:59.699 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:14:59.699 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:14:59.699 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:59.699 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:59.699 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:59.699 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:59.699 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:59.699 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:59.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-mon[35720]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:59.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-mon[35720]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:14:59.852 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:14:59.852 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:14:59.852 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:59.852 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:59.852 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:59.852 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:59.852 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:59.852 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:59.880 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:59 smithi045 ceph-mon[31585]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:14:59.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:59 smithi045 ceph-mon[31585]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:14:59.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:14:59.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:14:59.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:14:59.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:59.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:59.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:59.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:14:59.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:14:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:14:59.980 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:59.746+0000 7ff0bfd4c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-31T05:14:59.980 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:14:59 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:14:59.980+0000 7ff0bfd4c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-31T05:15:00.120 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:14:59 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:14:59.938+0000 7f4d9699c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-31T05:15:00.121 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:00.032+0000 7f4d9699c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-31T05:15:00.121 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:00.121+0000 7f4d9699c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-31T05:15:00.341 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:15:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:15:00.160+0000 7ff0bfd4c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:15:00.375 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:00.288+0000 7f4d9699c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-31T05:15:00.375 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:00.376+0000 7f4d9699c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-31T05:15:00.618 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:15:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:15:00.351+0000 7ff0bfd4c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-31T05:15:00.726 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:00.558+0000 7f4d9699c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-31T05:15:01.017 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:00.747+0000 7f4d9699c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-31T05:15:01.017 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:00.834+0000 7f4d9699c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-31T05:15:01.017 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:00.928+0000 7f4d9699c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-31T05:15:01.017 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:01.017+0000 7f4d9699c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-31T05:15:01.091 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:15:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:15:00.618+0000 7ff0bfd4c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:15:01.092 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:15:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:15:00.703+0000 7ff0bfd4c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-31T05:15:01.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:01.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:01.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: Reconfiguring mgr.x (unknown last config time)... 2024-01-31T05:15:01.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-31T05:15:01.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-31T05:15:01.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:01.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: Reconfiguring daemon mgr.x on smithi045 2024-01-31T05:15:01.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:01 smithi156 ceph-mon[35720]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:01.591 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:15:01.093+0000 7ff0bfd4c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-31T05:15:01.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:01.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:01.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: Reconfiguring mgr.x (unknown last config time)... 2024-01-31T05:15:01.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-31T05:15:01.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-31T05:15:01.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:01.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: Reconfiguring daemon mgr.x on smithi045 2024-01-31T05:15:01.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-mon[35659]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:01.633 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.633 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.633 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:01.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:01.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:01.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: Reconfiguring mgr.x (unknown last config time)... 2024-01-31T05:15:01.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-31T05:15:01.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-31T05:15:01.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:01.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: Reconfiguring daemon mgr.x on smithi045 2024-01-31T05:15:01.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:01 smithi045 ceph-mon[31585]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:02.202 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:15:01 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:15:01.873+0000 7ff0bfd4c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-31T05:15:02.407 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:02 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3213427534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b06f7af-6fee-4838-afe2-f178362f0a88"}]: dispatch 2024-01-31T05:15:02.408 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:02 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3213427534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b06f7af-6fee-4838-afe2-f178362f0a88"}]': finished 2024-01-31T05:15:02.408 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:02 smithi156 ceph-mon[35720]: osdmap e5: 1 total, 0 up, 1 in 2024-01-31T05:15:02.408 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:02.408 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:02.408 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:02 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3413585516' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:15:02.408 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:02 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:02.084+0000 7f4d9699c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-31T05:15:02.408 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:02 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:02.215+0000 7f4d9699c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-31T05:15:02.408 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:02 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:02.408+0000 7f4d9699c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-31T05:15:02.530 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:15:02 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:15:02.436+0000 7ff0bfd4c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-31T05:15:02.530 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:02 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3213427534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b06f7af-6fee-4838-afe2-f178362f0a88"}]: dispatch 2024-01-31T05:15:02.530 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:02 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3213427534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b06f7af-6fee-4838-afe2-f178362f0a88"}]': finished 2024-01-31T05:15:02.530 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:02 smithi186 ceph-mon[35659]: osdmap e5: 1 total, 0 up, 1 in 2024-01-31T05:15:02.530 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:02.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:02.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:02 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3413585516' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:15:02.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:02 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3213427534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b06f7af-6fee-4838-afe2-f178362f0a88"}]: dispatch 2024-01-31T05:15:02.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:02 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3213427534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b06f7af-6fee-4838-afe2-f178362f0a88"}]': finished 2024-01-31T05:15:02.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:02 smithi045 ceph-mon[31585]: osdmap e5: 1 total, 0 up, 1 in 2024-01-31T05:15:02.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:02.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:02.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:02 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3413585516' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:15:02.726 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:02 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:02.646+0000 7f4d9699c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-31T05:15:02.841 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:15:02 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:15:02.531+0000 7ff0bfd4c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-31T05:15:02.841 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:15:02 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:15:02.726+0000 7ff0bfd4c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-31T05:15:03.226 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:02 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:02.826+0000 7f4d9699c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:15:03.226 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:03 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:03.009+0000 7f4d9699c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-31T05:15:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:03.377+0000 7f2b4bfff700 1 -- 172.21.15.45:0/2773288135 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 14) v1 ==== 90768+0+0 (secure 0 0 0) 0x7f2b5c012990 con 0x7f2b641022b0 2024-01-31T05:15:03.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:03 smithi045 ceph-mon[31585]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:03.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:03 smithi045 ceph-mon[31585]: Standby manager daemon z started 2024-01-31T05:15:03.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:03 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.186:0/2432228790' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-31T05:15:03.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:03 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.186:0/2432228790' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-31T05:15:03.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:03 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.186:0/2432228790' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-31T05:15:03.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:03 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.186:0/2432228790' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-31T05:15:03.726 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:03 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:03.271+0000 7f4d9699c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:15:03.726 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:03 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:03.356+0000 7f4d9699c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-31T05:15:03.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:03 smithi156 ceph-mon[35720]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:03.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:03 smithi156 ceph-mon[35720]: Standby manager daemon z started 2024-01-31T05:15:03.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:03 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.186:0/2432228790' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-31T05:15:03.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:03 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.186:0/2432228790' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-31T05:15:03.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:03 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.186:0/2432228790' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-31T05:15:03.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:03 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.186:0/2432228790' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-31T05:15:03.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:03 smithi186 ceph-mon[35659]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:03.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:03 smithi186 ceph-mon[35659]: Standby manager daemon z started 2024-01-31T05:15:03.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:03 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.186:0/2432228790' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-31T05:15:03.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:03 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.186:0/2432228790' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-31T05:15:03.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:03 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.186:0/2432228790' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-31T05:15:03.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:03 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.186:0/2432228790' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-31T05:15:04.226 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:03 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:03.753+0000 7f4d9699c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-31T05:15:04.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:04 smithi045 ceph-mon[31585]: mgrmap e14: x(active, since 76s), standbys: z 2024-01-31T05:15:04.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:04 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-31T05:15:04.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:04 smithi156 ceph-mon[35720]: mgrmap e14: x(active, since 76s), standbys: z 2024-01-31T05:15:04.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:04 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-31T05:15:04.726 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:04 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:04.534+0000 7f4d9699c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-31T05:15:04.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:04 smithi186 ceph-mon[35659]: mgrmap e14: x(active, since 76s), standbys: z 2024-01-31T05:15:04.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:04 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-31T05:15:05.379 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:05.102+0000 7f4d9699c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-31T05:15:05.379 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:05.198+0000 7f4d9699c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-31T05:15:05.488 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:05 smithi045 ceph-mon[31585]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:05.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:05 smithi156 ceph-mon[35720]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:05.726 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:15:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:15:05.393+0000 7f4d9699c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-31T05:15:05.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:05 smithi186 ceph-mon[35659]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:06.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:06.392+0000 7f2b4bfff700 1 -- 172.21.15.45:0/2773288135 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 15) v1 ==== 135839+0+0 (secure 0 0 0) 0x7f2b5c019bc0 con 0x7f2b641022b0 2024-01-31T05:15:06.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:06 smithi045 ceph-mon[31585]: Standby manager daemon y started 2024-01-31T05:15:06.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:06 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.156:0/109554784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-31T05:15:06.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:06 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.156:0/109554784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-31T05:15:06.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:06 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.156:0/109554784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-31T05:15:06.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:06 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.156:0/109554784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-31T05:15:06.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:06 smithi156 ceph-mon[35720]: Standby manager daemon y started 2024-01-31T05:15:06.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:06 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.156:0/109554784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-31T05:15:06.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:06 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.156:0/109554784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-31T05:15:06.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:06 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.156:0/109554784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-31T05:15:06.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:06 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.156:0/109554784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-31T05:15:06.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:06 smithi186 ceph-mon[35659]: Standby manager daemon y started 2024-01-31T05:15:06.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:06 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.156:0/109554784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-31T05:15:06.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:06 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.156:0/109554784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-31T05:15:06.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:06 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.156:0/109554784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-31T05:15:06.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:06 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.156:0/109554784' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-31T05:15:07.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:07 smithi045 ceph-mon[31585]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-01-31T05:15:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-31T05:15:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:07 smithi045 ceph-mon[31585]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:07.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:07 smithi156 ceph-mon[35720]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-01-31T05:15:07.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-31T05:15:07.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:07 smithi156 ceph-mon[35720]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:07.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:07.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:07.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:07.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:07.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:07.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:07.784 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:07 smithi186 ceph-mon[35659]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-01-31T05:15:07.785 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-31T05:15:07.785 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:07 smithi186 ceph-mon[35659]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:07.785 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:07.785 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:07.785 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:07.785 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:07.785 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:07.785 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:09.834 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:09 smithi045 ceph-mon[31585]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:09.834 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-01-31T05:15:09.834 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:09.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:09 smithi186 ceph-mon[35659]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:09.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-01-31T05:15:09.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:09.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:09 smithi156 ceph-mon[35720]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:09.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-01-31T05:15:09.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:10.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:10 smithi186 ceph-mon[35659]: Deploying daemon osd.0 on smithi045 2024-01-31T05:15:10.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:10 smithi045 ceph-mon[31585]: Deploying daemon osd.0 on smithi045 2024-01-31T05:15:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:10 smithi156 ceph-mon[35720]: Deploying daemon osd.0 on smithi045 2024-01-31T05:15:11.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:11 smithi186 ceph-mon[35659]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:11.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:11 smithi045 ceph-mon[31585]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:11 smithi156 ceph-mon[35720]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:13.483 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:13 smithi045 ceph-mon[31585]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:13 smithi156 ceph-mon[35720]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:13.844 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:13 smithi186 ceph-mon[35659]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:15.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:15.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:15.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:15.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:15 smithi186 ceph-mon[35659]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:15.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:15.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:15.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:15.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:15 smithi045 ceph-mon[31585]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:15.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:15.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:15.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:15.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:15 smithi156 ceph-mon[35720]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.625+0000 7f2b4bfff700 1 -- 172.21.15.45:0/2773288135 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2b6402d5a0 con 0x7f2b4c038d20 2024-01-31T05:15:17.625 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 0 on host 'smithi045' 2024-01-31T05:15:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.627+0000 7f2b69786700 1 -- 172.21.15.45:0/2773288135 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2b4c038d20 msgr2=0x7f2b4c03b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:15:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.627+0000 7f2b69786700 1 --2- 172.21.15.45:0/2773288135 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2b4c038d20 0x7f2b4c03b1e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2b64101770 tx=0x7f2b54009190 comp rx=0 tx=0).stop 2024-01-31T05:15:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.627+0000 7f2b69786700 1 -- 172.21.15.45:0/2773288135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b641022b0 msgr2=0x7f2b6419ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:15:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.627+0000 7f2b69786700 1 --2- 172.21.15.45:0/2773288135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b641022b0 0x7f2b6419ecd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c007bd0 tx=0x7f2b5c0094c0 comp rx=0 tx=0).stop 2024-01-31T05:15:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.628+0000 7f2b69786700 1 -- 172.21.15.45:0/2773288135 shutdown_connections 2024-01-31T05:15:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.628+0000 7f2b69786700 1 --2- 172.21.15.45:0/2773288135 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2b4c038d20 0x7f2b4c03b1e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.628+0000 7f2b69786700 1 --2- 172.21.15.45:0/2773288135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b641022b0 0x7f2b6419ecd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.628+0000 7f2b69786700 1 --2- 172.21.15.45:0/2773288135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b64100710 0x7f2b6406aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:17.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.628+0000 7f2b69786700 1 --2- 172.21.15.45:0/2773288135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b64101910 0x7f2b6419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:17.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.628+0000 7f2b69786700 1 -- 172.21.15.45:0/2773288135 >> 172.21.15.45:0/2773288135 conn(0x7f2b640fbcd0 msgr2=0x7f2b64104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:15:17.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.628+0000 7f2b69786700 1 -- 172.21.15.45:0/2773288135 shutdown_connections 2024-01-31T05:15:17.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:15:17.628+0000 7f2b69786700 1 -- 172.21.15.45:0/2773288135 wait complete. 2024-01-31T05:15:17.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:17 smithi186 ceph-mon[35659]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:17.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:17 smithi045 ceph-mon[31585]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:17 smithi156 ceph-mon[35720]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:18.222 DEBUG:teuthology.orchestra.run.smithi045:osd.0> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.0.service 2024-01-31T05:15:18.225 INFO:tasks.cephadm:Deploying osd.1 on smithi156 with /dev/vg_nvme/lv_4... 2024-01-31T05:15:18.225 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-01-31T05:15:18.465 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.b/config 2024-01-31T05:15:18.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:18 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:18.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:18 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:18.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:18 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:18.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:18 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:18.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:18 smithi186 ceph-mon[35659]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-01-31T05:15:18.884 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:18 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:18.884 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:18 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:18.884 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:18 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:18.884 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:18 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:18.884 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:18 smithi156 ceph-mon[35720]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-01-31T05:15:18.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:18 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:18.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:18 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:18.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:18 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:18.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:18 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:18.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:18 smithi045 ceph-mon[31585]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-01-31T05:15:19.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:19 smithi156 ceph-mon[35720]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:19.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:19 smithi156 ceph-mon[35720]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-01-31T05:15:19.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:19 smithi156 ceph-mon[35720]: osdmap e6: 1 total, 0 up, 1 in 2024-01-31T05:15:19.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:19 smithi156 ceph-mon[35720]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-01-31T05:15:19.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:19 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:19.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:19 smithi045 ceph-mon[31585]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:19.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:19 smithi045 ceph-mon[31585]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-01-31T05:15:19.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:19 smithi045 ceph-mon[31585]: osdmap e6: 1 total, 0 up, 1 in 2024-01-31T05:15:19.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:19 smithi045 ceph-mon[31585]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-01-31T05:15:19.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:20.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:19 smithi186 ceph-mon[35659]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:20.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:19 smithi186 ceph-mon[35659]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-01-31T05:15:20.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:19 smithi186 ceph-mon[35659]: osdmap e6: 1 total, 0 up, 1 in 2024-01-31T05:15:20.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:19 smithi186 ceph-mon[35659]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-01-31T05:15:20.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:19 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:20.533 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-01-31T05:15:20.533 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-01-31T05:15:20.533 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-01-31T05:15:20.534 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-01-31T05:15:20.534 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00860456 s, 1.2 GB/s 2024-01-31T05:15:20.534 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-01-31T05:15:20.643 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:15:20 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0[46341]: 2024-01-31T05:15:20.382+0000 7fe4fdcdf700 -1 osd.0 0 waiting for initial osdmap 2024-01-31T05:15:20.643 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:15:20 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0[46341]: 2024-01-31T05:15:20.385+0000 7fe4f62d2700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:15:20.946 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:20 smithi156 ceph-mon[35720]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-01-31T05:15:20.946 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:20 smithi156 ceph-mon[35720]: osdmap e7: 1 total, 0 up, 1 in 2024-01-31T05:15:20.946 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:20 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:20.946 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:20 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:20.947 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:20 smithi156 ceph-mon[35720]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' 2024-01-31T05:15:21.025 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:20 smithi045 ceph-mon[31585]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-01-31T05:15:21.025 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:20 smithi045 ceph-mon[31585]: osdmap e7: 1 total, 0 up, 1 in 2024-01-31T05:15:21.025 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:20 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:21.025 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:20 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:21.025 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:20 smithi045 ceph-mon[31585]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' 2024-01-31T05:15:21.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:20 smithi186 ceph-mon[35659]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-01-31T05:15:21.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:20 smithi186 ceph-mon[35659]: osdmap e7: 1 total, 0 up, 1 in 2024-01-31T05:15:21.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:20 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:21.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:20 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:21.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:20 smithi186 ceph-mon[35659]: from='osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784]' entity='osd.0' 2024-01-31T05:15:21.283 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_4 2024-01-31T05:15:21.521 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.b/config 2024-01-31T05:15:21.850 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: purged_snaps scrub starts 2024-01-31T05:15:21.850 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: purged_snaps scrub ok 2024-01-31T05:15:21.850 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:21.850 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:21.850 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:21.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:21.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:15:21.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:21.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:21.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:21.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:21.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:21.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784] boot 2024-01-31T05:15:21.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: osdmap e8: 1 total, 1 up, 1 in 2024-01-31T05:15:21.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:22.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: purged_snaps scrub starts 2024-01-31T05:15:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: purged_snaps scrub ok 2024-01-31T05:15:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:15:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784] boot 2024-01-31T05:15:22.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: osdmap e8: 1 total, 1 up, 1 in 2024-01-31T05:15:22.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:22.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: purged_snaps scrub starts 2024-01-31T05:15:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: purged_snaps scrub ok 2024-01-31T05:15:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-31T05:15:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:15:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: osd.0 [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784] boot 2024-01-31T05:15:22.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: osdmap e8: 1 total, 1 up, 1 in 2024-01-31T05:15:22.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:15:22.840 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:22 smithi156 ceph-mon[35720]: Detected new or changed devices on smithi045 2024-01-31T05:15:22.840 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:22 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi045 to 17093M 2024-01-31T05:15:22.840 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:22.840 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:22.840 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:23.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.041+0000 7fb97b868700 1 -- 172.21.15.156:0/2201874977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb974101cc0 msgr2=0x7fb9741020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:15:23.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.041+0000 7fb97b868700 1 --2- 172.21.15.156:0/2201874977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb974101cc0 0x7fb9741020e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb964009990 tx=0x7fb964009ca0 comp rx=0 tx=0).stop 2024-01-31T05:15:23.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.041+0000 7fb97b868700 1 -- 172.21.15.156:0/2201874977 shutdown_connections 2024-01-31T05:15:23.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.041+0000 7fb97b868700 1 --2- 172.21.15.156:0/2201874977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9741013a0 0x7fb974108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:23.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.041+0000 7fb97b868700 1 --2- 172.21.15.156:0/2201874977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb974101cc0 0x7fb9741020e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:23.045 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.041+0000 7fb97b868700 1 --2- 172.21.15.156:0/2201874977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974100a00 0x7fb974100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:23.045 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.041+0000 7fb97b868700 1 -- 172.21.15.156:0/2201874977 >> 172.21.15.156:0/2201874977 conn(0x7fb9740fb470 msgr2=0x7fb9740fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:15:23.045 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.041+0000 7fb97b868700 1 -- 172.21.15.156:0/2201874977 shutdown_connections 2024-01-31T05:15:23.045 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.041+0000 7fb97b868700 1 -- 172.21.15.156:0/2201874977 wait complete. 2024-01-31T05:15:23.045 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.042+0000 7fb97b868700 1 Processor -- start 2024-01-31T05:15:23.045 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.042+0000 7fb97b868700 1 -- start start 2024-01-31T05:15:23.045 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.042+0000 7fb97b868700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb974100a00 0x7fb974197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:15:23.045 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.042+0000 7fb97b868700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9741013a0 0x7fb974197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:15:23.045 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.042+0000 7fb97b868700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974101cc0 0x7fb97419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:15:23.045 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.042+0000 7fb97b868700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97419f270 con 0x7fb974101cc0 2024-01-31T05:15:23.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.042+0000 7fb97b868700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97419f3b0 con 0x7fb9741013a0 2024-01-31T05:15:23.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.042+0000 7fb97b868700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97419f540 con 0x7fb974100a00 2024-01-31T05:15:23.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.042+0000 7fb978e03700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9741013a0 0x7fb974197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:15:23.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb978e03700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9741013a0 0x7fb974197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60428/0 (socket says 172.21.15.156:60428) 2024-01-31T05:15:23.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb978e03700 1 -- 172.21.15.156:0/3785657265 learned_addr learned my addr 172.21.15.156:0/3785657265 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-01-31T05:15:23.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb979e05700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974101cc0 0x7fb97419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:15:23.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb979604700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb974100a00 0x7fb974197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:15:23.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb978e03700 1 -- 172.21.15.156:0/3785657265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb974100a00 msgr2=0x7fb974197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:15:23.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb978e03700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb974100a00 0x7fb974197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:23.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb978e03700 1 -- 172.21.15.156:0/3785657265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974101cc0 msgr2=0x7fb97419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:15:23.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb978e03700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974101cc0 0x7fb97419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:23.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb978e03700 1 -- 172.21.15.156:0/3785657265 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9640095f0 con 0x7fb9741013a0 2024-01-31T05:15:23.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb978e03700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9741013a0 0x7fb974197c70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb97000e9a0 tx=0x7fb97000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:15:23.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb96a7fc700 1 -- 172.21.15.156:0/3785657265 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970009790 con 0x7fb9741013a0 2024-01-31T05:15:23.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb96a7fc700 1 -- 172.21.15.156:0/3785657265 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb97000cbd0 con 0x7fb9741013a0 2024-01-31T05:15:23.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb97b868700 1 -- 172.21.15.156:0/3785657265 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97419f820 con 0x7fb9741013a0 2024-01-31T05:15:23.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb96a7fc700 1 -- 172.21.15.156:0/3785657265 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970018680 con 0x7fb9741013a0 2024-01-31T05:15:23.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.043+0000 7fb97b868700 1 -- 172.21.15.156:0/3785657265 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97419fd70 con 0x7fb9741013a0 2024-01-31T05:15:23.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.044+0000 7fb97b868700 1 -- 172.21.15.156:0/3785657265 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb974107bd0 con 0x7fb9741013a0 2024-01-31T05:15:23.048 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.045+0000 7fb96a7fc700 1 -- 172.21.15.156:0/3785657265 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 15) v1 ==== 135839+0+0 (secure 0 0 0) 0x7fb97000cd40 con 0x7fb9741013a0 2024-01-31T05:15:23.048 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.046+0000 7fb96a7fc700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb9600a1360 0x7fb9600a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:15:23.048 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.046+0000 7fb96a7fc700 1 -- 172.21.15.156:0/3785657265 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fb970014070 con 0x7fb9741013a0 2024-01-31T05:15:23.048 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.046+0000 7fb979604700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb9600a1360 0x7fb9600a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:15:23.048 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.046+0000 7fb979604700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb9600a1360 0x7fb9600a3820 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb96400b7b0 tx=0x7fb964005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:15:23.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.050+0000 7fb96a7fc700 1 -- 172.21.15.156:0/3785657265 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb970052600 con 0x7fb9741013a0 2024-01-31T05:15:23.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:22 smithi186 ceph-mon[35659]: Detected new or changed devices on smithi045 2024-01-31T05:15:23.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:22 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi045 to 17093M 2024-01-31T05:15:23.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:23.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:23.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:23.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:22 smithi045 ceph-mon[31585]: Detected new or changed devices on smithi045 2024-01-31T05:15:23.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:22 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi045 to 17093M 2024-01-31T05:15:23.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:23.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:23.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:23.250 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:23.249+0000 7fb97b868700 1 -- 172.21.15.156:0/3785657265 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb97402d4a0 con 0x7fb9600a1360 2024-01-31T05:15:23.944 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:23 smithi156 ceph-mon[35720]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:23.944 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:23 smithi156 ceph-mon[35720]: osdmap e9: 1 total, 1 up, 1 in 2024-01-31T05:15:23.944 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:23 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:15:23.944 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:23 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:15:23.944 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:23 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:24.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:23 smithi186 ceph-mon[35659]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:24.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:23 smithi186 ceph-mon[35659]: osdmap e9: 1 total, 1 up, 1 in 2024-01-31T05:15:24.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:23 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:15:24.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:23 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:15:24.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:23 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:24.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:23 smithi045 ceph-mon[31585]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:23 smithi045 ceph-mon[31585]: osdmap e9: 1 total, 1 up, 1 in 2024-01-31T05:15:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:15:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:15:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:24.943 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:24 smithi156 ceph-mon[35720]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:15:24.943 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:24 smithi156 ceph-mon[35720]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:25.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:24 smithi186 ceph-mon[35659]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:15:25.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:24 smithi186 ceph-mon[35659]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:25.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:24 smithi045 ceph-mon[31585]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:15:25.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:24 smithi045 ceph-mon[31585]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:26.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:25 smithi156 ceph-mon[35720]: from='client.? 172.21.15.156:0/289006536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ae3ab3f-342a-4e5a-93a2-a674831b30bd"}]: dispatch 2024-01-31T05:15:26.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:25 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ae3ab3f-342a-4e5a-93a2-a674831b30bd"}]: dispatch 2024-01-31T05:15:26.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:25 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ae3ab3f-342a-4e5a-93a2-a674831b30bd"}]': finished 2024-01-31T05:15:26.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:25 smithi156 ceph-mon[35720]: osdmap e10: 2 total, 1 up, 2 in 2024-01-31T05:15:26.047 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:26.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:25 smithi186 ceph-mon[35659]: from='client.? 172.21.15.156:0/289006536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ae3ab3f-342a-4e5a-93a2-a674831b30bd"}]: dispatch 2024-01-31T05:15:26.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:25 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ae3ab3f-342a-4e5a-93a2-a674831b30bd"}]: dispatch 2024-01-31T05:15:26.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:25 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ae3ab3f-342a-4e5a-93a2-a674831b30bd"}]': finished 2024-01-31T05:15:26.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:25 smithi186 ceph-mon[35659]: osdmap e10: 2 total, 1 up, 2 in 2024-01-31T05:15:26.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:25 smithi045 ceph-mon[31585]: from='client.? 172.21.15.156:0/289006536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ae3ab3f-342a-4e5a-93a2-a674831b30bd"}]: dispatch 2024-01-31T05:15:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:25 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ae3ab3f-342a-4e5a-93a2-a674831b30bd"}]: dispatch 2024-01-31T05:15:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:25 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ae3ab3f-342a-4e5a-93a2-a674831b30bd"}]': finished 2024-01-31T05:15:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:25 smithi045 ceph-mon[31585]: osdmap e10: 2 total, 1 up, 2 in 2024-01-31T05:15:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:27.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:26 smithi186 ceph-mon[35659]: from='client.? 172.21.15.156:0/3457039079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:15:27.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:26 smithi186 ceph-mon[35659]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:27.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:26 smithi045 ceph-mon[31585]: from='client.? 172.21.15.156:0/3457039079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:15:27.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:26 smithi045 ceph-mon[31585]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:27.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:26 smithi156 ceph-mon[35720]: from='client.? 172.21.15.156:0/3457039079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:15:27.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:26 smithi156 ceph-mon[35720]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:29.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:29 smithi186 ceph-mon[35659]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:29.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:29 smithi045 ceph-mon[31585]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:29.911 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:29 smithi156 ceph-mon[35720]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:31.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:31 smithi186 ceph-mon[35659]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:31.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:31 smithi156 ceph-mon[35720]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:31.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:31 smithi045 ceph-mon[31585]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:33.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:33 smithi045 ceph-mon[31585]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:33.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:33 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-01-31T05:15:33.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:33 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:33.675 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:33 smithi156 ceph-mon[35720]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:33.676 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:33 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-01-31T05:15:33.676 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:33 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:33.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:33 smithi186 ceph-mon[35659]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:33.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:33 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-01-31T05:15:33.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:33 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:34.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:34 smithi045 ceph-mon[31585]: Deploying daemon osd.1 on smithi156 2024-01-31T05:15:34.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:34 smithi156 ceph-mon[35720]: Deploying daemon osd.1 on smithi156 2024-01-31T05:15:34.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:34 smithi186 ceph-mon[35659]: Deploying daemon osd.1 on smithi156 2024-01-31T05:15:35.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:35 smithi045 ceph-mon[31585]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:35.701 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:35 smithi156 ceph-mon[35720]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:35.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:35 smithi186 ceph-mon[35659]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:37.809 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:37 smithi156 ceph-mon[35720]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:37.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:37 smithi186 ceph-mon[35659]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:37.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:37 smithi045 ceph-mon[31585]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:39.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:39.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:39.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:39.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:39 smithi156 ceph-mon[35720]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:39.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:39.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:39.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:39.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:39 smithi186 ceph-mon[35659]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:39.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:39.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:39.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:39.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:39 smithi045 ceph-mon[31585]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:41.615 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.615+0000 7fb96a7fc700 1 -- 172.21.15.156:0/3785657265 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb97402d4a0 con 0x7fb9600a1360 2024-01-31T05:15:41.615 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 1 on host 'smithi156' 2024-01-31T05:15:41.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.618+0000 7fb97b868700 1 -- 172.21.15.156:0/3785657265 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb9600a1360 msgr2=0x7fb9600a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:15:41.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.618+0000 7fb97b868700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb9600a1360 0x7fb9600a3820 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb96400b7b0 tx=0x7fb964005fb0 comp rx=0 tx=0).stop 2024-01-31T05:15:41.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.618+0000 7fb97b868700 1 -- 172.21.15.156:0/3785657265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9741013a0 msgr2=0x7fb974197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:15:41.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.618+0000 7fb97b868700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9741013a0 0x7fb974197c70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb97000e9a0 tx=0x7fb97000ecb0 comp rx=0 tx=0).stop 2024-01-31T05:15:41.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.618+0000 7fb97b868700 1 -- 172.21.15.156:0/3785657265 shutdown_connections 2024-01-31T05:15:41.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.618+0000 7fb97b868700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb9600a1360 0x7fb9600a3820 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:41.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.618+0000 7fb97b868700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974101cc0 0x7fb97419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:41.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.618+0000 7fb97b868700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb974100a00 0x7fb974197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:41.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.618+0000 7fb97b868700 1 --2- 172.21.15.156:0/3785657265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9741013a0 0x7fb974197c70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:41.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.618+0000 7fb97b868700 1 -- 172.21.15.156:0/3785657265 >> 172.21.15.156:0/3785657265 conn(0x7fb9740fb470 msgr2=0x7fb974104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:15:41.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.618+0000 7fb97b868700 1 -- 172.21.15.156:0/3785657265 shutdown_connections 2024-01-31T05:15:41.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:15:41.618+0000 7fb97b868700 1 -- 172.21.15.156:0/3785657265 wait complete. 2024-01-31T05:15:41.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:41 smithi186 ceph-mon[35659]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:41.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:41 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:41.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:41 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:41.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:41 smithi045 ceph-mon[31585]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:41.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:41 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:41.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:41 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:41.930 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:41 smithi156 ceph-mon[35720]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:41.930 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:41 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:41.930 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:41 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:42.224 DEBUG:teuthology.orchestra.run.smithi156:osd.1> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.1.service 2024-01-31T05:15:42.227 INFO:tasks.cephadm:Deploying osd.2 on smithi186 with /dev/vg_nvme/lv_4... 2024-01-31T05:15:42.227 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-01-31T05:15:42.475 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.c/config 2024-01-31T05:15:42.476 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:15:42 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1[40319]: 2024-01-31T05:15:42.342+0000 7fc81871a7c0 -1 osd.1 0 log_to_monitors true 2024-01-31T05:15:42.863 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:42 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:42.863 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:42 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:42.863 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:42 smithi186 ceph-mon[35659]: from='osd.1 [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-31T05:15:42.863 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:42 smithi186 ceph-mon[35659]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-31T05:15:42.863 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:42 smithi186 ceph-mon[35659]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:42 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:42 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:42 smithi156 ceph-mon[35720]: from='osd.1 [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-31T05:15:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:42 smithi156 ceph-mon[35720]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-31T05:15:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:42 smithi156 ceph-mon[35720]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:43.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:42 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:43.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:42 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:43.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:42 smithi045 ceph-mon[31585]: from='osd.1 [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-31T05:15:43.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:42 smithi045 ceph-mon[31585]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-31T05:15:43.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:42 smithi045 ceph-mon[31585]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:43.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:43 smithi186 ceph-mon[35659]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-01-31T05:15:43.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:43 smithi186 ceph-mon[35659]: from='osd.1 [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:15:43.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:43 smithi186 ceph-mon[35659]: osdmap e11: 2 total, 1 up, 2 in 2024-01-31T05:15:43.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:43 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:43.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:43 smithi186 ceph-mon[35659]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:15:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:43 smithi156 ceph-mon[35720]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-01-31T05:15:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:43 smithi156 ceph-mon[35720]: from='osd.1 [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:15:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:43 smithi156 ceph-mon[35720]: osdmap e11: 2 total, 1 up, 2 in 2024-01-31T05:15:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:43 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:43.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:43 smithi156 ceph-mon[35720]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:15:43.977 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:15:43 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1[40319]: 2024-01-31T05:15:43.696+0000 7fc80d57f700 -1 osd.1 0 waiting for initial osdmap 2024-01-31T05:15:43.977 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:15:43 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1[40319]: 2024-01-31T05:15:43.700+0000 7fc809b7a700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:15:44.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:43 smithi045 ceph-mon[31585]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-01-31T05:15:44.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:43 smithi045 ceph-mon[31585]: from='osd.1 [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:15:44.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:43 smithi045 ceph-mon[31585]: osdmap e11: 2 total, 1 up, 2 in 2024-01-31T05:15:44.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:43 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:44.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:43 smithi045 ceph-mon[31585]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:15:44.425 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-01-31T05:15:44.425 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-01-31T05:15:44.425 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-01-31T05:15:44.425 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-01-31T05:15:44.425 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112148 s, 935 MB/s 2024-01-31T05:15:44.425 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-01-31T05:15:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:44 smithi156 ceph-mon[35720]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-01-31T05:15:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:44 smithi156 ceph-mon[35720]: osdmap e12: 2 total, 1 up, 2 in 2024-01-31T05:15:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:44 smithi156 ceph-mon[35720]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:45.080 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:44 smithi186 ceph-mon[35659]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-01-31T05:15:45.080 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:44 smithi186 ceph-mon[35659]: osdmap e12: 2 total, 1 up, 2 in 2024-01-31T05:15:45.080 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:45.080 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:45.080 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:44 smithi186 ceph-mon[35659]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:45.107 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch daemon add osd smithi186:vg_nvme/lv_4 2024-01-31T05:15:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:44 smithi045 ceph-mon[31585]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-01-31T05:15:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:44 smithi045 ceph-mon[31585]: osdmap e12: 2 total, 1 up, 2 in 2024-01-31T05:15:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:44 smithi045 ceph-mon[31585]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-01-31T05:15:45.351 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.c/config 2024-01-31T05:15:45.757 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: purged_snaps scrub starts 2024-01-31T05:15:45.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: purged_snaps scrub ok 2024-01-31T05:15:45.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: osd.1 [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514] boot 2024-01-31T05:15:45.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: osdmap e13: 2 total, 2 up, 2 in 2024-01-31T05:15:45.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:45.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: Detected new or changed devices on smithi156 2024-01-31T05:15:45.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:45.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:45.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:15:45.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi156 to 17080M 2024-01-31T05:15:45.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:45.758 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:45.759 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:45.759 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:45.759 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:45.759 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:45.759 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:45.759 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: purged_snaps scrub starts 2024-01-31T05:15:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: purged_snaps scrub ok 2024-01-31T05:15:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: osd.1 [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514] boot 2024-01-31T05:15:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: osdmap e13: 2 total, 2 up, 2 in 2024-01-31T05:15:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: Detected new or changed devices on smithi156 2024-01-31T05:15:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:15:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi156 to 17080M 2024-01-31T05:15:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:45.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:45.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:45.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:46.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: purged_snaps scrub starts 2024-01-31T05:15:46.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: purged_snaps scrub ok 2024-01-31T05:15:46.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: osd.1 [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514] boot 2024-01-31T05:15:46.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: osdmap e13: 2 total, 2 up, 2 in 2024-01-31T05:15:46.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:15:46.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: Detected new or changed devices on smithi156 2024-01-31T05:15:46.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:46.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:46.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:15:46.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi156 to 17080M 2024-01-31T05:15:46.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:46.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:46.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:46.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:46.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:15:46.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:46.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:15:46.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:15:46.997 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.929+0000 7f0eab26a700 1 -- 172.21.15.186:0/1121148910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea4102a10 msgr2=0x7f0ea4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:15:46.997 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.929+0000 7f0eab26a700 1 --2- 172.21.15.186:0/1121148910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea4102a10 0x7f0ea4102e70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009990 tx=0x7f0ea0009ca0 comp rx=0 tx=0).stop 2024-01-31T05:15:46.997 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.929+0000 7f0eab26a700 1 -- 172.21.15.186:0/1121148910 shutdown_connections 2024-01-31T05:15:46.997 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.929+0000 7f0eab26a700 1 --2- 172.21.15.186:0/1121148910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41033b0 0x7f0ea4109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:46.997 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.929+0000 7f0eab26a700 1 --2- 172.21.15.186:0/1121148910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4101810 0x7f0ea4101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:46.997 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.929+0000 7f0eab26a700 1 --2- 172.21.15.186:0/1121148910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea4102a10 0x7f0ea4102e70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:46.997 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.929+0000 7f0eab26a700 1 -- 172.21.15.186:0/1121148910 >> 172.21.15.186:0/1121148910 conn(0x7f0ea40fcdf0 msgr2=0x7f0ea40ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:15:46.997 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.929+0000 7f0eab26a700 1 -- 172.21.15.186:0/1121148910 shutdown_connections 2024-01-31T05:15:46.997 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.929+0000 7f0eab26a700 1 -- 172.21.15.186:0/1121148910 wait complete. 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0eab26a700 1 Processor -- start 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0eab26a700 1 -- start start 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0eab26a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea4101810 0x7f0ea4197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0eab26a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea4102a10 0x7f0ea4197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0eab26a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea41033b0 0x7f0ea419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0eab26a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea419f3d0 con 0x7f0ea4101810 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0eab26a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea419f510 con 0x7f0ea41033b0 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0eab26a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea419f6a0 con 0x7f0ea4102a10 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0ea8805700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea4102a10 0x7f0ea4197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0ea8805700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea4102a10 0x7f0ea4197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:57190/0 (socket says 172.21.15.186:57190) 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0ea8805700 1 -- 172.21.15.186:0/1794283533 learned_addr learned my addr 172.21.15.186:0/1794283533 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0ea9807700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea41033b0 0x7f0ea419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:15:46.998 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0ea9006700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea4101810 0x7f0ea4197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:15:46.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0ea8805700 1 -- 172.21.15.186:0/1794283533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea41033b0 msgr2=0x7f0ea419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:15:46.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0ea8805700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea41033b0 0x7f0ea419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:46.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0ea8805700 1 -- 172.21.15.186:0/1794283533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea4101810 msgr2=0x7f0ea4197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:15:46.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0ea8805700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea4101810 0x7f0ea4197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:15:46.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.930+0000 7f0ea8805700 1 -- 172.21.15.186:0/1794283533 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea00095f0 con 0x7f0ea4102a10 2024-01-31T05:15:46.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.931+0000 7f0ea8805700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea4102a10 0x7f0ea4197dd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0000c00 tx=0x7f0ea000f7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:15:46.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.931+0000 7f0e9a7fc700 1 -- 172.21.15.186:0/1794283533 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea001d070 con 0x7f0ea4102a10 2024-01-31T05:15:46.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.931+0000 7f0eab26a700 1 -- 172.21.15.186:0/1794283533 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea419f920 con 0x7f0ea4102a10 2024-01-31T05:15:46.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.931+0000 7f0e9a7fc700 1 -- 172.21.15.186:0/1794283533 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ea0003e20 con 0x7f0ea4102a10 2024-01-31T05:15:46.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.931+0000 7f0e9a7fc700 1 -- 172.21.15.186:0/1794283533 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00173f0 con 0x7f0ea4102a10 2024-01-31T05:15:46.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.931+0000 7f0eab26a700 1 -- 172.21.15.186:0/1794283533 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea419fdb0 con 0x7f0ea4102a10 2024-01-31T05:15:46.999 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.932+0000 7f0eab26a700 1 -- 172.21.15.186:0/1794283533 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea4108de0 con 0x7f0ea4102a10 2024-01-31T05:15:47.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.933+0000 7f0e9a7fc700 1 -- 172.21.15.186:0/1794283533 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) v1 ==== 135839+0+0 (secure 0 0 0) 0x7f0ea0003960 con 0x7f0ea4102a10 2024-01-31T05:15:47.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.934+0000 7f0e9a7fc700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0e900a12e0 0x7f0e900a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:15:47.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.934+0000 7f0e9a7fc700 1 -- 172.21.15.186:0/1794283533 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f0ea00cc730 con 0x7f0ea4102a10 2024-01-31T05:15:47.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.934+0000 7f0ea9006700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0e900a12e0 0x7f0e900a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:15:47.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.935+0000 7f0ea9006700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0e900a12e0 0x7f0e900a37a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e94006fd0 tx=0x7f0e94008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:15:47.000 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:46.938+0000 7f0e9a7fc700 1 -- 172.21.15.186:0/1794283533 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f0ea0097490 con 0x7f0ea4102a10 2024-01-31T05:15:47.138 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:15:47.138+0000 7f0eab26a700 1 -- 172.21.15.186:0/1794283533 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ea402d5a0 con 0x7f0e900a12e0 2024-01-31T05:15:47.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:47 smithi156 ceph-mon[35720]: osdmap e14: 2 total, 2 up, 2 in 2024-01-31T05:15:47.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:47 smithi156 ceph-mon[35720]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:47.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:15:47.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:15:47.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:47.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:47 smithi186 ceph-mon[35659]: osdmap e14: 2 total, 2 up, 2 in 2024-01-31T05:15:47.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:47 smithi186 ceph-mon[35659]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:47.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:15:47.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:15:47.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:47.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:47 smithi045 ceph-mon[31585]: osdmap e14: 2 total, 2 up, 2 in 2024-01-31T05:15:47.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:47 smithi045 ceph-mon[31585]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:47.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:15:47.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:15:47.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:48.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:48 smithi045 ceph-mon[31585]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:15:48.645 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:48 smithi186 ceph-mon[35659]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:15:48.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:48 smithi156 ceph-mon[35720]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:15:49.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:49 smithi045 ceph-mon[31585]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:49.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:49 smithi156 ceph-mon[35720]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:49.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:49 smithi186 ceph-mon[35659]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:50.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:50 smithi186 ceph-mon[35659]: from='client.? 172.21.15.186:0/4176036323' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c8acbd3-748d-4107-85e4-766d8820c5ee"}]: dispatch 2024-01-31T05:15:50.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:50 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c8acbd3-748d-4107-85e4-766d8820c5ee"}]: dispatch 2024-01-31T05:15:50.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:50 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c8acbd3-748d-4107-85e4-766d8820c5ee"}]': finished 2024-01-31T05:15:50.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:50 smithi186 ceph-mon[35659]: osdmap e15: 3 total, 2 up, 3 in 2024-01-31T05:15:50.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:15:50.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:50 smithi045 ceph-mon[31585]: from='client.? 172.21.15.186:0/4176036323' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c8acbd3-748d-4107-85e4-766d8820c5ee"}]: dispatch 2024-01-31T05:15:50.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:50 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c8acbd3-748d-4107-85e4-766d8820c5ee"}]: dispatch 2024-01-31T05:15:50.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:50 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c8acbd3-748d-4107-85e4-766d8820c5ee"}]': finished 2024-01-31T05:15:50.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:50 smithi045 ceph-mon[31585]: osdmap e15: 3 total, 2 up, 3 in 2024-01-31T05:15:50.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:15:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:50 smithi156 ceph-mon[35720]: from='client.? 172.21.15.186:0/4176036323' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c8acbd3-748d-4107-85e4-766d8820c5ee"}]: dispatch 2024-01-31T05:15:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:50 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c8acbd3-748d-4107-85e4-766d8820c5ee"}]: dispatch 2024-01-31T05:15:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:50 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c8acbd3-748d-4107-85e4-766d8820c5ee"}]': finished 2024-01-31T05:15:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:50 smithi156 ceph-mon[35720]: osdmap e15: 3 total, 2 up, 3 in 2024-01-31T05:15:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:15:51.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:51 smithi045 ceph-mon[31585]: from='client.? 172.21.15.186:0/4100477041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:15:51.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:51 smithi045 ceph-mon[31585]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:51.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:51 smithi156 ceph-mon[35720]: from='client.? 172.21.15.186:0/4100477041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:15:51.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:51 smithi156 ceph-mon[35720]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:51.844 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:51 smithi186 ceph-mon[35659]: from='client.? 172.21.15.186:0/4100477041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:15:51.844 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:51 smithi186 ceph-mon[35659]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:53.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:53 smithi045 ceph-mon[31585]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:53.677 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:53 smithi186 ceph-mon[35659]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:53 smithi156 ceph-mon[35720]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:55.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:55 smithi186 ceph-mon[35659]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:55.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:55 smithi045 ceph-mon[31585]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:55 smithi156 ceph-mon[35720]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:57.835 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:57 smithi186 ceph-mon[35659]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:57.835 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-01-31T05:15:57.835 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:57.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:57 smithi045 ceph-mon[31585]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:57.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-01-31T05:15:57.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:57.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:57 smithi156 ceph-mon[35720]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:57.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-01-31T05:15:57.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:15:58.820 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:58 smithi186 ceph-mon[35659]: Deploying daemon osd.2 on smithi186 2024-01-31T05:15:58.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:58 smithi045 ceph-mon[31585]: Deploying daemon osd.2 on smithi186 2024-01-31T05:15:58.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:58 smithi156 ceph-mon[35720]: Deploying daemon osd.2 on smithi186 2024-01-31T05:15:59.860 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:15:59 smithi186 ceph-mon[35659]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:59.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:15:59 smithi045 ceph-mon[31585]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:15:59.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:15:59 smithi156 ceph-mon[35720]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:01.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:01 smithi045 ceph-mon[31585]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:01.901 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:01 smithi186 ceph-mon[35659]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:01.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:01 smithi156 ceph-mon[35720]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:03.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:03 smithi045 ceph-mon[31585]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:03.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:03 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:03.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:03 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:03.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:03 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:03.953 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:03 smithi186 ceph-mon[35659]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:03.953 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:03 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:03.953 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:03 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:03.953 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:03 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:03 smithi156 ceph-mon[35720]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:03 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:03 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:03 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:05.838 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:05 smithi186 ceph-mon[35659]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:05.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:05 smithi045 ceph-mon[31585]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:05.905 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 2 on host 'smithi186' 2024-01-31T05:16:05.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.905+0000 7f0e9a7fc700 1 -- 172.21.15.186:0/1794283533 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0ea402d5a0 con 0x7f0e900a12e0 2024-01-31T05:16:05.907 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.908+0000 7f0eab26a700 1 -- 172.21.15.186:0/1794283533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0e900a12e0 msgr2=0x7f0e900a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:05.907 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.908+0000 7f0eab26a700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0e900a12e0 0x7f0e900a37a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e94006fd0 tx=0x7f0e94008040 comp rx=0 tx=0).stop 2024-01-31T05:16:05.907 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.908+0000 7f0eab26a700 1 -- 172.21.15.186:0/1794283533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea4102a10 msgr2=0x7f0ea4197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:05.907 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.908+0000 7f0eab26a700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea4102a10 0x7f0ea4197dd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0000c00 tx=0x7f0ea000f7b0 comp rx=0 tx=0).stop 2024-01-31T05:16:05.907 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.908+0000 7f0eab26a700 1 -- 172.21.15.186:0/1794283533 shutdown_connections 2024-01-31T05:16:05.907 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.908+0000 7f0eab26a700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea41033b0 0x7f0ea419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:05.907 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.908+0000 7f0eab26a700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0e900a12e0 0x7f0e900a37a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:05.907 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.908+0000 7f0eab26a700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea4101810 0x7f0ea4197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:05.908 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.908+0000 7f0eab26a700 1 --2- 172.21.15.186:0/1794283533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea4102a10 0x7f0ea4197dd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:05.908 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.908+0000 7f0eab26a700 1 -- 172.21.15.186:0/1794283533 >> 172.21.15.186:0/1794283533 conn(0x7f0ea40fcdf0 msgr2=0x7f0ea4105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:16:05.908 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.908+0000 7f0eab26a700 1 -- 172.21.15.186:0/1794283533 shutdown_connections 2024-01-31T05:16:05.908 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:05.908+0000 7f0eab26a700 1 -- 172.21.15.186:0/1794283533 wait complete. 2024-01-31T05:16:05.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:05 smithi156 ceph-mon[35720]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:06.476 DEBUG:teuthology.orchestra.run.smithi186:osd.2> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.2.service 2024-01-31T05:16:06.478 INFO:tasks.cephadm:Deploying osd.3 on smithi045 with /dev/vg_nvme/lv_3... 2024-01-31T05:16:06.478 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-01-31T05:16:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:16:06.787 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:16:06 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2[40830]: 2024-01-31T05:16:06.543+0000 7f5377d6a7c0 -1 osd.2 0 log_to_monitors true 2024-01-31T05:16:07.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:06 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:07.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:06 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:06 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:06 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:06 smithi186 ceph-mon[35659]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:06 smithi186 ceph-mon[35659]: from='osd.2 [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-31T05:16:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:06 smithi186 ceph-mon[35659]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-31T05:16:07.141 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:06 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:07.141 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:06 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:07.141 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:06 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:07.141 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:06 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:07.141 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:06 smithi045 ceph-mon[31585]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:07.141 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:06 smithi045 ceph-mon[31585]: from='osd.2 [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-31T05:16:07.141 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:06 smithi045 ceph-mon[31585]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-31T05:16:07.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:06 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:07.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:06 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:07.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:06 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:07.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:06 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:07.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:06 smithi156 ceph-mon[35720]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:07.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:06 smithi156 ceph-mon[35720]: from='osd.2 [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-31T05:16:07.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:06 smithi156 ceph-mon[35720]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-31T05:16:07.992 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:07 smithi045 ceph-mon[31585]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-01-31T05:16:07.992 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:07 smithi045 ceph-mon[31585]: osdmap e16: 3 total, 2 up, 3 in 2024-01-31T05:16:07.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:07.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:07 smithi045 ceph-mon[31585]: from='osd.2 [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:16:07.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:07 smithi045 ceph-mon[31585]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:16:08.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:07 smithi186 ceph-mon[35659]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-01-31T05:16:08.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:07 smithi186 ceph-mon[35659]: osdmap e16: 3 total, 2 up, 3 in 2024-01-31T05:16:08.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:08.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:07 smithi186 ceph-mon[35659]: from='osd.2 [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:16:08.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:07 smithi186 ceph-mon[35659]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:16:08.092 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:16:07 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2[40830]: 2024-01-31T05:16:07.954+0000 7f536cbcf700 -1 osd.2 0 waiting for initial osdmap 2024-01-31T05:16:08.092 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:16:07 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2[40830]: 2024-01-31T05:16:07.958+0000 7f53691ca700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:16:08.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:07 smithi156 ceph-mon[35720]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-01-31T05:16:08.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:07 smithi156 ceph-mon[35720]: osdmap e16: 3 total, 2 up, 3 in 2024-01-31T05:16:08.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:08.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:07 smithi156 ceph-mon[35720]: from='osd.2 [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:16:08.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:07 smithi156 ceph-mon[35720]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:16:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-01-31T05:16:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-01-31T05:16:08.539 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-01-31T05:16:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-01-31T05:16:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00873169 s, 1.2 GB/s 2024-01-31T05:16:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-01-31T05:16:09.136 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:08 smithi045 ceph-mon[31585]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-01-31T05:16:09.136 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:08 smithi045 ceph-mon[31585]: osdmap e17: 3 total, 2 up, 3 in 2024-01-31T05:16:09.136 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:08 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:09.136 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:08 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:09.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:08 smithi045 ceph-mon[31585]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:09.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2024-01-31T05:16:09.170 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:08 smithi186 ceph-mon[35659]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-01-31T05:16:09.170 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:08 smithi186 ceph-mon[35659]: osdmap e17: 3 total, 2 up, 3 in 2024-01-31T05:16:09.170 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:08 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:09.170 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:08 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:09.170 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:08 smithi186 ceph-mon[35659]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:08 smithi156 ceph-mon[35720]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-01-31T05:16:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:08 smithi156 ceph-mon[35720]: osdmap e17: 3 total, 2 up, 3 in 2024-01-31T05:16:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:08 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:08 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:08 smithi156 ceph-mon[35720]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-31T05:16:09.416 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:16:10.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: purged_snaps scrub starts 2024-01-31T05:16:10.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: purged_snaps scrub ok 2024-01-31T05:16:10.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:10.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: osd.2 [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396] boot 2024-01-31T05:16:10.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: osdmap e18: 3 total, 3 up, 3 in 2024-01-31T05:16:10.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:10.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: Detected new or changed devices on smithi186 2024-01-31T05:16:10.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:10.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi186 to 17080M 2024-01-31T05:16:10.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:10.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:10.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:10.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:10.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:10.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: purged_snaps scrub starts 2024-01-31T05:16:10.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: purged_snaps scrub ok 2024-01-31T05:16:10.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:10.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: osd.2 [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396] boot 2024-01-31T05:16:10.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: osdmap e18: 3 total, 3 up, 3 in 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: Detected new or changed devices on smithi186 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi186 to 17080M 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:10.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: purged_snaps scrub starts 2024-01-31T05:16:10.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: purged_snaps scrub ok 2024-01-31T05:16:10.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:10.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: osd.2 [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396] boot 2024-01-31T05:16:10.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: osdmap e18: 3 total, 3 up, 3 in 2024-01-31T05:16:10.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:16:10.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: Detected new or changed devices on smithi186 2024-01-31T05:16:10.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:10.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi186 to 17080M 2024-01-31T05:16:10.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:10.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:10.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:10.344 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:10.344 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:10.344 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:10.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.990+0000 7f47ca948700 1 -- 172.21.15.45:0/183081411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c41022b0 msgr2=0x7f47c4108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:10.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.990+0000 7f47ca948700 1 --2- 172.21.15.45:0/183081411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c41022b0 0x7f47c4108b60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00b0d0 tx=0x7f47bc00b3e0 comp rx=0 tx=0).stop 2024-01-31T05:16:10.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.990+0000 7f47ca948700 1 -- 172.21.15.45:0/183081411 shutdown_connections 2024-01-31T05:16:10.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.990+0000 7f47ca948700 1 --2- 172.21.15.45:0/183081411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c41022b0 0x7f47c4108b60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.990+0000 7f47ca948700 1 --2- 172.21.15.45:0/183081411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47c4100710 0x7f47c4100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.990+0000 7f47ca948700 1 --2- 172.21.15.45:0/183081411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c4101910 0x7f47c4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.990+0000 7f47ca948700 1 -- 172.21.15.45:0/183081411 >> 172.21.15.45:0/183081411 conn(0x7f47c40fbcd0 msgr2=0x7f47c40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:16:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.990+0000 7f47ca948700 1 -- 172.21.15.45:0/183081411 shutdown_connections 2024-01-31T05:16:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.990+0000 7f47ca948700 1 -- 172.21.15.45:0/183081411 wait complete. 2024-01-31T05:16:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.991+0000 7f47ca948700 1 Processor -- start 2024-01-31T05:16:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.991+0000 7f47ca948700 1 -- start start 2024-01-31T05:16:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.991+0000 7f47ca948700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c4100710 0x7f47c406aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:16:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.991+0000 7f47ca948700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47c4101910 0x7f47c419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:16:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.991+0000 7f47ca948700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c41022b0 0x7f47c419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:16:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.991+0000 7f47ca948700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c419f2b0 con 0x7f47c4100710 2024-01-31T05:16:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.991+0000 7f47ca948700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c419f420 con 0x7f47c4101910 2024-01-31T05:16:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.991+0000 7f47ca948700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c419f5e0 con 0x7f47c41022b0 2024-01-31T05:16:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.991+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c4100710 0x7f47c406aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:16:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.991+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c4100710 0x7f47c406aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48200/0 (socket says 172.21.15.45:48200) 2024-01-31T05:16:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.991+0000 7f47c3fff700 1 -- 172.21.15.45:0/1602298550 learned_addr learned my addr 172.21.15.45:0/1602298550 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:16:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c37fe700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47c4101910 0x7f47c419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:16:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c8ee5700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c41022b0 0x7f47c419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:16:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c3fff700 1 -- 172.21.15.45:0/1602298550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c41022b0 msgr2=0x7f47c419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c3fff700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c41022b0 0x7f47c419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c3fff700 1 -- 172.21.15.45:0/1602298550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47c4101910 msgr2=0x7f47c419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c3fff700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47c4101910 0x7f47c419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c3fff700 1 -- 172.21.15.45:0/1602298550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc009d00 con 0x7f47c4100710 2024-01-31T05:16:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c37fe700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47c4101910 0x7f47c419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:16:10.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c8ee5700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c41022b0 0x7f47c419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:16:10.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c3fff700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c4100710 0x7f47c406aa60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47b400c970 tx=0x7f47b400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:16:10.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c17fa700 1 -- 172.21.15.45:0/1602298550 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4004220 con 0x7f47c4100710 2024-01-31T05:16:10.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c17fa700 1 -- 172.21.15.45:0/1602298550 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47b4007d60 con 0x7f47c4100710 2024-01-31T05:16:10.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47ca948700 1 -- 172.21.15.45:0/1602298550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c419f8c0 con 0x7f47c4100710 2024-01-31T05:16:10.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47c17fa700 1 -- 172.21.15.45:0/1602298550 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4003d00 con 0x7f47c4100710 2024-01-31T05:16:10.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.992+0000 7f47ca948700 1 -- 172.21.15.45:0/1602298550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c419fdc0 con 0x7f47c4100710 2024-01-31T05:16:10.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.993+0000 7f47ca948700 1 -- 172.21.15.45:0/1602298550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c4107ad0 con 0x7f47c4100710 2024-01-31T05:16:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.994+0000 7f47c17fa700 1 -- 172.21.15.45:0/1602298550 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) v1 ==== 135839+0+0 (secure 0 0 0) 0x7f47b4004380 con 0x7f47c4100710 2024-01-31T05:16:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.995+0000 7f47c17fa700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f47ac0a12e0 0x7f47ac0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:16:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.995+0000 7f47c37fe700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f47ac0a12e0 0x7f47ac0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:16:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.995+0000 7f47c17fa700 1 -- 172.21.15.45:0/1602298550 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f47b40cb440 con 0x7f47c4100710 2024-01-31T05:16:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.996+0000 7f47c37fe700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f47ac0a12e0 0x7f47ac0a37a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f47b8005fd0 tx=0x7f47b8005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:16:10.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:10.999+0000 7f47c17fa700 1 -- 172.21.15.45:0/1602298550 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f47b4095ef0 con 0x7f47c4100710 2024-01-31T05:16:11.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:11.208+0000 7f47ca948700 1 -- 172.21.15.45:0/1602298550 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f47c402d5a0 con 0x7f47ac0a12e0 2024-01-31T05:16:11.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:11 smithi045 ceph-mon[31585]: osdmap e19: 3 total, 3 up, 3 in 2024-01-31T05:16:11.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:11 smithi045 ceph-mon[31585]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:11.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-01-31T05:16:11.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:16:11.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:16:11.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:11 smithi156 ceph-mon[35720]: osdmap e19: 3 total, 3 up, 3 in 2024-01-31T05:16:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:11 smithi156 ceph-mon[35720]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-01-31T05:16:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:16:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:16:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:11.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:11 smithi186 ceph-mon[35659]: osdmap e19: 3 total, 3 up, 3 in 2024-01-31T05:16:11.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:11 smithi186 ceph-mon[35659]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:11.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-01-31T05:16:11.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:16:11.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:16:11.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:12.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:12 smithi045 ceph-mon[31585]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:16:12.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:12 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-01-31T05:16:12.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:12 smithi045 ceph-mon[31585]: osdmap e20: 3 total, 3 up, 3 in 2024-01-31T05:16:12.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:12 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-01-31T05:16:12.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:12 smithi156 ceph-mon[35720]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:16:12.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:12 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-01-31T05:16:12.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:12 smithi156 ceph-mon[35720]: osdmap e20: 3 total, 3 up, 3 in 2024-01-31T05:16:12.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:12 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-01-31T05:16:12.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:12 smithi186 ceph-mon[35659]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:16:12.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:12 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-01-31T05:16:12.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:12 smithi186 ceph-mon[35659]: osdmap e20: 3 total, 3 up, 3 in 2024-01-31T05:16:12.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:12 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-01-31T05:16:12.958 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:16:12 smithi045 sudo[51094]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-31T05:16:12.958 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:16:12 smithi045 sudo[51094]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-31T05:16:12.958 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:16:12 smithi045 sudo[51094]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-31T05:16:12.958 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:16:12 smithi045 sudo[51094]: pam_unix(sudo:session): session closed for user root 2024-01-31T05:16:13.392 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:16:13 smithi045 sudo[51115]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-31T05:16:13.393 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:16:13 smithi045 sudo[51115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-31T05:16:13.393 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:16:13 smithi045 sudo[51115]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-31T05:16:13.393 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:16:13 smithi045 sudo[51115]: pam_unix(sudo:session): session closed for user root 2024-01-31T05:16:13.561 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-01-31T05:16:13.561 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:13 smithi156 ceph-mon[35720]: osdmap e21: 3 total, 3 up, 3 in 2024-01-31T05:16:13.561 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:13 smithi156 ceph-mon[35720]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:13.561 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:13.561 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:16:13 smithi156 sudo[44496]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-31T05:16:13.562 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:16:13 smithi156 sudo[44496]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-31T05:16:13.562 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:16:13 smithi156 sudo[44496]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-31T05:16:13.562 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:16:13 smithi156 sudo[44496]: pam_unix(sudo:session): session closed for user root 2024-01-31T05:16:13.827 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-01-31T05:16:13.827 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:13 smithi186 ceph-mon[35659]: osdmap e21: 3 total, 3 up, 3 in 2024-01-31T05:16:13.827 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:13 smithi186 ceph-mon[35659]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:13.827 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:13.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-01-31T05:16:13.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:13 smithi045 ceph-mon[31585]: osdmap e21: 3 total, 3 up, 3 in 2024-01-31T05:16:13.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:13 smithi045 ceph-mon[31585]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:13.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:13.976 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:16:13 smithi156 sudo[44499]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-31T05:16:13.976 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:16:13 smithi156 sudo[44499]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-31T05:16:13.976 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:16:13 smithi156 sudo[44499]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-31T05:16:13.976 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:16:13 smithi156 sudo[44499]: pam_unix(sudo:session): session closed for user root 2024-01-31T05:16:14.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:14.075+0000 7f47c17fa700 1 -- 172.21.15.45:0/1602298550 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f47b40962c0 con 0x7f47c4100710 2024-01-31T05:16:14.080 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:16:13 smithi186 sudo[44756]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-31T05:16:14.080 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:16:13 smithi186 sudo[44756]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-31T05:16:14.080 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:16:13 smithi186 sudo[44756]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-31T05:16:14.080 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:16:13 smithi186 sudo[44756]: pam_unix(sudo:session): session closed for user root 2024-01-31T05:16:14.341 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:16:14 smithi186 sudo[44759]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-31T05:16:14.341 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:16:14 smithi186 sudo[44759]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-31T05:16:14.341 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:16:14 smithi186 sudo[44759]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-31T05:16:14.342 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:16:14 smithi186 sudo[44759]: pam_unix(sudo:session): session closed for user root 2024-01-31T05:16:14.623 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 sudo[51198]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-31T05:16:14.624 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 sudo[51198]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-31T05:16:14.624 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 sudo[51198]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-31T05:16:14.624 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 sudo[51198]: pam_unix(sudo:session): session closed for user root 2024-01-31T05:16:14.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 ceph-mon[35659]: osdmap e22: 3 total, 3 up, 3 in 2024-01-31T05:16:14.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:14.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/286756702' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d"}]: dispatch 2024-01-31T05:16:14.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/286756702' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d"}]': finished 2024-01-31T05:16:14.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 ceph-mon[35659]: osdmap e23: 4 total, 3 up, 4 in 2024-01-31T05:16:14.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:14.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 ceph-mon[35659]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-01-31T05:16:14.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:14.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 ceph-mon[35659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-31T05:16:14.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 sudo[44762]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-31T05:16:14.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 sudo[44762]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-31T05:16:14.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 sudo[44762]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-31T05:16:14.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:14 smithi186 sudo[44762]: pam_unix(sudo:session): session closed for user root 2024-01-31T05:16:14.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 sudo[51250]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-31T05:16:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 sudo[51250]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-31T05:16:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 sudo[51250]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-31T05:16:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 sudo[51250]: pam_unix(sudo:session): session closed for user root 2024-01-31T05:16:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 ceph-mon[31585]: osdmap e22: 3 total, 3 up, 3 in 2024-01-31T05:16:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/286756702' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d"}]: dispatch 2024-01-31T05:16:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/286756702' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d"}]': finished 2024-01-31T05:16:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 ceph-mon[31585]: osdmap e23: 4 total, 3 up, 4 in 2024-01-31T05:16:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 ceph-mon[31585]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-01-31T05:16:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:14 smithi045 ceph-mon[31585]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-31T05:16:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:14 smithi156 ceph-mon[35720]: osdmap e22: 3 total, 3 up, 3 in 2024-01-31T05:16:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:14 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:14 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/286756702' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d"}]: dispatch 2024-01-31T05:16:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:14 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/286756702' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d"}]': finished 2024-01-31T05:16:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:14 smithi156 ceph-mon[35720]: osdmap e23: 4 total, 3 up, 4 in 2024-01-31T05:16:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:14 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:14.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:14 smithi156 ceph-mon[35720]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-01-31T05:16:14.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:14 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:14.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:14 smithi156 ceph-mon[35720]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-31T05:16:15.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 sudo[44765]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-31T05:16:15.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 sudo[44765]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-31T05:16:15.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 sudo[44765]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-31T05:16:15.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 sudo[44765]: pam_unix(sudo:session): session closed for user root 2024-01-31T05:16:15.609 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 sudo[44502]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-31T05:16:15.609 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 sudo[44502]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-31T05:16:15.610 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 sudo[44502]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-31T05:16:15.610 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 sudo[44502]: pam_unix(sudo:session): session closed for user root 2024-01-31T05:16:15.610 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 sudo[44505]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-31T05:16:15.610 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 sudo[44505]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-31T05:16:15.610 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 sudo[44505]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-31T05:16:15.610 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 sudo[44505]: pam_unix(sudo:session): session closed for user root 2024-01-31T05:16:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2975495289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:16:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-31T05:16:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:16:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:16:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:16:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-31T05:16:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:16:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:16:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:16:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-31T05:16:15.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-31T05:16:15.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:16:15.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:16:15.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:16:15.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:15 smithi045 ceph-mon[31585]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-31T05:16:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2975495289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:16:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-31T05:16:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:16:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:16:15.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:16:15.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-31T05:16:15.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:16:15.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:16:15.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:16:15.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-31T05:16:15.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-31T05:16:15.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:16:15.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:16:15.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:16:15.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:15 smithi156 ceph-mon[35720]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-31T05:16:16.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2975495289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:16:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-31T05:16:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:16:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:16:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:16:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-31T05:16:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:16:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:16:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:16:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-31T05:16:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-31T05:16:16.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:16:16.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:16:16.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:16:16.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:15 smithi186 ceph-mon[35659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-31T05:16:17.724 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:17 smithi045 ceph-mon[31585]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:17 smithi156 ceph-mon[35720]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:18.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:17 smithi186 ceph-mon[35659]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:19.848 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:19 smithi045 ceph-mon[31585]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:19.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:19 smithi156 ceph-mon[35720]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:20.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:19 smithi186 ceph-mon[35659]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:21.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:21 smithi045 ceph-mon[31585]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:21.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-01-31T05:16:21.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:21.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:21 smithi156 ceph-mon[35720]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:21.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-01-31T05:16:21.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:22.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:21 smithi186 ceph-mon[35659]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-01-31T05:16:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:22.811 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:22 smithi045 ceph-mon[31585]: Deploying daemon osd.3 on smithi045 2024-01-31T05:16:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:22 smithi156 ceph-mon[35720]: Deploying daemon osd.3 on smithi045 2024-01-31T05:16:23.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:22 smithi186 ceph-mon[35659]: Deploying daemon osd.3 on smithi045 2024-01-31T05:16:23.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:23 smithi045 ceph-mon[31585]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:23.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:23 smithi156 ceph-mon[35720]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:24.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:23 smithi186 ceph-mon[35659]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:25.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:25 smithi045 ceph-mon[31585]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:25.913 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:25 smithi186 ceph-mon[35659]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:25 smithi156 ceph-mon[35720]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:27.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:27 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:27.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:27 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:27.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:27 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:27.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:27 smithi045 ceph-mon[31585]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:27.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:27 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:27.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:27 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:27.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:27 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:27.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:27 smithi156 ceph-mon[35720]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:27.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:27 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:27.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:27 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:27.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:27 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:27.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:27 smithi186 ceph-mon[35659]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:29.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:29 smithi045 ceph-mon[31585]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:29.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:29 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:29.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:29 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:29.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:29 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:29.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:29 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:29.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:29 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:29.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:29 smithi156 ceph-mon[35720]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:29.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:29 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:29.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:29 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:29.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:29 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:29.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:29 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:29.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:29 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:30.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.018+0000 7f47c17fa700 1 -- 172.21.15.45:0/1602298550 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f47c402d5a0 con 0x7f47ac0a12e0 2024-01-31T05:16:30.017 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 3 on host 'smithi045' 2024-01-31T05:16:30.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.020+0000 7f47ca948700 1 -- 172.21.15.45:0/1602298550 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f47ac0a12e0 msgr2=0x7f47ac0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:30.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.020+0000 7f47ca948700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f47ac0a12e0 0x7f47ac0a37a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f47b8005fd0 tx=0x7f47b8005dc0 comp rx=0 tx=0).stop 2024-01-31T05:16:30.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.020+0000 7f47ca948700 1 -- 172.21.15.45:0/1602298550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c4100710 msgr2=0x7f47c406aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:30.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.020+0000 7f47ca948700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c4100710 0x7f47c406aa60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47b400c970 tx=0x7f47b400cd30 comp rx=0 tx=0).stop 2024-01-31T05:16:30.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.020+0000 7f47ca948700 1 -- 172.21.15.45:0/1602298550 shutdown_connections 2024-01-31T05:16:30.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.020+0000 7f47ca948700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c41022b0 0x7f47c419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:30.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.020+0000 7f47ca948700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f47ac0a12e0 0x7f47ac0a37a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:30.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.020+0000 7f47ca948700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c4100710 0x7f47c406aa60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:30.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.020+0000 7f47ca948700 1 --2- 172.21.15.45:0/1602298550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47c4101910 0x7f47c419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:30.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.020+0000 7f47ca948700 1 -- 172.21.15.45:0/1602298550 >> 172.21.15.45:0/1602298550 conn(0x7f47c40fbcd0 msgr2=0x7f47c4104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:16:30.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.020+0000 7f47ca948700 1 -- 172.21.15.45:0/1602298550 shutdown_connections 2024-01-31T05:16:30.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:16:30.020+0000 7f47ca948700 1 -- 172.21.15.45:0/1602298550 wait complete. 2024-01-31T05:16:30.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:29 smithi186 ceph-mon[35659]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:30.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:29 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:30.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:29 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:30.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:29 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:30.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:29 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:30.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:29 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:30.616 DEBUG:teuthology.orchestra.run.smithi045:osd.3> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.3.service 2024-01-31T05:16:30.618 INFO:tasks.cephadm:Deploying osd.4 on smithi156 with /dev/vg_nvme/lv_3... 2024-01-31T05:16:30.618 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-01-31T05:16:30.861 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.b/config 2024-01-31T05:16:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:31 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:31 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:31 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:31 smithi045 ceph-mon[31585]: from='osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-01-31T05:16:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:31 smithi045 ceph-mon[31585]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:31.309 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:31 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:31.309 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:31 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:31.309 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:31 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:31.309 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:31 smithi156 ceph-mon[35720]: from='osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-01-31T05:16:31.309 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:31 smithi156 ceph-mon[35720]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:31.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:31 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:31.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:31 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:31.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:31 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:31.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:31 smithi186 ceph-mon[35659]: from='osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-01-31T05:16:31.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:31 smithi186 ceph-mon[35659]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:32.167 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:32 smithi156 ceph-mon[35720]: from='osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-01-31T05:16:32.167 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:32 smithi156 ceph-mon[35720]: osdmap e24: 4 total, 3 up, 4 in 2024-01-31T05:16:32.167 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:32 smithi156 ceph-mon[35720]: from='osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-01-31T05:16:32.167 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:32 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:32.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:32 smithi186 ceph-mon[35659]: from='osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-01-31T05:16:32.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:32 smithi186 ceph-mon[35659]: osdmap e24: 4 total, 3 up, 4 in 2024-01-31T05:16:32.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:32 smithi186 ceph-mon[35659]: from='osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-01-31T05:16:32.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:32 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:32.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:32 smithi045 ceph-mon[31585]: from='osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-01-31T05:16:32.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:32 smithi045 ceph-mon[31585]: osdmap e24: 4 total, 3 up, 4 in 2024-01-31T05:16:32.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:32 smithi045 ceph-mon[31585]: from='osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-01-31T05:16:32.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:32 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:32.393 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:16:32 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3[53235]: 2024-01-31T05:16:32.020+0000 7f26d9e02700 -1 osd.3 0 waiting for initial osdmap 2024-01-31T05:16:32.393 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:16:32 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3[53235]: 2024-01-31T05:16:32.027+0000 7f26d63fd700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:16:32.700 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-01-31T05:16:32.700 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-01-31T05:16:32.700 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-01-31T05:16:32.700 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-01-31T05:16:32.700 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00849349 s, 1.2 GB/s 2024-01-31T05:16:32.700 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-01-31T05:16:33.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:33 smithi045 ceph-mon[31585]: from='osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-01-31T05:16:33.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:33 smithi045 ceph-mon[31585]: osdmap e25: 4 total, 3 up, 4 in 2024-01-31T05:16:33.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:33 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:33.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:33 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:33.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:33 smithi045 ceph-mon[31585]: osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645] boot 2024-01-31T05:16:33.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:33 smithi045 ceph-mon[31585]: osdmap e26: 4 total, 4 up, 4 in 2024-01-31T05:16:33.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:33 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:33.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:33 smithi045 ceph-mon[31585]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:33.302 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:33 smithi156 ceph-mon[35720]: from='osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-01-31T05:16:33.302 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:33 smithi156 ceph-mon[35720]: osdmap e25: 4 total, 3 up, 4 in 2024-01-31T05:16:33.302 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:33 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:33.302 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:33 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:33.302 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:33 smithi156 ceph-mon[35720]: osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645] boot 2024-01-31T05:16:33.302 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:33 smithi156 ceph-mon[35720]: osdmap e26: 4 total, 4 up, 4 in 2024-01-31T05:16:33.302 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:33 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:33.302 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:33 smithi156 ceph-mon[35720]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:33.328 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_3 2024-01-31T05:16:33.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:33 smithi186 ceph-mon[35659]: from='osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-01-31T05:16:33.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:33 smithi186 ceph-mon[35659]: osdmap e25: 4 total, 3 up, 4 in 2024-01-31T05:16:33.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:33 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:33.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:33 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:33.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:33 smithi186 ceph-mon[35659]: osd.3 [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645] boot 2024-01-31T05:16:33.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:33 smithi186 ceph-mon[35659]: osdmap e26: 4 total, 4 up, 4 in 2024-01-31T05:16:33.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:33 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:16:33.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:33 smithi186 ceph-mon[35659]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-01-31T05:16:33.572 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.b/config 2024-01-31T05:16:34.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: purged_snaps scrub starts 2024-01-31T05:16:34.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: purged_snaps scrub ok 2024-01-31T05:16:34.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: osdmap e27: 4 total, 4 up, 4 in 2024-01-31T05:16:34.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: Detected new or changed devices on smithi045 2024-01-31T05:16:34.216 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:34.217 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:34.217 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:34.217 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:34.217 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi045 to 8546M 2024-01-31T05:16:34.217 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:34.217 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:34.217 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:34.217 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:34 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:34.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: purged_snaps scrub starts 2024-01-31T05:16:34.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: purged_snaps scrub ok 2024-01-31T05:16:34.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: osdmap e27: 4 total, 4 up, 4 in 2024-01-31T05:16:34.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: Detected new or changed devices on smithi045 2024-01-31T05:16:34.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:34.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:34.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:34.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:34.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi045 to 8546M 2024-01-31T05:16:34.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:34.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:34.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:34.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:34 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: purged_snaps scrub starts 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: purged_snaps scrub ok 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: osdmap e27: 4 total, 4 up, 4 in 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: Detected new or changed devices on smithi045 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi045 to 8546M 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:34.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:34 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:34.981 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.979+0000 7f5335ea2700 1 -- 172.21.15.156:0/3741626949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5330102300 msgr2=0x7f5330108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:34.981 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.979+0000 7f5335ea2700 1 --2- 172.21.15.156:0/3741626949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5330102300 0x7f5330108bb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f532800b0d0 tx=0x7f532800b3e0 comp rx=0 tx=0).stop 2024-01-31T05:16:34.981 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.979+0000 7f5335ea2700 1 -- 172.21.15.156:0/3741626949 shutdown_connections 2024-01-31T05:16:34.981 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.979+0000 7f5335ea2700 1 --2- 172.21.15.156:0/3741626949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5330102300 0x7f5330108bb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:34.981 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.979+0000 7f5335ea2700 1 --2- 172.21.15.156:0/3741626949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5330100760 0x7f5330100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.979+0000 7f5335ea2700 1 --2- 172.21.15.156:0/3741626949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5330101960 0x7f5330101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.979+0000 7f5335ea2700 1 -- 172.21.15.156:0/3741626949 >> 172.21.15.156:0/3741626949 conn(0x7f53300fbd00 msgr2=0x7f53300fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.979+0000 7f5335ea2700 1 -- 172.21.15.156:0/3741626949 shutdown_connections 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.979+0000 7f5335ea2700 1 -- 172.21.15.156:0/3741626949 wait complete. 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.979+0000 7f5335ea2700 1 Processor -- start 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f5335ea2700 1 -- start start 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f5335ea2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5330100760 0x7f533006aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f5335ea2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5330101960 0x7f533019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f5335ea2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5330102300 0x7f53301a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f5335ea2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53301a15a0 con 0x7f5330101960 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f5335ea2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53301a1710 con 0x7f5330102300 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f5335ea2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53301a18d0 con 0x7f5330100760 2024-01-31T05:16:34.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5330102300 0x7f53301a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5330102300 0x7f53301a0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:55796/0 (socket says 172.21.15.156:55796) 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532ffff700 1 -- 172.21.15.156:0/45323972 learned_addr learned my addr 172.21.15.156:0/45323972 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532effd700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5330101960 0x7f533019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532f7fe700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5330100760 0x7f533006aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532ffff700 1 -- 172.21.15.156:0/45323972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5330100760 msgr2=0x7f533006aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532ffff700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5330100760 0x7f533006aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532ffff700 1 -- 172.21.15.156:0/45323972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5330101960 msgr2=0x7f533019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532ffff700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5330101960 0x7f533019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532ffff700 1 -- 172.21.15.156:0/45323972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5328009d00 con 0x7f5330102300 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532ffff700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5330102300 0x7f53301a0ef0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5328007bd0 tx=0x7f5328009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532cff9700 1 -- 172.21.15.156:0/45323972 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532800e040 con 0x7f5330102300 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f532cff9700 1 -- 172.21.15.156:0/45323972 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f532801dd50 con 0x7f5330102300 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.980+0000 7f5335ea2700 1 -- 172.21.15.156:0/45323972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53301a1b50 con 0x7f5330102300 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.981+0000 7f532cff9700 1 -- 172.21.15.156:0/45323972 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53280123f0 con 0x7f5330102300 2024-01-31T05:16:34.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.981+0000 7f5335ea2700 1 -- 172.21.15.156:0/45323972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53301a2040 con 0x7f5330102300 2024-01-31T05:16:34.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.982+0000 7f5335ea2700 1 -- 172.21.15.156:0/45323972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5330107b20 con 0x7f5330102300 2024-01-31T05:16:34.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.982+0000 7f532cff9700 1 -- 172.21.15.156:0/45323972 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f532801d890 con 0x7f5330102300 2024-01-31T05:16:34.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.983+0000 7f532cff9700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f53180a1370 0x7f53180a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:16:34.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.983+0000 7f532cff9700 1 -- 172.21.15.156:0/45323972 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f53280cd640 con 0x7f5330102300 2024-01-31T05:16:34.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.983+0000 7f532f7fe700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f53180a1370 0x7f53180a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:16:34.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.984+0000 7f532f7fe700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f53180a1370 0x7f53180a3830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53301017c0 tx=0x7f5320009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:16:34.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:34.989+0000 7f532cff9700 1 -- 172.21.15.156:0/45323972 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5328097d50 con 0x7f5330102300 2024-01-31T05:16:35.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:35.188+0000 7f5335ea2700 1 -- 172.21.15.156:0/45323972 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f533002d540 con 0x7f53180a1370 2024-01-31T05:16:35.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:35 smithi156 ceph-mon[35720]: osdmap e28: 4 total, 4 up, 4 in 2024-01-31T05:16:35.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:35 smithi156 ceph-mon[35720]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:35.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:16:35.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:16:35.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:35.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:35 smithi186 ceph-mon[35659]: osdmap e28: 4 total, 4 up, 4 in 2024-01-31T05:16:35.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:35 smithi186 ceph-mon[35659]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:16:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:16:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:35.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:35 smithi045 ceph-mon[31585]: osdmap e28: 4 total, 4 up, 4 in 2024-01-31T05:16:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:35 smithi045 ceph-mon[31585]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:16:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:16:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:36.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:36 smithi045 ceph-mon[31585]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:16:36.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:36 smithi045 ceph-mon[31585]: osdmap e29: 4 total, 4 up, 4 in 2024-01-31T05:16:36.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:36 smithi156 ceph-mon[35720]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:16:36.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:36 smithi156 ceph-mon[35720]: osdmap e29: 4 total, 4 up, 4 in 2024-01-31T05:16:36.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:36 smithi186 ceph-mon[35659]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:16:36.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:36 smithi186 ceph-mon[35659]: osdmap e29: 4 total, 4 up, 4 in 2024-01-31T05:16:37.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:37 smithi186 ceph-mon[35659]: osdmap e30: 4 total, 4 up, 4 in 2024-01-31T05:16:37.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:37 smithi186 ceph-mon[35659]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-31T05:16:37.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:37 smithi156 ceph-mon[35720]: osdmap e30: 4 total, 4 up, 4 in 2024-01-31T05:16:37.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:37 smithi156 ceph-mon[35720]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-31T05:16:37.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:37 smithi045 ceph-mon[31585]: osdmap e30: 4 total, 4 up, 4 in 2024-01-31T05:16:37.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:37 smithi045 ceph-mon[31585]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-31T05:16:38.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:38 smithi156 ceph-mon[35720]: from='client.? 172.21.15.156:0/362531155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae77293-4355-4aac-87a5-220b0754e21c"}]: dispatch 2024-01-31T05:16:38.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:38 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae77293-4355-4aac-87a5-220b0754e21c"}]: dispatch 2024-01-31T05:16:38.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:38 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ae77293-4355-4aac-87a5-220b0754e21c"}]': finished 2024-01-31T05:16:38.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:38 smithi156 ceph-mon[35720]: osdmap e31: 5 total, 4 up, 5 in 2024-01-31T05:16:38.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:38 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:38.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:38 smithi186 ceph-mon[35659]: from='client.? 172.21.15.156:0/362531155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae77293-4355-4aac-87a5-220b0754e21c"}]: dispatch 2024-01-31T05:16:38.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:38 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae77293-4355-4aac-87a5-220b0754e21c"}]: dispatch 2024-01-31T05:16:38.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:38 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ae77293-4355-4aac-87a5-220b0754e21c"}]': finished 2024-01-31T05:16:38.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:38 smithi186 ceph-mon[35659]: osdmap e31: 5 total, 4 up, 5 in 2024-01-31T05:16:38.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:38 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:38.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:38 smithi045 ceph-mon[31585]: from='client.? 172.21.15.156:0/362531155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae77293-4355-4aac-87a5-220b0754e21c"}]: dispatch 2024-01-31T05:16:38.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:38 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae77293-4355-4aac-87a5-220b0754e21c"}]: dispatch 2024-01-31T05:16:38.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:38 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ae77293-4355-4aac-87a5-220b0754e21c"}]': finished 2024-01-31T05:16:38.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:38 smithi045 ceph-mon[31585]: osdmap e31: 5 total, 4 up, 5 in 2024-01-31T05:16:38.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:38 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:39.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:39 smithi156 ceph-mon[35720]: from='client.? 172.21.15.156:0/3378696603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:16:39.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:39 smithi156 ceph-mon[35720]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-01-31T05:16:39.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:39 smithi186 ceph-mon[35659]: from='client.? 172.21.15.156:0/3378696603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:16:39.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:39 smithi186 ceph-mon[35659]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-01-31T05:16:39.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:39 smithi045 ceph-mon[31585]: from='client.? 172.21.15.156:0/3378696603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:16:39.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:39 smithi045 ceph-mon[31585]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-01-31T05:16:41.833 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:41 smithi156 ceph-mon[35720]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-01-31T05:16:41.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:41 smithi045 ceph-mon[31585]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-01-31T05:16:42.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:41 smithi186 ceph-mon[35659]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-01-31T05:16:43.685 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:43 smithi156 ceph-mon[35720]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-01-31T05:16:43.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:43 smithi186 ceph-mon[35659]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-01-31T05:16:43.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:43 smithi045 ceph-mon[31585]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-01-31T05:16:45.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:45 smithi156 ceph-mon[35720]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:45.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-01-31T05:16:45.707 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:45.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:45 smithi045 ceph-mon[31585]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:45.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-01-31T05:16:45.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:46.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:45 smithi186 ceph-mon[35659]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:46.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-01-31T05:16:46.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:46.888 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:46 smithi156 ceph-mon[35720]: Deploying daemon osd.4 on smithi156 2024-01-31T05:16:46.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:46 smithi045 ceph-mon[31585]: Deploying daemon osd.4 on smithi156 2024-01-31T05:16:47.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:46 smithi186 ceph-mon[35659]: Deploying daemon osd.4 on smithi156 2024-01-31T05:16:47.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:47 smithi156 ceph-mon[35720]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:47.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:47 smithi045 ceph-mon[31585]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:48.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:47 smithi186 ceph-mon[35659]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:49.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:49 smithi045 ceph-mon[31585]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:49.904 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:49 smithi156 ceph-mon[35720]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:50.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:49 smithi186 ceph-mon[35659]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:51.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:51 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:51.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:51 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:51.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:51 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:51.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:51 smithi156 ceph-mon[35720]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:51.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:51 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:51.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:51 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:51.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:51 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:51.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:51 smithi186 ceph-mon[35659]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:51.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:51 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:51.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:51 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:51.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:51 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:51.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:51 smithi045 ceph-mon[31585]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:53.624 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:53 smithi156 ceph-mon[35720]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:53.624 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:53.625 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:53.625 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:53.625 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:53.625 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:53.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:53 smithi186 ceph-mon[35659]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:53.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:53.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:53.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:53.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:53.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:53.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:53 smithi045 ceph-mon[31585]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:53.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:53.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:53.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:53.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:53.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:54.002 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.003+0000 7f532cff9700 1 -- 172.21.15.156:0/45323972 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f533002d540 con 0x7f53180a1370 2024-01-31T05:16:54.002 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 4 on host 'smithi156' 2024-01-31T05:16:54.004 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.005+0000 7f5335ea2700 1 -- 172.21.15.156:0/45323972 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f53180a1370 msgr2=0x7f53180a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:54.004 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.005+0000 7f5335ea2700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f53180a1370 0x7f53180a3830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53301017c0 tx=0x7f5320009190 comp rx=0 tx=0).stop 2024-01-31T05:16:54.004 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.005+0000 7f5335ea2700 1 -- 172.21.15.156:0/45323972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5330102300 msgr2=0x7f53301a0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:54.004 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.005+0000 7f5335ea2700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5330102300 0x7f53301a0ef0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5328007bd0 tx=0x7f5328009440 comp rx=0 tx=0).stop 2024-01-31T05:16:54.004 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.005+0000 7f5335ea2700 1 -- 172.21.15.156:0/45323972 shutdown_connections 2024-01-31T05:16:54.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.005+0000 7f5335ea2700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5330102300 0x7f53301a0ef0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:54.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.005+0000 7f5335ea2700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5330100760 0x7f533006aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:54.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.005+0000 7f5335ea2700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f53180a1370 0x7f53180a3830 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:54.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.005+0000 7f5335ea2700 1 --2- 172.21.15.156:0/45323972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5330101960 0x7f533019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:54.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.005+0000 7f5335ea2700 1 -- 172.21.15.156:0/45323972 >> 172.21.15.156:0/45323972 conn(0x7f53300fbd00 msgr2=0x7f5330104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:16:54.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.005+0000 7f5335ea2700 1 -- 172.21.15.156:0/45323972 shutdown_connections 2024-01-31T05:16:54.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:16:54.005+0000 7f5335ea2700 1 -- 172.21.15.156:0/45323972 wait complete. 2024-01-31T05:16:54.570 DEBUG:teuthology.orchestra.run.smithi156:osd.4> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.4.service 2024-01-31T05:16:54.572 INFO:tasks.cephadm:Deploying osd.5 on smithi186 with /dev/vg_nvme/lv_3... 2024-01-31T05:16:54.572 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-01-31T05:16:54.816 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.c/config 2024-01-31T05:16:55.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:54 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:55.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:54 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:55.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:54 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:55.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:54 smithi156 ceph-mon[35720]: from='osd.4 [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-31T05:16:55.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:54 smithi156 ceph-mon[35720]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-31T05:16:55.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:54 smithi156 ceph-mon[35720]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:55.245 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:54 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:55.245 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:54 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:55.245 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:54 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:55.245 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:54 smithi186 ceph-mon[35659]: from='osd.4 [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-31T05:16:55.245 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:54 smithi186 ceph-mon[35659]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-31T05:16:55.245 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:54 smithi186 ceph-mon[35659]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:55.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:54 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:55.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:54 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:55.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:54 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:16:55.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:54 smithi045 ceph-mon[31585]: from='osd.4 [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-31T05:16:55.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:54 smithi045 ceph-mon[31585]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-31T05:16:55.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:54 smithi045 ceph-mon[31585]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:56.073 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:55 smithi186 ceph-mon[35659]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-01-31T05:16:56.073 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:55 smithi186 ceph-mon[35659]: osdmap e32: 5 total, 4 up, 5 in 2024-01-31T05:16:56.073 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:55 smithi186 ceph-mon[35659]: from='osd.4 [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:16:56.073 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:55 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:56.073 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:55 smithi186 ceph-mon[35659]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:16:56.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:55 smithi156 ceph-mon[35720]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-01-31T05:16:56.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:55 smithi156 ceph-mon[35720]: osdmap e32: 5 total, 4 up, 5 in 2024-01-31T05:16:56.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:55 smithi156 ceph-mon[35720]: from='osd.4 [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:16:56.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:55 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:56.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:55 smithi156 ceph-mon[35720]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:16:56.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:55 smithi045 ceph-mon[31585]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-01-31T05:16:56.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:55 smithi045 ceph-mon[31585]: osdmap e32: 5 total, 4 up, 5 in 2024-01-31T05:16:56.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:55 smithi045 ceph-mon[31585]: from='osd.4 [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:16:56.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:55 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:56.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:55 smithi045 ceph-mon[31585]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:16:56.623 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-01-31T05:16:56.623 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-01-31T05:16:56.623 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-01-31T05:16:56.623 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-01-31T05:16:56.623 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114862 s, 913 MB/s 2024-01-31T05:16:56.623 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-01-31T05:16:56.968 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:16:56 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4[47345]: 2024-01-31T05:16:56.664+0000 7fa634732700 -1 osd.4 0 waiting for initial osdmap 2024-01-31T05:16:56.968 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:16:56 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4[47345]: 2024-01-31T05:16:56.670+0000 7fa62cd25700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:16:57.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:56 smithi186 ceph-mon[35659]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-01-31T05:16:57.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:56 smithi186 ceph-mon[35659]: osdmap e33: 5 total, 4 up, 5 in 2024-01-31T05:16:57.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:56 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:57.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:56 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:57.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:56 smithi186 ceph-mon[35659]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:57.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:56 smithi186 ceph-mon[35659]: from='osd.4 ' entity='osd.4' 2024-01-31T05:16:57.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:56 smithi156 ceph-mon[35720]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-01-31T05:16:57.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:56 smithi156 ceph-mon[35720]: osdmap e33: 5 total, 4 up, 5 in 2024-01-31T05:16:57.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:56 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:57.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:56 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:57.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:56 smithi156 ceph-mon[35720]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:57.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:56 smithi156 ceph-mon[35720]: from='osd.4 ' entity='osd.4' 2024-01-31T05:16:57.250 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch daemon add osd smithi186:vg_nvme/lv_3 2024-01-31T05:16:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:56 smithi045 ceph-mon[31585]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-01-31T05:16:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:56 smithi045 ceph-mon[31585]: osdmap e33: 5 total, 4 up, 5 in 2024-01-31T05:16:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:56 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:56 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:56 smithi045 ceph-mon[31585]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-01-31T05:16:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:56 smithi045 ceph-mon[31585]: from='osd.4 ' entity='osd.4' 2024-01-31T05:16:57.500 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.c/config 2024-01-31T05:16:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: purged_snaps scrub starts 2024-01-31T05:16:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: purged_snaps scrub ok 2024-01-31T05:16:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: Detected new or changed devices on smithi156 2024-01-31T05:16:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi156 to 8540M 2024-01-31T05:16:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: osd.4 [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752] boot 2024-01-31T05:16:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: osdmap e34: 5 total, 5 up, 5 in 2024-01-31T05:16:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:58.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: purged_snaps scrub starts 2024-01-31T05:16:58.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: purged_snaps scrub ok 2024-01-31T05:16:58.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:58.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: Detected new or changed devices on smithi156 2024-01-31T05:16:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi156 to 8540M 2024-01-31T05:16:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: osd.4 [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752] boot 2024-01-31T05:16:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: osdmap e34: 5 total, 5 up, 5 in 2024-01-31T05:16:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: purged_snaps scrub starts 2024-01-31T05:16:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: purged_snaps scrub ok 2024-01-31T05:16:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: Detected new or changed devices on smithi156 2024-01-31T05:16:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:16:58.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi156 to 8540M 2024-01-31T05:16:58.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:58.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:16:58.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:16:58.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:16:58.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: osd.4 [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752] boot 2024-01-31T05:16:58.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: osdmap e34: 5 total, 5 up, 5 in 2024-01-31T05:16:58.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:16:58.943 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.942+0000 7f23843e1700 1 -- 172.21.15.186:0/1691144629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c109700 msgr2=0x7f237c10bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:58.943 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.942+0000 7f23843e1700 1 --2- 172.21.15.186:0/1691144629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c109700 0x7f237c10bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f237400b0a0 tx=0x7f237400b3b0 comp rx=0 tx=0).stop 2024-01-31T05:16:58.943 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.942+0000 7f23843e1700 1 -- 172.21.15.186:0/1691144629 shutdown_connections 2024-01-31T05:16:58.943 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.942+0000 7f23843e1700 1 --2- 172.21.15.186:0/1691144629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c109700 0x7f237c10bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:58.943 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.942+0000 7f23843e1700 1 --2- 172.21.15.186:0/1691144629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f237c075790 0x7f237c073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:58.943 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.942+0000 7f23843e1700 1 --2- 172.21.15.186:0/1691144629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237c0741c0 0x7f237c074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:58.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.942+0000 7f23843e1700 1 -- 172.21.15.186:0/1691144629 >> 172.21.15.186:0/1691144629 conn(0x7f237c0fcdc0 msgr2=0x7f237c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:16:58.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.942+0000 7f23843e1700 1 -- 172.21.15.186:0/1691144629 shutdown_connections 2024-01-31T05:16:58.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.942+0000 7f23843e1700 1 -- 172.21.15.186:0/1691144629 wait complete. 2024-01-31T05:16:58.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.942+0000 7f23843e1700 1 Processor -- start 2024-01-31T05:16:58.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.942+0000 7f23843e1700 1 -- start start 2024-01-31T05:16:58.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f23843e1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c0741c0 0x7f237c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:16:58.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f23843e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f237c075790 0x7f237c199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:16:58.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f23843e1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237c109700 0x7f237c1a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:16:58.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f23843e1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c1a1490 con 0x7f237c075790 2024-01-31T05:16:58.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f23843e1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c1a15d0 con 0x7f237c109700 2024-01-31T05:16:58.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f23843e1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c1a1760 con 0x7f237c0741c0 2024-01-31T05:16:58.944 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f238217d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c0741c0 0x7f237c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f238217d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c0741c0 0x7f237c199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:51620/0 (socket says 172.21.15.186:51620) 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f238217d700 1 -- 172.21.15.186:0/1637959416 learned_addr learned my addr 172.21.15.186:0/1637959416 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f238297e700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237c109700 0x7f237c1a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f238197c700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f237c075790 0x7f237c199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f238217d700 1 -- 172.21.15.186:0/1637959416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237c109700 msgr2=0x7f237c1a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f238217d700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237c109700 0x7f237c1a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f238217d700 1 -- 172.21.15.186:0/1637959416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f237c075790 msgr2=0x7f237c199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f238217d700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f237c075790 0x7f237c199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f238217d700 1 -- 172.21.15.186:0/1637959416 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2374009d00 con 0x7f237c0741c0 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f238217d700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c0741c0 0x7f237c199900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f237800c9e0 tx=0x7f237800ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f23737fe700 1 -- 172.21.15.186:0/1637959416 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378007ac0 con 0x7f237c0741c0 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f23843e1700 1 -- 172.21.15.186:0/1637959416 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c1a1a40 con 0x7f237c0741c0 2024-01-31T05:16:58.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f23737fe700 1 -- 172.21.15.186:0/1637959416 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f237800ce80 con 0x7f237c0741c0 2024-01-31T05:16:58.946 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.943+0000 7f23737fe700 1 -- 172.21.15.186:0/1637959416 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378018660 con 0x7f237c0741c0 2024-01-31T05:16:58.946 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.944+0000 7f23843e1700 1 -- 172.21.15.186:0/1637959416 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237c1a1f40 con 0x7f237c0741c0 2024-01-31T05:16:58.946 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.944+0000 7f23843e1700 1 -- 172.21.15.186:0/1637959416 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237c108510 con 0x7f237c0741c0 2024-01-31T05:16:58.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.946+0000 7f23737fe700 1 -- 172.21.15.186:0/1637959416 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f237800f450 con 0x7f237c0741c0 2024-01-31T05:16:58.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.946+0000 7f23737fe700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f23680a1370 0x7f23680a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:16:58.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.946+0000 7f23737fe700 1 -- 172.21.15.186:0/1637959416 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f23780cbca0 con 0x7f237c0741c0 2024-01-31T05:16:58.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.947+0000 7f238197c700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f23680a1370 0x7f23680a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:16:58.947 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.947+0000 7f238197c700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f23680a1370 0x7f23680a3830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f236c009c60 tx=0x7f236c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:16:58.950 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:58.951+0000 7f23737fe700 1 -- 172.21.15.186:0/1637959416 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f2378096220 con 0x7f237c0741c0 2024-01-31T05:16:59.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:58 smithi186 ceph-mon[35659]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:16:59.148 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:16:59.149+0000 7f23843e1700 1 -- 172.21.15.186:0/1637959416 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f237c02d540 con 0x7f23680a1370 2024-01-31T05:16:59.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:58 smithi156 ceph-mon[35720]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:16:59.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:58 smithi045 ceph-mon[31585]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:00.074 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:59 smithi186 ceph-mon[35659]: osdmap e35: 5 total, 5 up, 5 in 2024-01-31T05:17:00.074 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:59 smithi186 ceph-mon[35659]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:17:00.074 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:17:00.074 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:17:00.074 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:16:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:00.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:59 smithi156 ceph-mon[35720]: osdmap e35: 5 total, 5 up, 5 in 2024-01-31T05:17:00.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:59 smithi156 ceph-mon[35720]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:17:00.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:17:00.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:17:00.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:16:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:00.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:59 smithi045 ceph-mon[31585]: osdmap e35: 5 total, 5 up, 5 in 2024-01-31T05:17:00.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:59 smithi045 ceph-mon[31585]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:17:00.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:17:00.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:17:00.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:16:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:01.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:00 smithi186 ceph-mon[35659]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:01.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:00 smithi045 ceph-mon[31585]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:01.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:00 smithi156 ceph-mon[35720]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:02.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:01 smithi186 ceph-mon[35659]: from='client.? 172.21.15.186:0/889661039' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69641e46-7955-4b81-990e-bf656f6d7ff2"}]: dispatch 2024-01-31T05:17:02.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:01 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69641e46-7955-4b81-990e-bf656f6d7ff2"}]: dispatch 2024-01-31T05:17:02.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:01 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69641e46-7955-4b81-990e-bf656f6d7ff2"}]': finished 2024-01-31T05:17:02.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:01 smithi186 ceph-mon[35659]: osdmap e36: 6 total, 5 up, 6 in 2024-01-31T05:17:02.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:02.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:01 smithi045 ceph-mon[31585]: from='client.? 172.21.15.186:0/889661039' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69641e46-7955-4b81-990e-bf656f6d7ff2"}]: dispatch 2024-01-31T05:17:02.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:01 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69641e46-7955-4b81-990e-bf656f6d7ff2"}]: dispatch 2024-01-31T05:17:02.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:01 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69641e46-7955-4b81-990e-bf656f6d7ff2"}]': finished 2024-01-31T05:17:02.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:01 smithi045 ceph-mon[31585]: osdmap e36: 6 total, 5 up, 6 in 2024-01-31T05:17:02.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:02.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:01 smithi156 ceph-mon[35720]: from='client.? 172.21.15.186:0/889661039' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69641e46-7955-4b81-990e-bf656f6d7ff2"}]: dispatch 2024-01-31T05:17:02.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:01 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69641e46-7955-4b81-990e-bf656f6d7ff2"}]: dispatch 2024-01-31T05:17:02.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:01 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69641e46-7955-4b81-990e-bf656f6d7ff2"}]': finished 2024-01-31T05:17:02.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:01 smithi156 ceph-mon[35720]: osdmap e36: 6 total, 5 up, 6 in 2024-01-31T05:17:02.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:03.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:03 smithi156 ceph-mon[35720]: from='client.? 172.21.15.186:0/96185453' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:17:03.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:03 smithi156 ceph-mon[35720]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:03.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:03 smithi186 ceph-mon[35659]: from='client.? 172.21.15.186:0/96185453' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:17:03.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:03 smithi186 ceph-mon[35659]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:03.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:03 smithi045 ceph-mon[31585]: from='client.? 172.21.15.186:0/96185453' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:17:03.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:03 smithi045 ceph-mon[31585]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:05.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:05 smithi045 ceph-mon[31585]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:05.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:05 smithi156 ceph-mon[35720]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:06.085 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:05 smithi186 ceph-mon[35659]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:07.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:07 smithi045 ceph-mon[31585]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:07.916 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:07 smithi186 ceph-mon[35659]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:07.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:07 smithi156 ceph-mon[35720]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:09.878 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:09 smithi186 ceph-mon[35659]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:09.879 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-01-31T05:17:09.879 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:09.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:09 smithi045 ceph-mon[31585]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:09.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-01-31T05:17:09.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:09.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:09 smithi156 ceph-mon[35720]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:09.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-01-31T05:17:09.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:10.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:10 smithi186 ceph-mon[35659]: Deploying daemon osd.5 on smithi186 2024-01-31T05:17:10.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:10 smithi045 ceph-mon[31585]: Deploying daemon osd.5 on smithi186 2024-01-31T05:17:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:10 smithi156 ceph-mon[35720]: Deploying daemon osd.5 on smithi186 2024-01-31T05:17:11.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:11 smithi045 ceph-mon[31585]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:11 smithi156 ceph-mon[35720]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:12.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:11 smithi186 ceph-mon[35659]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:13.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:13 smithi186 ceph-mon[35659]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:13 smithi156 ceph-mon[35720]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:13.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:13 smithi045 ceph-mon[31585]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:15.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:15 smithi186 ceph-mon[35659]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:15.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:15.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:15.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:17:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:15 smithi045 ceph-mon[31585]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:17:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:15 smithi156 ceph-mon[35720]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:17:17.680 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:17 smithi186 ceph-mon[35659]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:17.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:17 smithi045 ceph-mon[31585]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:17 smithi156 ceph-mon[35720]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:18.064 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 5 on host 'smithi186' 2024-01-31T05:17:18.064 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.064+0000 7f23737fe700 1 -- 172.21.15.186:0/1637959416 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f237c02d540 con 0x7f23680a1370 2024-01-31T05:17:18.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.066+0000 7f23843e1700 1 -- 172.21.15.186:0/1637959416 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f23680a1370 msgr2=0x7f23680a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:17:18.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.066+0000 7f23843e1700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f23680a1370 0x7f23680a3830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f236c009c60 tx=0x7f236c009260 comp rx=0 tx=0).stop 2024-01-31T05:17:18.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.066+0000 7f23843e1700 1 -- 172.21.15.186:0/1637959416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c0741c0 msgr2=0x7f237c199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:17:18.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.066+0000 7f23843e1700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c0741c0 0x7f237c199900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f237800c9e0 tx=0x7f237800ccf0 comp rx=0 tx=0).stop 2024-01-31T05:17:18.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.066+0000 7f23843e1700 1 -- 172.21.15.186:0/1637959416 shutdown_connections 2024-01-31T05:17:18.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.066+0000 7f23843e1700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237c109700 0x7f237c1a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:18.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.066+0000 7f23843e1700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c0741c0 0x7f237c199900 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:18.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.066+0000 7f23843e1700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f23680a1370 0x7f23680a3830 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:18.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.066+0000 7f23843e1700 1 --2- 172.21.15.186:0/1637959416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f237c075790 0x7f237c199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:18.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.066+0000 7f23843e1700 1 -- 172.21.15.186:0/1637959416 >> 172.21.15.186:0/1637959416 conn(0x7f237c0fcdc0 msgr2=0x7f237c105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:17:18.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.067+0000 7f23843e1700 1 -- 172.21.15.186:0/1637959416 shutdown_connections 2024-01-31T05:17:18.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:17:18.067+0000 7f23843e1700 1 -- 172.21.15.186:0/1637959416 wait complete. 2024-01-31T05:17:18.727 DEBUG:teuthology.orchestra.run.smithi186:osd.5> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.5.service 2024-01-31T05:17:18.729 INFO:tasks.cephadm:Deploying osd.6 on smithi045 with /dev/vg_nvme/lv_2... 2024-01-31T05:17:18.729 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-01-31T05:17:18.979 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:17:19.270 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:19 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:19.271 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:19 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:19.271 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:19 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:19.271 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:19 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:19.271 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:19 smithi186 ceph-mon[35659]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:19.271 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:19 smithi186 ceph-mon[35659]: from='osd.5 [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-31T05:17:19.271 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:19 smithi186 ceph-mon[35659]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-31T05:17:19.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:19.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:19.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:19.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:19.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:19 smithi045 ceph-mon[31585]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:19.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:19 smithi045 ceph-mon[31585]: from='osd.5 [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-31T05:17:19.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:19 smithi045 ceph-mon[31585]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-31T05:17:19.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:19 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:19.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:19 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:19.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:19 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:19.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:19 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:19.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:19 smithi156 ceph-mon[35720]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:19.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:19 smithi156 ceph-mon[35720]: from='osd.5 [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-31T05:17:19.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:19 smithi156 ceph-mon[35720]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-31T05:17:20.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:20 smithi186 ceph-mon[35659]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-01-31T05:17:20.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:20 smithi186 ceph-mon[35659]: osdmap e37: 6 total, 5 up, 6 in 2024-01-31T05:17:20.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:20 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:20.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:20 smithi186 ceph-mon[35659]: from='osd.5 [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:17:20.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:20 smithi186 ceph-mon[35659]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:17:20.592 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:17:20 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:17:20.379+0000 7f014713d700 -1 osd.5 0 waiting for initial osdmap 2024-01-31T05:17:20.592 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:17:20 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:17:20.386+0000 7f0143738700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:17:20.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:20 smithi045 ceph-mon[31585]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-01-31T05:17:20.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:20 smithi045 ceph-mon[31585]: osdmap e37: 6 total, 5 up, 6 in 2024-01-31T05:17:20.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:20 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:20.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:20 smithi045 ceph-mon[31585]: from='osd.5 [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:17:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:20 smithi045 ceph-mon[31585]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:17:20.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:20 smithi156 ceph-mon[35720]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-01-31T05:17:20.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:20 smithi156 ceph-mon[35720]: osdmap e37: 6 total, 5 up, 6 in 2024-01-31T05:17:20.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:20 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:20.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:20 smithi156 ceph-mon[35720]: from='osd.5 [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:17:20.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:20 smithi156 ceph-mon[35720]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:17:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-01-31T05:17:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-01-31T05:17:20.840 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-01-31T05:17:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-01-31T05:17:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00851798 s, 1.2 GB/s 2024-01-31T05:17:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-01-31T05:17:21.388 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2024-01-31T05:17:21.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:21 smithi045 ceph-mon[31585]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-01-31T05:17:21.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:21 smithi045 ceph-mon[31585]: osdmap e38: 6 total, 5 up, 6 in 2024-01-31T05:17:21.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:21.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:21.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:21 smithi045 ceph-mon[31585]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:21.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:21 smithi186 ceph-mon[35659]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-01-31T05:17:21.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:21 smithi186 ceph-mon[35659]: osdmap e38: 6 total, 5 up, 6 in 2024-01-31T05:17:21.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:21.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:21.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:21 smithi186 ceph-mon[35659]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:17:21.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:21 smithi156 ceph-mon[35720]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-01-31T05:17:21.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:21 smithi156 ceph-mon[35720]: osdmap e38: 6 total, 5 up, 6 in 2024-01-31T05:17:21.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:21.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:21.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:21 smithi156 ceph-mon[35720]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-31T05:17:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: purged_snaps scrub starts 2024-01-31T05:17:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: purged_snaps scrub ok 2024-01-31T05:17:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: osd.5 [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307] boot 2024-01-31T05:17:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: osdmap e39: 6 total, 6 up, 6 in 2024-01-31T05:17:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:22.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:22.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:17:22.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:17:22.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:22.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:17:22.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: purged_snaps scrub starts 2024-01-31T05:17:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: purged_snaps scrub ok 2024-01-31T05:17:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: osd.5 [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307] boot 2024-01-31T05:17:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: osdmap e39: 6 total, 6 up, 6 in 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:17:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: purged_snaps scrub starts 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: purged_snaps scrub ok 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: osd.5 [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307] boot 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: osdmap e39: 6 total, 6 up, 6 in 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:17:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:22.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:17:22.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:22.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:17:22.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.238+0000 7f7ff04fe700 1 -- 172.21.15.45:0/2854813736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fe8101810 msgr2=0x7f7fe8101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:17:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.238+0000 7f7ff04fe700 1 --2- 172.21.15.45:0/2854813736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fe8101810 0x7f7fe8101c30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7fe40099c0 tx=0x7f7fe4009cd0 comp rx=0 tx=0).stop 2024-01-31T05:17:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.239+0000 7f7ff04fe700 1 -- 172.21.15.45:0/2854813736 shutdown_connections 2024-01-31T05:17:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.239+0000 7f7ff04fe700 1 --2- 172.21.15.45:0/2854813736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe81033b0 0x7f7fe8109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.239+0000 7f7ff04fe700 1 --2- 172.21.15.45:0/2854813736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fe8101810 0x7f7fe8101c30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.239+0000 7f7ff04fe700 1 --2- 172.21.15.45:0/2854813736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe8102a10 0x7f7fe8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.239+0000 7f7ff04fe700 1 -- 172.21.15.45:0/2854813736 >> 172.21.15.45:0/2854813736 conn(0x7f7fe80fcdf0 msgr2=0x7f7fe80ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.239+0000 7f7ff04fe700 1 -- 172.21.15.45:0/2854813736 shutdown_connections 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.239+0000 7f7ff04fe700 1 -- 172.21.15.45:0/2854813736 wait complete. 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.239+0000 7f7ff04fe700 1 Processor -- start 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.239+0000 7f7ff04fe700 1 -- start start 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.239+0000 7f7ff04fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe8101810 0x7f7fe8197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7ff04fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe8102a10 0x7f7fe8197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7ff04fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fe81033b0 0x7f7fe819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7ff04fe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe819f320 con 0x7f7fe81033b0 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7ff04fe700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe819f460 con 0x7f7fe8102a10 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7ff04fe700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe819f5f0 con 0x7f7fe8101810 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7feea9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fe81033b0 0x7f7fe819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7feea9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fe81033b0 0x7f7fe819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44856/0 (socket says 172.21.15.45:44856) 2024-01-31T05:17:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7feea9b700 1 -- 172.21.15.45:0/1030521746 learned_addr learned my addr 172.21.15.45:0/1030521746 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7feda99700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe8102a10 0x7f7fe8197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7fee29a700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe8101810 0x7f7fe8197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7feea9b700 1 -- 172.21.15.45:0/1030521746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe8101810 msgr2=0x7f7fe8197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7feea9b700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe8101810 0x7f7fe8197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7feea9b700 1 -- 172.21.15.45:0/1030521746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe8102a10 msgr2=0x7f7fe8197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7feea9b700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe8102a10 0x7f7fe8197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7feea9b700 1 -- 172.21.15.45:0/1030521746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe40095f0 con 0x7f7fe81033b0 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7feea9b700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fe81033b0 0x7f7fe819ed50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000add0 tx=0x7f7fe000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7fdf7fe700 1 -- 172.21.15.45:0/1030521746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000f070 con 0x7f7fe81033b0 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7ff04fe700 1 -- 172.21.15.45:0/1030521746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe819f8d0 con 0x7f7fe81033b0 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7fdf7fe700 1 -- 172.21.15.45:0/1030521746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fe0007b30 con 0x7f7fe81033b0 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.240+0000 7f7fdf7fe700 1 -- 172.21.15.45:0/1030521746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0005310 con 0x7f7fe81033b0 2024-01-31T05:17:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.241+0000 7f7ff04fe700 1 -- 172.21.15.45:0/1030521746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe819fdd0 con 0x7f7fe81033b0 2024-01-31T05:17:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.242+0000 7f7ff04fe700 1 -- 172.21.15.45:0/1030521746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe8108ca0 con 0x7f7fe81033b0 2024-01-31T05:17:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.242+0000 7f7fdf7fe700 1 -- 172.21.15.45:0/1030521746 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f7fe000c940 con 0x7f7fe81033b0 2024-01-31T05:17:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.243+0000 7f7fdf7fe700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f7fd40a1370 0x7f7fd40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:17:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.243+0000 7f7fee29a700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f7fd40a1370 0x7f7fd40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:17:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.243+0000 7f7fdf7fe700 1 -- 172.21.15.45:0/1030521746 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f7fe00cb250 con 0x7f7fe81033b0 2024-01-31T05:17:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.243+0000 7f7fee29a700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f7fd40a1370 0x7f7fd40a3830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009990 tx=0x7f7fe4005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:17:23.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.248+0000 7f7fdf7fe700 1 -- 172.21.15.45:0/1030521746 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7fe0095600 con 0x7f7fe81033b0 2024-01-31T05:17:23.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:23.465+0000 7f7ff04fe700 1 -- 172.21.15.45:0/1030521746 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fe802d540 con 0x7f7fd40a1370 2024-01-31T05:17:23.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:23 smithi045 ceph-mon[31585]: Detected new or changed devices on smithi186 2024-01-31T05:17:23.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:23 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi186 to 8540M 2024-01-31T05:17:23.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:23 smithi045 ceph-mon[31585]: osdmap e40: 6 total, 6 up, 6 in 2024-01-31T05:17:23.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:23 smithi045 ceph-mon[31585]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:23.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:23 smithi156 ceph-mon[35720]: Detected new or changed devices on smithi186 2024-01-31T05:17:23.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:23 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi186 to 8540M 2024-01-31T05:17:23.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:23 smithi156 ceph-mon[35720]: osdmap e40: 6 total, 6 up, 6 in 2024-01-31T05:17:23.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:23 smithi156 ceph-mon[35720]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:23.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:23 smithi186 ceph-mon[35659]: Detected new or changed devices on smithi186 2024-01-31T05:17:23.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:23 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi186 to 8540M 2024-01-31T05:17:23.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:23 smithi186 ceph-mon[35659]: osdmap e40: 6 total, 6 up, 6 in 2024-01-31T05:17:23.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:23 smithi186 ceph-mon[35659]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:24.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:24 smithi045 ceph-mon[31585]: osdmap e41: 6 total, 6 up, 6 in 2024-01-31T05:17:24.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:24 smithi045 ceph-mon[31585]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:17:24.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:24 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:17:24.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:24 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:17:24.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:24 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:24.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:24 smithi156 ceph-mon[35720]: osdmap e41: 6 total, 6 up, 6 in 2024-01-31T05:17:24.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:24 smithi156 ceph-mon[35720]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:17:24.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:24 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:17:24.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:24 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:17:24.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:24 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:24.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:24 smithi186 ceph-mon[35659]: osdmap e41: 6 total, 6 up, 6 in 2024-01-31T05:17:24.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:24 smithi186 ceph-mon[35659]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:17:24.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:24 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:17:24.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:24 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:17:24.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:24 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:25.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:25 smithi156 ceph-mon[35720]: osdmap e42: 6 total, 6 up, 6 in 2024-01-31T05:17:25.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:25 smithi156 ceph-mon[35720]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:25.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:25 smithi186 ceph-mon[35659]: osdmap e42: 6 total, 6 up, 6 in 2024-01-31T05:17:25.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:25 smithi186 ceph-mon[35659]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:25.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:25 smithi045 ceph-mon[31585]: osdmap e42: 6 total, 6 up, 6 in 2024-01-31T05:17:25.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:25 smithi045 ceph-mon[31585]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:26.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:26 smithi156 ceph-mon[35720]: osdmap e43: 6 total, 6 up, 6 in 2024-01-31T05:17:26.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:26 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2293392361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9449230f-ba33-45ee-993a-12a376e2c9ff"}]: dispatch 2024-01-31T05:17:26.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:26 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2293392361' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9449230f-ba33-45ee-993a-12a376e2c9ff"}]': finished 2024-01-31T05:17:26.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:26 smithi156 ceph-mon[35720]: osdmap e44: 7 total, 6 up, 7 in 2024-01-31T05:17:26.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:26 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:26.756 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:26 smithi186 ceph-mon[35659]: osdmap e43: 6 total, 6 up, 6 in 2024-01-31T05:17:26.756 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:26 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2293392361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9449230f-ba33-45ee-993a-12a376e2c9ff"}]: dispatch 2024-01-31T05:17:26.756 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:26 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2293392361' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9449230f-ba33-45ee-993a-12a376e2c9ff"}]': finished 2024-01-31T05:17:26.756 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:26 smithi186 ceph-mon[35659]: osdmap e44: 7 total, 6 up, 7 in 2024-01-31T05:17:26.756 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:26 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:26.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:26 smithi045 ceph-mon[31585]: osdmap e43: 6 total, 6 up, 6 in 2024-01-31T05:17:26.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:26 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2293392361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9449230f-ba33-45ee-993a-12a376e2c9ff"}]: dispatch 2024-01-31T05:17:26.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:26 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2293392361' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9449230f-ba33-45ee-993a-12a376e2c9ff"}]': finished 2024-01-31T05:17:26.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:26 smithi045 ceph-mon[31585]: osdmap e44: 7 total, 6 up, 7 in 2024-01-31T05:17:26.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:26 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:27.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:27 smithi156 ceph-mon[35720]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-31T05:17:27.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:27 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/913092813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:17:27.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:27 smithi186 ceph-mon[35659]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-31T05:17:27.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:27 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/913092813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:17:27.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:27 smithi045 ceph-mon[31585]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-31T05:17:27.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:27 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/913092813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:17:29.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:29 smithi045 ceph-mon[31585]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-01-31T05:17:29.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:29 smithi156 ceph-mon[35720]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-01-31T05:17:30.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:29 smithi186 ceph-mon[35659]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-01-31T05:17:31.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:31 smithi045 ceph-mon[31585]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-01-31T05:17:31.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:31 smithi156 ceph-mon[35720]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-01-31T05:17:32.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:31 smithi186 ceph-mon[35659]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-01-31T05:17:33.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:33 smithi045 ceph-mon[31585]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-31T05:17:33.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:33 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-01-31T05:17:33.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:33 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:33.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:33 smithi156 ceph-mon[35720]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-31T05:17:33.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:33 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-01-31T05:17:33.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:33 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:33.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:33 smithi186 ceph-mon[35659]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-31T05:17:33.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:33 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-01-31T05:17:33.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:33 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:34.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:34 smithi045 ceph-mon[31585]: Deploying daemon osd.6 on smithi045 2024-01-31T05:17:34.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:34 smithi156 ceph-mon[35720]: Deploying daemon osd.6 on smithi045 2024-01-31T05:17:34.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:34 smithi186 ceph-mon[35659]: Deploying daemon osd.6 on smithi045 2024-01-31T05:17:35.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:35 smithi045 ceph-mon[31585]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-01-31T05:17:35.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:35 smithi156 ceph-mon[35720]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-01-31T05:17:35.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:35 smithi186 ceph-mon[35659]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-01-31T05:17:37.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:37 smithi186 ceph-mon[35659]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:37.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:37 smithi045 ceph-mon[31585]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:37 smithi156 ceph-mon[35720]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:39.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:39 smithi045 ceph-mon[31585]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:39.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:39.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:39.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:17:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:39 smithi156 ceph-mon[35720]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:17:40.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:39 smithi186 ceph-mon[35659]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:40.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:40.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:17:41.856 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:41 smithi045 ceph-mon[31585]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:41 smithi156 ceph-mon[35720]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:42.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:41 smithi186 ceph-mon[35659]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.593+0000 7f7fdf7fe700 1 -- 172.21.15.45:0/1030521746 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7fe802d540 con 0x7f7fd40a1370 2024-01-31T05:17:42.594 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 6 on host 'smithi045' 2024-01-31T05:17:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.595+0000 7f7ff04fe700 1 -- 172.21.15.45:0/1030521746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f7fd40a1370 msgr2=0x7f7fd40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:17:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.595+0000 7f7ff04fe700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f7fd40a1370 0x7f7fd40a3830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009990 tx=0x7f7fe4005900 comp rx=0 tx=0).stop 2024-01-31T05:17:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.595+0000 7f7ff04fe700 1 -- 172.21.15.45:0/1030521746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fe81033b0 msgr2=0x7f7fe819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:17:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.595+0000 7f7ff04fe700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fe81033b0 0x7f7fe819ed50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000add0 tx=0x7f7fe000c330 comp rx=0 tx=0).stop 2024-01-31T05:17:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.595+0000 7f7ff04fe700 1 -- 172.21.15.45:0/1030521746 shutdown_connections 2024-01-31T05:17:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.595+0000 7f7ff04fe700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f7fd40a1370 0x7f7fd40a3830 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.595+0000 7f7ff04fe700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fe81033b0 0x7f7fe819ed50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.595+0000 7f7ff04fe700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe8101810 0x7f7fe8197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.595+0000 7f7ff04fe700 1 --2- 172.21.15.45:0/1030521746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fe8102a10 0x7f7fe8197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.595+0000 7f7ff04fe700 1 -- 172.21.15.45:0/1030521746 >> 172.21.15.45:0/1030521746 conn(0x7f7fe80fcdf0 msgr2=0x7f7fe8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:17:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.595+0000 7f7ff04fe700 1 -- 172.21.15.45:0/1030521746 shutdown_connections 2024-01-31T05:17:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:17:42.595+0000 7f7ff04fe700 1 -- 172.21.15.45:0/1030521746 wait complete. 2024-01-31T05:17:42.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:42 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:42.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:42 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:42 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:42 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:17:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:42 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:42 smithi045 ceph-mon[31585]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:42 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:42 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:42 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:17:43.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:42 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:43.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:42 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:43.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:42 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:42 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:17:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:42 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:42 smithi186 ceph-mon[35659]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:42 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:42 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:42 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:17:43.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:42 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:43.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:42 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:43.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:42 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:43.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:42 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:17:43.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:42 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:43.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:42 smithi156 ceph-mon[35720]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:43.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:42 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:43.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:42 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:43.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:42 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:17:43.234 DEBUG:teuthology.orchestra.run.smithi045:osd.6> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.6.service 2024-01-31T05:17:43.276 INFO:tasks.cephadm:Deploying osd.7 on smithi156 with /dev/vg_nvme/lv_2... 2024-01-31T05:17:43.277 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-01-31T05:17:43.522 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.b/config 2024-01-31T05:17:43.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:43 smithi045 ceph-mon[31585]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-01-31T05:17:43.914 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:43 smithi156 ceph-mon[35720]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-01-31T05:17:44.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:43 smithi186 ceph-mon[35659]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-01-31T05:17:44.870 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:44 smithi156 ceph-mon[35720]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-01-31T05:17:44.870 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:44 smithi156 ceph-mon[35720]: osdmap e45: 7 total, 6 up, 7 in 2024-01-31T05:17:44.870 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:44 smithi156 ceph-mon[35720]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-01-31T05:17:44.870 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:44 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:44.870 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:44 smithi156 ceph-mon[35720]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:45.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:44 smithi186 ceph-mon[35659]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-01-31T05:17:45.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:44 smithi186 ceph-mon[35659]: osdmap e45: 7 total, 6 up, 7 in 2024-01-31T05:17:45.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:44 smithi186 ceph-mon[35659]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-01-31T05:17:45.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:44 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:45.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:44 smithi186 ceph-mon[35659]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:44 smithi045 ceph-mon[31585]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-01-31T05:17:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:44 smithi045 ceph-mon[31585]: osdmap e45: 7 total, 6 up, 7 in 2024-01-31T05:17:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:44 smithi045 ceph-mon[31585]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-01-31T05:17:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:44 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:44 smithi045 ceph-mon[31585]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:45.417 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-01-31T05:17:45.417 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-01-31T05:17:45.417 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-01-31T05:17:45.417 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-01-31T05:17:45.417 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00846623 s, 1.2 GB/s 2024-01-31T05:17:45.417 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-01-31T05:17:45.791 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:45 smithi156 ceph-mon[35720]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-01-31T05:17:45.791 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:45 smithi156 ceph-mon[35720]: osdmap e46: 7 total, 6 up, 7 in 2024-01-31T05:17:45.791 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:45.792 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:45 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:45.792 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:45 smithi156 ceph-mon[35720]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' 2024-01-31T05:17:45.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:45 smithi045 ceph-mon[31585]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-01-31T05:17:45.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:45 smithi045 ceph-mon[31585]: osdmap e46: 7 total, 6 up, 7 in 2024-01-31T05:17:45.890 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:45.890 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:45 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:45.890 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:45 smithi045 ceph-mon[31585]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' 2024-01-31T05:17:45.890 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:17:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:17:45.505+0000 7f4f60e2c700 -1 osd.6 0 waiting for initial osdmap 2024-01-31T05:17:45.890 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:17:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:17:45.514+0000 7f4f5941f700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:17:46.085 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_2 2024-01-31T05:17:46.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:45 smithi186 ceph-mon[35659]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-01-31T05:17:46.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:45 smithi186 ceph-mon[35659]: osdmap e46: 7 total, 6 up, 7 in 2024-01-31T05:17:46.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:46.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:45 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:46.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:45 smithi186 ceph-mon[35659]: from='osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681]' entity='osd.6' 2024-01-31T05:17:46.323 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.b/config 2024-01-31T05:17:47.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: purged_snaps scrub starts 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: purged_snaps scrub ok 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: Detected new or changed devices on smithi045 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi045 to 5697M 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:17:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:47.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681] boot 2024-01-31T05:17:47.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: osdmap e47: 7 total, 7 up, 7 in 2024-01-31T05:17:47.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:47.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:46 smithi186 ceph-mon[35659]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:47.130 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: purged_snaps scrub starts 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: purged_snaps scrub ok 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: Detected new or changed devices on smithi045 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi045 to 5697M 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681] boot 2024-01-31T05:17:47.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: osdmap e47: 7 total, 7 up, 7 in 2024-01-31T05:17:47.132 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:47.132 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:46 smithi156 ceph-mon[35720]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:47.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: purged_snaps scrub starts 2024-01-31T05:17:47.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: purged_snaps scrub ok 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: Detected new or changed devices on smithi045 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi045 to 5697M 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:17:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: osd.6 [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681] boot 2024-01-31T05:17:47.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: osdmap e47: 7 total, 7 up, 7 in 2024-01-31T05:17:47.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:17:47.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:46 smithi045 ceph-mon[31585]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-01-31T05:17:47.920 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.921+0000 7f59f86d4700 1 -- 172.21.15.156:0/3331501126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59f0103450 msgr2=0x7f59f0109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:17:47.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.921+0000 7f59f86d4700 1 --2- 172.21.15.156:0/3331501126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59f0103450 0x7f59f0109d00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f59e800b0a0 tx=0x7f59e800b3b0 comp rx=0 tx=0).stop 2024-01-31T05:17:47.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.921+0000 7f59f86d4700 1 -- 172.21.15.156:0/3331501126 shutdown_connections 2024-01-31T05:17:47.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.921+0000 7f59f86d4700 1 --2- 172.21.15.156:0/3331501126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59f0103450 0x7f59f0109d00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:47.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.921+0000 7f59f86d4700 1 --2- 172.21.15.156:0/3331501126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f00ffad0 0x7f59f00ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:47.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.921+0000 7f59f86d4700 1 --2- 172.21.15.156:0/3331501126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59f0102ab0 0x7f59f0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:47.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.921+0000 7f59f86d4700 1 -- 172.21.15.156:0/3331501126 >> 172.21.15.156:0/3331501126 conn(0x7f59f00fb450 msgr2=0x7f59f00fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:17:47.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.921+0000 7f59f86d4700 1 -- 172.21.15.156:0/3331501126 shutdown_connections 2024-01-31T05:17:47.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.921+0000 7f59f86d4700 1 -- 172.21.15.156:0/3331501126 wait complete. 2024-01-31T05:17:47.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f86d4700 1 Processor -- start 2024-01-31T05:17:47.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f86d4700 1 -- start start 2024-01-31T05:17:47.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f86d4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f00ffad0 0x7f59f01977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:17:47.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f86d4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59f0102ab0 0x7f59f0197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:17:47.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f86d4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59f0103450 0x7f59f019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:17:47.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f86d4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f019f340 con 0x7f59f00ffad0 2024-01-31T05:17:47.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f86d4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f019f480 con 0x7f59f0103450 2024-01-31T05:17:47.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f86d4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f019f610 con 0x7f59f0102ab0 2024-01-31T05:17:47.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f6c71700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59f0103450 0x7f59f019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:17:47.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f6c71700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59f0103450 0x7f59f019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:52252/0 (socket says 172.21.15.156:52252) 2024-01-31T05:17:47.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f6c71700 1 -- 172.21.15.156:0/902699533 learned_addr learned my addr 172.21.15.156:0/902699533 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-01-31T05:17:47.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f5c6f700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59f0102ab0 0x7f59f0197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:17:47.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.922+0000 7f59f6470700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f00ffad0 0x7f59f01977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:17:47.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.923+0000 7f59f6c71700 1 -- 172.21.15.156:0/902699533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59f0102ab0 msgr2=0x7f59f0197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:17:47.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.923+0000 7f59f6c71700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59f0102ab0 0x7f59f0197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:47.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.923+0000 7f59f6c71700 1 -- 172.21.15.156:0/902699533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f00ffad0 msgr2=0x7f59f01977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:17:47.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.923+0000 7f59f6c71700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f00ffad0 0x7f59f01977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:17:47.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.923+0000 7f59f6c71700 1 -- 172.21.15.156:0/902699533 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e8009d00 con 0x7f59f0103450 2024-01-31T05:17:47.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.923+0000 7f59f6c71700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59f0103450 0x7f59f019ed70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f59e8004d00 tx=0x7f59e8004d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:17:47.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.923+0000 7f59e77fe700 1 -- 172.21.15.156:0/902699533 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e800e040 con 0x7f59f0103450 2024-01-31T05:17:47.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.923+0000 7f59f86d4700 1 -- 172.21.15.156:0/902699533 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f019f890 con 0x7f59f0103450 2024-01-31T05:17:47.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.923+0000 7f59e77fe700 1 -- 172.21.15.156:0/902699533 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59e8009440 con 0x7f59f0103450 2024-01-31T05:17:47.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.923+0000 7f59e77fe700 1 -- 172.21.15.156:0/902699533 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e801b910 con 0x7f59f0103450 2024-01-31T05:17:47.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.923+0000 7f59f86d4700 1 -- 172.21.15.156:0/902699533 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f019fcb0 con 0x7f59f0103450 2024-01-31T05:17:47.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.924+0000 7f59f86d4700 1 -- 172.21.15.156:0/902699533 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f0108c70 con 0x7f59f0103450 2024-01-31T05:17:47.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.925+0000 7f59e77fe700 1 -- 172.21.15.156:0/902699533 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f59e801dbe0 con 0x7f59f0103450 2024-01-31T05:17:47.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.926+0000 7f59e77fe700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f59dc0a1370 0x7f59dc0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:17:47.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.926+0000 7f59e77fe700 1 -- 172.21.15.156:0/902699533 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f59e80cdf60 con 0x7f59f0103450 2024-01-31T05:17:47.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.926+0000 7f59f6470700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f59dc0a1370 0x7f59dc0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:17:47.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.926+0000 7f59f6470700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f59dc0a1370 0x7f59dc0a3830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f59f0069290 tx=0x7f59e000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:17:47.928 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:47.929+0000 7f59e77fe700 1 -- 172.21.15.156:0/902699533 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f59e8098180 con 0x7f59f0103450 2024-01-31T05:17:48.129 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:17:48.130+0000 7f59f86d4700 1 -- 172.21.15.156:0/902699533 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f59f002d4a0 con 0x7f59dc0a1370 2024-01-31T05:17:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:48 smithi156 ceph-mon[35720]: osdmap e48: 7 total, 7 up, 7 in 2024-01-31T05:17:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:48 smithi156 ceph-mon[35720]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:17:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:17:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:17:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:48.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:48 smithi156 ceph-mon[35720]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:49.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:48 smithi186 ceph-mon[35659]: osdmap e48: 7 total, 7 up, 7 in 2024-01-31T05:17:49.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:48 smithi186 ceph-mon[35659]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:17:49.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:17:49.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:17:49.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:49.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:48 smithi186 ceph-mon[35659]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:49.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:48 smithi045 ceph-mon[31585]: osdmap e48: 7 total, 7 up, 7 in 2024-01-31T05:17:49.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:48 smithi045 ceph-mon[31585]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:17:49.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:17:49.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:17:49.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:49.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:48 smithi045 ceph-mon[31585]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:51 smithi045 ceph-mon[31585]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:51 smithi045 ceph-mon[31585]: from='client.? 172.21.15.156:0/1798940666' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de8fa2cd-6e29-40b8-836f-90a64e4b09a1"}]: dispatch 2024-01-31T05:17:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:51 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de8fa2cd-6e29-40b8-836f-90a64e4b09a1"}]: dispatch 2024-01-31T05:17:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:51 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de8fa2cd-6e29-40b8-836f-90a64e4b09a1"}]': finished 2024-01-31T05:17:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:51 smithi045 ceph-mon[31585]: osdmap e49: 8 total, 7 up, 8 in 2024-01-31T05:17:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:51 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:17:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:51 smithi045 ceph-mon[31585]: from='client.? 172.21.15.156:0/2745117802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:17:51.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:51 smithi156 ceph-mon[35720]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:51.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:51 smithi156 ceph-mon[35720]: from='client.? 172.21.15.156:0/1798940666' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de8fa2cd-6e29-40b8-836f-90a64e4b09a1"}]: dispatch 2024-01-31T05:17:51.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:51 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de8fa2cd-6e29-40b8-836f-90a64e4b09a1"}]: dispatch 2024-01-31T05:17:51.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:51 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de8fa2cd-6e29-40b8-836f-90a64e4b09a1"}]': finished 2024-01-31T05:17:51.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:51 smithi156 ceph-mon[35720]: osdmap e49: 8 total, 7 up, 8 in 2024-01-31T05:17:51.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:51 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:17:51.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:51 smithi156 ceph-mon[35720]: from='client.? 172.21.15.156:0/2745117802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:17:52.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:51 smithi186 ceph-mon[35659]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:52.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:51 smithi186 ceph-mon[35659]: from='client.? 172.21.15.156:0/1798940666' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de8fa2cd-6e29-40b8-836f-90a64e4b09a1"}]: dispatch 2024-01-31T05:17:52.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:51 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de8fa2cd-6e29-40b8-836f-90a64e4b09a1"}]: dispatch 2024-01-31T05:17:52.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:51 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de8fa2cd-6e29-40b8-836f-90a64e4b09a1"}]': finished 2024-01-31T05:17:52.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:51 smithi186 ceph-mon[35659]: osdmap e49: 8 total, 7 up, 8 in 2024-01-31T05:17:52.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:51 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:17:52.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:51 smithi186 ceph-mon[35659]: from='client.? 172.21.15.156:0/2745117802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:17:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:53 smithi156 ceph-mon[35720]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:53.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:53 smithi186 ceph-mon[35659]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:53.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:53 smithi045 ceph-mon[31585]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:55.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:55 smithi045 ceph-mon[31585]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:55.920 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:55 smithi156 ceph-mon[35720]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:56.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:55 smithi186 ceph-mon[35659]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:57.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:57 smithi045 ceph-mon[31585]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:57.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:57 smithi156 ceph-mon[35720]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:58.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:57 smithi186 ceph-mon[35659]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:58.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:58 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-01-31T05:17:58.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:58 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:58.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:58 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-01-31T05:17:58.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:58 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:59.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:58 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-01-31T05:17:59.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:58 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:17:59.838 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:59 smithi156 ceph-mon[35720]: Deploying daemon osd.7 on smithi156 2024-01-31T05:17:59.838 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:17:59 smithi156 ceph-mon[35720]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:17:59.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:59 smithi045 ceph-mon[31585]: Deploying daemon osd.7 on smithi156 2024-01-31T05:17:59.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:17:59 smithi045 ceph-mon[31585]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:00.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:59 smithi186 ceph-mon[35659]: Deploying daemon osd.7 on smithi156 2024-01-31T05:18:00.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:17:59 smithi186 ceph-mon[35659]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:01.849 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:01 smithi156 ceph-mon[35720]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:01.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:01 smithi045 ceph-mon[31585]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:02.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:01 smithi186 ceph-mon[35659]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:03.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:03 smithi156 ceph-mon[35720]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:03.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:03 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:03.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:03 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:03.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:03 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:03.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:03 smithi186 ceph-mon[35659]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:03.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:03 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:03.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:03 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:03.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:03 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:03.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:03 smithi045 ceph-mon[31585]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:03.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:03 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:03.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:03 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:03.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:03 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:05.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:05 smithi045 ceph-mon[31585]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:05.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:05 smithi156 ceph-mon[35720]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:06.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:05 smithi186 ceph-mon[35659]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:07.095 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 7 on host 'smithi156' 2024-01-31T05:18:07.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.092+0000 7f59e77fe700 1 -- 172.21.15.156:0/902699533 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f59f002d4a0 con 0x7f59dc0a1370 2024-01-31T05:18:07.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.095+0000 7f59f86d4700 1 -- 172.21.15.156:0/902699533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f59dc0a1370 msgr2=0x7f59dc0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:18:07.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.095+0000 7f59f86d4700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f59dc0a1370 0x7f59dc0a3830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f59f0069290 tx=0x7f59e000a380 comp rx=0 tx=0).stop 2024-01-31T05:18:07.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.095+0000 7f59f86d4700 1 -- 172.21.15.156:0/902699533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59f0103450 msgr2=0x7f59f019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:18:07.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.095+0000 7f59f86d4700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59f0103450 0x7f59f019ed70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f59e8004d00 tx=0x7f59e8004d30 comp rx=0 tx=0).stop 2024-01-31T05:18:07.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.095+0000 7f59f86d4700 1 -- 172.21.15.156:0/902699533 shutdown_connections 2024-01-31T05:18:07.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.095+0000 7f59f86d4700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59f0103450 0x7f59f019ed70 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:07.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.095+0000 7f59f86d4700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f59dc0a1370 0x7f59dc0a3830 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:07.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.095+0000 7f59f86d4700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f00ffad0 0x7f59f01977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:07.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.095+0000 7f59f86d4700 1 --2- 172.21.15.156:0/902699533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59f0102ab0 0x7f59f0197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:07.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.095+0000 7f59f86d4700 1 -- 172.21.15.156:0/902699533 >> 172.21.15.156:0/902699533 conn(0x7f59f00fb450 msgr2=0x7f59f0105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:18:07.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.095+0000 7f59f86d4700 1 -- 172.21.15.156:0/902699533 shutdown_connections 2024-01-31T05:18:07.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:18:07.095+0000 7f59f86d4700 1 -- 172.21.15.156:0/902699533 wait complete. 2024-01-31T05:18:07.467 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.468 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.468 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:07.468 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:07.468 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.468 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:07 smithi156 ceph-mon[35720]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:07.468 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.468 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.468 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:07 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:07.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:07.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:07.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:07 smithi186 ceph-mon[35659]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:07.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:07 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:07 smithi045 ceph-mon[31585]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:07.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:07 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:07.761 DEBUG:teuthology.orchestra.run.smithi156:osd.7> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.7.service 2024-01-31T05:18:07.763 INFO:tasks.cephadm:Deploying osd.8 on smithi186 with /dev/vg_nvme/lv_2... 2024-01-31T05:18:07.763 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-01-31T05:18:08.010 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.c/config 2024-01-31T05:18:08.453 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:08 smithi186 ceph-mon[35659]: from='osd.7 [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-31T05:18:08.453 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:08 smithi186 ceph-mon[35659]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-31T05:18:08.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:08 smithi156 ceph-mon[35720]: from='osd.7 [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-31T05:18:08.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:08 smithi156 ceph-mon[35720]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-31T05:18:08.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:08 smithi045 ceph-mon[31585]: from='osd.7 [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-31T05:18:08.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:08 smithi045 ceph-mon[31585]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-31T05:18:09.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:09 smithi186 ceph-mon[35659]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-01-31T05:18:09.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:09 smithi186 ceph-mon[35659]: from='osd.7 [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:18:09.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:09 smithi186 ceph-mon[35659]: osdmap e50: 8 total, 7 up, 8 in 2024-01-31T05:18:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:09 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:09 smithi186 ceph-mon[35659]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:18:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:09 smithi186 ceph-mon[35659]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:09.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:09 smithi156 ceph-mon[35720]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-01-31T05:18:09.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:09 smithi156 ceph-mon[35720]: from='osd.7 [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:18:09.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:09 smithi156 ceph-mon[35720]: osdmap e50: 8 total, 7 up, 8 in 2024-01-31T05:18:09.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:09 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:09.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:09 smithi156 ceph-mon[35720]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:18:09.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:09 smithi156 ceph-mon[35720]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:09.477 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:18:09 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:18:09.267+0000 7f9c10e8f700 -1 osd.7 0 waiting for initial osdmap 2024-01-31T05:18:09.477 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:18:09 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:18:09.277+0000 7f9c0d48a700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:18:09.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:09 smithi045 ceph-mon[31585]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-01-31T05:18:09.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:09 smithi045 ceph-mon[31585]: from='osd.7 [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:18:09.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:09 smithi045 ceph-mon[31585]: osdmap e50: 8 total, 7 up, 8 in 2024-01-31T05:18:09.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:09 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:09.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:09 smithi045 ceph-mon[31585]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:18:09.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:09 smithi045 ceph-mon[31585]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:09.929 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-01-31T05:18:09.929 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-01-31T05:18:09.929 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-01-31T05:18:09.929 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-01-31T05:18:09.929 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011154 s, 940 MB/s 2024-01-31T05:18:09.929 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-01-31T05:18:10.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:10 smithi186 ceph-mon[35659]: purged_snaps scrub starts 2024-01-31T05:18:10.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:10 smithi186 ceph-mon[35659]: purged_snaps scrub ok 2024-01-31T05:18:10.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:10 smithi186 ceph-mon[35659]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-01-31T05:18:10.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:10 smithi186 ceph-mon[35659]: osdmap e51: 8 total, 7 up, 8 in 2024-01-31T05:18:10.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:10 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:10.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:10 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:10.540 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch daemon add osd smithi186:vg_nvme/lv_2 2024-01-31T05:18:10.589 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:10 smithi156 ceph-mon[35720]: purged_snaps scrub starts 2024-01-31T05:18:10.590 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:10 smithi156 ceph-mon[35720]: purged_snaps scrub ok 2024-01-31T05:18:10.590 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:10 smithi156 ceph-mon[35720]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-01-31T05:18:10.590 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:10 smithi156 ceph-mon[35720]: osdmap e51: 8 total, 7 up, 8 in 2024-01-31T05:18:10.590 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:10 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:10.590 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:10 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:10.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:10 smithi045 ceph-mon[31585]: purged_snaps scrub starts 2024-01-31T05:18:10.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:10 smithi045 ceph-mon[31585]: purged_snaps scrub ok 2024-01-31T05:18:10.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:10 smithi045 ceph-mon[31585]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-01-31T05:18:10.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:10 smithi045 ceph-mon[31585]: osdmap e51: 8 total, 7 up, 8 in 2024-01-31T05:18:10.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:10 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:10.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:10 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:10.788 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.c/config 2024-01-31T05:18:11.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:11.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: osd.7 [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879] boot 2024-01-31T05:18:11.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: osdmap e52: 8 total, 8 up, 8 in 2024-01-31T05:18:11.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:11.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:11.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: Detected new or changed devices on smithi156 2024-01-31T05:18:11.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:11.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:11.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:11.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi156 to 5693M 2024-01-31T05:18:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:11.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:11.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:11.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: osd.7 [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879] boot 2024-01-31T05:18:11.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: osdmap e52: 8 total, 8 up, 8 in 2024-01-31T05:18:11.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:11.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:11.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: Detected new or changed devices on smithi156 2024-01-31T05:18:11.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi156 to 5693M 2024-01-31T05:18:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: osd.7 [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879] boot 2024-01-31T05:18:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: osdmap e52: 8 total, 8 up, 8 in 2024-01-31T05:18:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:18:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-01-31T05:18:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: Detected new or changed devices on smithi156 2024-01-31T05:18:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi156 to 5693M 2024-01-31T05:18:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:12.372 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.370+0000 7f0f48226700 1 -- 172.21.15.186:0/3850566649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f401033d0 msgr2=0x7f0f40109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:18:12.372 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.370+0000 7f0f48226700 1 --2- 172.21.15.186:0/3850566649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f401033d0 0x7f0f40109c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009990 tx=0x7f0f38009ca0 comp rx=0 tx=0).stop 2024-01-31T05:18:12.372 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.370+0000 7f0f48226700 1 -- 172.21.15.186:0/3850566649 shutdown_connections 2024-01-31T05:18:12.373 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.370+0000 7f0f48226700 1 --2- 172.21.15.186:0/3850566649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f401033d0 0x7f0f40109c80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:12.373 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.370+0000 7f0f48226700 1 --2- 172.21.15.186:0/3850566649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f40101830 0x7f0f40101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:12.373 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.370+0000 7f0f48226700 1 --2- 172.21.15.186:0/3850566649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f40102a30 0x7f0f40102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:12.373 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.370+0000 7f0f48226700 1 -- 172.21.15.186:0/3850566649 >> 172.21.15.186:0/3850566649 conn(0x7f0f400fcdf0 msgr2=0x7f0f400ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:18:12.373 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.370+0000 7f0f48226700 1 -- 172.21.15.186:0/3850566649 shutdown_connections 2024-01-31T05:18:12.373 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.370+0000 7f0f48226700 1 -- 172.21.15.186:0/3850566649 wait complete. 2024-01-31T05:18:12.373 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f48226700 1 Processor -- start 2024-01-31T05:18:12.373 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f48226700 1 -- start start 2024-01-31T05:18:12.373 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f48226700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f40101830 0x7f0f40197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:18:12.373 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f48226700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f40102a30 0x7f0f40197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:18:12.373 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f48226700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f401033d0 0x7f0f4019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:18:12.373 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f48226700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4019f110 con 0x7f0f40102a30 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f48226700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4019f250 con 0x7f0f401033d0 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f48226700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4019f3e0 con 0x7f0f40101830 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f45fc2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f40101830 0x7f0f40197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f45fc2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f40101830 0x7f0f40197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58244/0 (socket says 172.21.15.186:58244) 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f45fc2700 1 -- 172.21.15.186:0/4140552318 learned_addr learned my addr 172.21.15.186:0/4140552318 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f457c1700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f40102a30 0x7f0f40197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f45fc2700 1 -- 172.21.15.186:0/4140552318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f401033d0 msgr2=0x7f0f4019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f45fc2700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f401033d0 0x7f0f4019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f45fc2700 1 -- 172.21.15.186:0/4140552318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f40102a30 msgr2=0x7f0f40197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f45fc2700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f40102a30 0x7f0f40197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f45fc2700 1 -- 172.21.15.186:0/4140552318 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f380095f0 con 0x7f0f40101830 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f45fc2700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f40101830 0x7f0f40197580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c007dd0 tx=0x7f0f3c00a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:18:12.374 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.371+0000 7f0f36ffd700 1 -- 172.21.15.186:0/4140552318 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00ad60 con 0x7f0f40101830 2024-01-31T05:18:12.375 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.372+0000 7f0f48226700 1 -- 172.21.15.186:0/4140552318 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4019f6c0 con 0x7f0f40101830 2024-01-31T05:18:12.375 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.372+0000 7f0f36ffd700 1 -- 172.21.15.186:0/4140552318 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f3c0128e0 con 0x7f0f40101830 2024-01-31T05:18:12.375 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.372+0000 7f0f36ffd700 1 -- 172.21.15.186:0/4140552318 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c0117a0 con 0x7f0f40101830 2024-01-31T05:18:12.375 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.372+0000 7f0f48226700 1 -- 172.21.15.186:0/4140552318 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4019fc10 con 0x7f0f40101830 2024-01-31T05:18:12.375 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.373+0000 7f0f48226700 1 -- 172.21.15.186:0/4140552318 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f40108b20 con 0x7f0f40101830 2024-01-31T05:18:12.376 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.374+0000 7f0f36ffd700 1 -- 172.21.15.186:0/4140552318 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f0f3c012a50 con 0x7f0f40101830 2024-01-31T05:18:12.376 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.374+0000 7f0f36ffd700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0f2c0a1370 0x7f0f2c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:18:12.376 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.374+0000 7f0f36ffd700 1 -- 172.21.15.186:0/4140552318 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f0f3c0cc960 con 0x7f0f40101830 2024-01-31T05:18:12.376 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.375+0000 7f0f457c1700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0f2c0a1370 0x7f0f2c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:18:12.376 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.375+0000 7f0f457c1700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0f2c0a1370 0x7f0f2c0a3830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0f30005950 tx=0x7f0f300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:18:12.377 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.378+0000 7f0f36ffd700 1 -- 172.21.15.186:0/4140552318 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f0f3c096920 con 0x7f0f40101830 2024-01-31T05:18:12.576 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:12.577+0000 7f0f48226700 1 -- 172.21.15.186:0/4140552318 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f4002d5a0 con 0x7f0f2c0a1370 2024-01-31T05:18:12.576 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:12 smithi186 ceph-mon[35659]: osdmap e53: 8 total, 8 up, 8 in 2024-01-31T05:18:12.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:12 smithi045 ceph-mon[31585]: osdmap e53: 8 total, 8 up, 8 in 2024-01-31T05:18:12.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:12 smithi156 ceph-mon[35720]: osdmap e53: 8 total, 8 up, 8 in 2024-01-31T05:18:13.483 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:13 smithi186 ceph-mon[35659]: osdmap e54: 8 total, 8 up, 8 in 2024-01-31T05:18:13.483 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:13 smithi186 ceph-mon[35659]: osdmap e55: 8 total, 8 up, 8 in 2024-01-31T05:18:13.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:13 smithi186 ceph-mon[35659]: pgmap v178: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:13.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:13 smithi186 ceph-mon[35659]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:18:13.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:18:13.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:18:13.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:13.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:13 smithi045 ceph-mon[31585]: osdmap e54: 8 total, 8 up, 8 in 2024-01-31T05:18:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:13 smithi045 ceph-mon[31585]: osdmap e55: 8 total, 8 up, 8 in 2024-01-31T05:18:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:13 smithi045 ceph-mon[31585]: pgmap v178: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:13 smithi045 ceph-mon[31585]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:18:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:18:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:18:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:13 smithi156 ceph-mon[35720]: osdmap e54: 8 total, 8 up, 8 in 2024-01-31T05:18:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:13 smithi156 ceph-mon[35720]: osdmap e55: 8 total, 8 up, 8 in 2024-01-31T05:18:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:13 smithi156 ceph-mon[35720]: pgmap v178: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:13 smithi156 ceph-mon[35720]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:18:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:18:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:18:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:14.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:14 smithi156 ceph-mon[35720]: osdmap e56: 8 total, 8 up, 8 in 2024-01-31T05:18:14.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:14 smithi186 ceph-mon[35659]: osdmap e56: 8 total, 8 up, 8 in 2024-01-31T05:18:14.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:14 smithi045 ceph-mon[31585]: osdmap e56: 8 total, 8 up, 8 in 2024-01-31T05:18:15.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:15 smithi156 ceph-mon[35720]: pgmap v180: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:15.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:15 smithi156 ceph-mon[35720]: from='client.? 172.21.15.186:0/3375436134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "023fed4e-b2c5-4931-9bc8-04461d8bd939"}]: dispatch 2024-01-31T05:18:15.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:15 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "023fed4e-b2c5-4931-9bc8-04461d8bd939"}]: dispatch 2024-01-31T05:18:15.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:15 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "023fed4e-b2c5-4931-9bc8-04461d8bd939"}]': finished 2024-01-31T05:18:15.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:15 smithi156 ceph-mon[35720]: osdmap e57: 9 total, 8 up, 9 in 2024-01-31T05:18:15.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:15 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:15.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:15 smithi186 ceph-mon[35659]: pgmap v180: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:15.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:15 smithi186 ceph-mon[35659]: from='client.? 172.21.15.186:0/3375436134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "023fed4e-b2c5-4931-9bc8-04461d8bd939"}]: dispatch 2024-01-31T05:18:15.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:15 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "023fed4e-b2c5-4931-9bc8-04461d8bd939"}]: dispatch 2024-01-31T05:18:15.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:15 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "023fed4e-b2c5-4931-9bc8-04461d8bd939"}]': finished 2024-01-31T05:18:15.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:15 smithi186 ceph-mon[35659]: osdmap e57: 9 total, 8 up, 9 in 2024-01-31T05:18:15.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:15 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:15.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:15 smithi045 ceph-mon[31585]: pgmap v180: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:15.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:15 smithi045 ceph-mon[31585]: from='client.? 172.21.15.186:0/3375436134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "023fed4e-b2c5-4931-9bc8-04461d8bd939"}]: dispatch 2024-01-31T05:18:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:15 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "023fed4e-b2c5-4931-9bc8-04461d8bd939"}]: dispatch 2024-01-31T05:18:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:15 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "023fed4e-b2c5-4931-9bc8-04461d8bd939"}]': finished 2024-01-31T05:18:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:15 smithi045 ceph-mon[31585]: osdmap e57: 9 total, 8 up, 9 in 2024-01-31T05:18:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:15 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:16.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:16 smithi156 ceph-mon[35720]: from='client.? 172.21.15.186:0/2543446418' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:18:16.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:16 smithi186 ceph-mon[35659]: from='client.? 172.21.15.186:0/2543446418' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:18:16.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:16 smithi045 ceph-mon[31585]: from='client.? 172.21.15.186:0/2543446418' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:18:17.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:17 smithi156 ceph-mon[35720]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:17.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:17 smithi186 ceph-mon[35659]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:17 smithi045 ceph-mon[31585]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:19.663 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:19 smithi186 ceph-mon[35659]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:19.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:19 smithi045 ceph-mon[31585]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:19.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:19 smithi156 ceph-mon[35720]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:21.824 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:21 smithi186 ceph-mon[35659]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:21.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:21 smithi045 ceph-mon[31585]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:21.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:21 smithi156 ceph-mon[35720]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:23.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:23 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-01-31T05:18:23.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:23 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:23.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:23 smithi156 ceph-mon[35720]: Deploying daemon osd.8 on smithi186 2024-01-31T05:18:23.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:23 smithi156 ceph-mon[35720]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:23.736 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:23 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-01-31T05:18:23.736 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:23 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:23.736 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:23 smithi186 ceph-mon[35659]: Deploying daemon osd.8 on smithi186 2024-01-31T05:18:23.736 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:23 smithi186 ceph-mon[35659]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:23.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-01-31T05:18:23.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:23.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:23 smithi045 ceph-mon[31585]: Deploying daemon osd.8 on smithi186 2024-01-31T05:18:23.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:23 smithi045 ceph-mon[31585]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:25.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:25 smithi045 ceph-mon[31585]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:25.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:25 smithi186 ceph-mon[35659]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:25 smithi156 ceph-mon[35720]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:27.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:27 smithi186 ceph-mon[35659]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:27.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:27 smithi045 ceph-mon[31585]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:27 smithi156 ceph-mon[35720]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:29.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:28 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:29.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:28 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:29.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:28 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:29.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:28 smithi045 ceph-mon[31585]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:29.200 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:28 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:29.201 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:28 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:29.201 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:28 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:29.201 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:28 smithi186 ceph-mon[35659]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:29.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:28 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:29.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:28 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:29.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:28 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:29.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:28 smithi156 ceph-mon[35720]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:30.891 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.892+0000 7f0f36ffd700 1 -- 172.21.15.186:0/4140552318 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0f4002d5a0 con 0x7f0f2c0a1370 2024-01-31T05:18:30.891 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 8 on host 'smithi186' 2024-01-31T05:18:30.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.894+0000 7f0f48226700 1 -- 172.21.15.186:0/4140552318 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0f2c0a1370 msgr2=0x7f0f2c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:18:30.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.894+0000 7f0f48226700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0f2c0a1370 0x7f0f2c0a3830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0f30005950 tx=0x7f0f300058e0 comp rx=0 tx=0).stop 2024-01-31T05:18:30.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.894+0000 7f0f48226700 1 -- 172.21.15.186:0/4140552318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f40101830 msgr2=0x7f0f40197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:18:30.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.894+0000 7f0f48226700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f40101830 0x7f0f40197580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c007dd0 tx=0x7f0f3c00a340 comp rx=0 tx=0).stop 2024-01-31T05:18:30.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.894+0000 7f0f48226700 1 -- 172.21.15.186:0/4140552318 shutdown_connections 2024-01-31T05:18:30.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.894+0000 7f0f48226700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f401033d0 0x7f0f4019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:30.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.894+0000 7f0f48226700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f40101830 0x7f0f40197580 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:30.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.894+0000 7f0f48226700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0f2c0a1370 0x7f0f2c0a3830 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:30.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.894+0000 7f0f48226700 1 --2- 172.21.15.186:0/4140552318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f40102a30 0x7f0f40197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:30.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.894+0000 7f0f48226700 1 -- 172.21.15.186:0/4140552318 >> 172.21.15.186:0/4140552318 conn(0x7f0f400fcdf0 msgr2=0x7f0f40105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:18:30.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.894+0000 7f0f48226700 1 -- 172.21.15.186:0/4140552318 shutdown_connections 2024-01-31T05:18:30.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:18:30.894+0000 7f0f48226700 1 -- 172.21.15.186:0/4140552318 wait complete. 2024-01-31T05:18:31.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:31 smithi186 ceph-mon[35659]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:31.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:31 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:31.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:31 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:31.873 DEBUG:teuthology.orchestra.run.smithi186:osd.8> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.8.service 2024-01-31T05:18:31.875 INFO:tasks.cephadm:Deploying osd.9 on smithi045 with /dev/vg_nvme/lv_1... 2024-01-31T05:18:31.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-01-31T05:18:31.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:31 smithi045 ceph-mon[31585]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:31.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:31 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:31.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:31 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:31.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:31 smithi156 ceph-mon[35720]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:31.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:31 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:31.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:31 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:18:32.875 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:32 smithi186 ceph-mon[35659]: from='osd.8 [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-31T05:18:32.875 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:32 smithi186 ceph-mon[35659]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-31T05:18:32.875 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:32 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:32.875 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:32 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:32.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:32 smithi045 ceph-mon[31585]: from='osd.8 [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-31T05:18:32.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:32 smithi045 ceph-mon[31585]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-31T05:18:32.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:32 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:32.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:32 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:32 smithi156 ceph-mon[35720]: from='osd.8 [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-31T05:18:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:32 smithi156 ceph-mon[35720]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-31T05:18:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:32 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:32 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:33.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:33 smithi045 ceph-mon[31585]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:33.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:33 smithi045 ceph-mon[31585]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-01-31T05:18:33.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:33 smithi045 ceph-mon[31585]: osdmap e58: 9 total, 8 up, 9 in 2024-01-31T05:18:33.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:33 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:33.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:33 smithi045 ceph-mon[31585]: from='osd.8 [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:18:33.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:33 smithi045 ceph-mon[31585]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:18:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:33 smithi156 ceph-mon[35720]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:33 smithi156 ceph-mon[35720]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-01-31T05:18:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:33 smithi156 ceph-mon[35720]: osdmap e58: 9 total, 8 up, 9 in 2024-01-31T05:18:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:33 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:33 smithi156 ceph-mon[35720]: from='osd.8 [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:18:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:33 smithi156 ceph-mon[35720]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:18:33.977 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:33 smithi186 ceph-mon[35659]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:33.977 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:33 smithi186 ceph-mon[35659]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-01-31T05:18:33.977 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:33 smithi186 ceph-mon[35659]: osdmap e58: 9 total, 8 up, 9 in 2024-01-31T05:18:33.977 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:33 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:33.977 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:33 smithi186 ceph-mon[35659]: from='osd.8 [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:18:33.977 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:33 smithi186 ceph-mon[35659]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:18:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-01-31T05:18:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-01-31T05:18:34.172 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-01-31T05:18:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-01-31T05:18:34.172 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00906278 s, 1.2 GB/s 2024-01-31T05:18:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-01-31T05:18:34.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:34 smithi186 ceph-mon[35659]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-01-31T05:18:34.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:34 smithi186 ceph-mon[35659]: osdmap e59: 9 total, 8 up, 9 in 2024-01-31T05:18:34.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:34 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:34.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:34 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:34.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:34 smithi186 ceph-mon[35659]: from='osd.8 ' entity='osd.8' 2024-01-31T05:18:34.842 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:18:34 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:18:34.482+0000 7fa3916a3700 -1 osd.8 0 waiting for initial osdmap 2024-01-31T05:18:34.843 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:18:34 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:18:34.500+0000 7fa389c96700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:18:34.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:34 smithi045 ceph-mon[31585]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-01-31T05:18:34.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:34 smithi045 ceph-mon[31585]: osdmap e59: 9 total, 8 up, 9 in 2024-01-31T05:18:34.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:34 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:34.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:34 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:34.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:34 smithi045 ceph-mon[31585]: from='osd.8 ' entity='osd.8' 2024-01-31T05:18:34.927 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2024-01-31T05:18:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:34 smithi156 ceph-mon[35720]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-01-31T05:18:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:34 smithi156 ceph-mon[35720]: osdmap e59: 9 total, 8 up, 9 in 2024-01-31T05:18:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:34 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:34 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:34 smithi156 ceph-mon[35720]: from='osd.8 ' entity='osd.8' 2024-01-31T05:18:35.207 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:18:35.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: purged_snaps scrub starts 2024-01-31T05:18:35.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: purged_snaps scrub ok 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: osd.8 [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759] boot 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: osdmap e60: 9 total, 9 up, 9 in 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:35.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: purged_snaps scrub starts 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: purged_snaps scrub ok 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: osd.8 [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759] boot 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: osdmap e60: 9 total, 9 up, 9 in 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:35.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: purged_snaps scrub starts 2024-01-31T05:18:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: purged_snaps scrub ok 2024-01-31T05:18:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-01-31T05:18:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: osd.8 [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759] boot 2024-01-31T05:18:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: osdmap e60: 9 total, 9 up, 9 in 2024-01-31T05:18:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:18:35.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:35.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:35.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:35.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:35.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:18:35.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:35.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:35.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:35.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:36.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:36 smithi045 ceph-mon[31585]: Detected new or changed devices on smithi186 2024-01-31T05:18:36.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:36 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi186 to 5693M 2024-01-31T05:18:36.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:36 smithi045 ceph-mon[31585]: osdmap e61: 9 total, 9 up, 9 in 2024-01-31T05:18:36.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:36 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:36.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:36 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:36.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:36 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:36.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:36 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:36.923 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:36 smithi186 ceph-mon[35659]: Detected new or changed devices on smithi186 2024-01-31T05:18:36.923 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:36 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi186 to 5693M 2024-01-31T05:18:36.923 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:36 smithi186 ceph-mon[35659]: osdmap e61: 9 total, 9 up, 9 in 2024-01-31T05:18:36.923 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:36 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:36.923 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:36 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:36.923 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:36 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:36.923 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:36 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:36 smithi156 ceph-mon[35720]: Detected new or changed devices on smithi186 2024-01-31T05:18:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:36 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi186 to 5693M 2024-01-31T05:18:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:36 smithi156 ceph-mon[35720]: osdmap e61: 9 total, 9 up, 9 in 2024-01-31T05:18:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:36 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:36 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:36 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:36 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:36.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.979+0000 7f8e07e26700 1 -- 172.21.15.45:0/3498460439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e00101050 msgr2=0x7f8e001014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:18:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.979+0000 7f8e07e26700 1 --2- 172.21.15.45:0/3498460439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e00101050 0x7f8e001014d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009990 tx=0x7f8df0009ca0 comp rx=0 tx=0).stop 2024-01-31T05:18:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.979+0000 7f8e07e26700 1 -- 172.21.15.45:0/3498460439 shutdown_connections 2024-01-31T05:18:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.979+0000 7f8e07e26700 1 --2- 172.21.15.45:0/3498460439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e00101a10 0x7f8e00108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.979+0000 7f8e07e26700 1 --2- 172.21.15.45:0/3498460439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e000ffe60 0x7f8e00100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.979+0000 7f8e07e26700 1 --2- 172.21.15.45:0/3498460439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e00101050 0x7f8e001014d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.979+0000 7f8e07e26700 1 -- 172.21.15.45:0/3498460439 >> 172.21.15.45:0/3498460439 conn(0x7f8e000fb430 msgr2=0x7f8e000fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:18:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.979+0000 7f8e07e26700 1 -- 172.21.15.45:0/3498460439 shutdown_connections 2024-01-31T05:18:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.979+0000 7f8e07e26700 1 -- 172.21.15.45:0/3498460439 wait complete. 2024-01-31T05:18:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.979+0000 7f8e07e26700 1 Processor -- start 2024-01-31T05:18:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.979+0000 7f8e07e26700 1 -- start start 2024-01-31T05:18:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e07e26700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e000ffe60 0x7f8e00197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:18:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e07e26700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e00101050 0x7f8e00197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e07e26700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e00101a10 0x7f8e0019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e07e26700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0019f300 con 0x7f8e00101a10 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e07e26700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0019f440 con 0x7f8e00101050 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e07e26700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0019f5d0 con 0x7f8e000ffe60 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e063c3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e00101a10 0x7f8e0019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e063c3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e00101a10 0x7f8e0019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45096/0 (socket says 172.21.15.45:45096) 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e063c3700 1 -- 172.21.15.45:0/1424942277 learned_addr learned my addr 172.21.15.45:0/1424942277 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e05bc2700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e000ffe60 0x7f8e00197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e053c1700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e00101050 0x7f8e00197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e063c3700 1 -- 172.21.15.45:0/1424942277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e000ffe60 msgr2=0x7f8e00197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e063c3700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e000ffe60 0x7f8e00197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e063c3700 1 -- 172.21.15.45:0/1424942277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e00101050 msgr2=0x7f8e00197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e063c3700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e00101050 0x7f8e00197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e063c3700 1 -- 172.21.15.45:0/1424942277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df00095f0 con 0x7f8e00101a10 2024-01-31T05:18:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e063c3700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e00101a10 0x7f8e0019ed30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8df800ef60 tx=0x7f8df800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:18:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8df6ffd700 1 -- 172.21.15.45:0/1424942277 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8018070 con 0x7f8e00101a10 2024-01-31T05:18:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8df6ffd700 1 -- 172.21.15.45:0/1424942277 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8df800f040 con 0x7f8e00101a10 2024-01-31T05:18:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8e07e26700 1 -- 172.21.15.45:0/1424942277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0019f850 con 0x7f8e00101a10 2024-01-31T05:18:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.980+0000 7f8df6ffd700 1 -- 172.21.15.45:0/1424942277 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df800cd70 con 0x7f8e00101a10 2024-01-31T05:18:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.981+0000 7f8e07e26700 1 -- 172.21.15.45:0/1424942277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0019fd50 con 0x7f8e00101a10 2024-01-31T05:18:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.981+0000 7f8e07e26700 1 -- 172.21.15.45:0/1424942277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e001073c0 con 0x7f8e00101a10 2024-01-31T05:18:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.982+0000 7f8df6ffd700 1 -- 172.21.15.45:0/1424942277 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f8df8003a00 con 0x7f8e00101a10 2024-01-31T05:18:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.983+0000 7f8df6ffd700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8dec0a13f0 0x7f8dec0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:18:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.983+0000 7f8e05bc2700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8dec0a13f0 0x7f8dec0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:18:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.983+0000 7f8df6ffd700 1 -- 172.21.15.45:0/1424942277 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f8df80cbc70 con 0x7f8e00101a10 2024-01-31T05:18:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.984+0000 7f8e05bc2700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8dec0a13f0 0x7f8dec0a38b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e00100bc0 tx=0x7f8dfc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:18:36.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:36.988+0000 7f8df6ffd700 1 -- 172.21.15.45:0/1424942277 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8df8095b10 con 0x7f8e00101a10 2024-01-31T05:18:37.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:37.199+0000 7f8e07e26700 1 -- 172.21.15.45:0/1424942277 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e0002d540 con 0x7f8dec0a13f0 2024-01-31T05:18:37.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:37 smithi186 ceph-mon[35659]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:37.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:37 smithi186 ceph-mon[35659]: osdmap e62: 9 total, 9 up, 9 in 2024-01-31T05:18:37.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:18:37.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:18:37.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:37.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:37 smithi186 ceph-mon[35659]: osdmap e63: 9 total, 9 up, 9 in 2024-01-31T05:18:37.928 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:37 smithi045 ceph-mon[31585]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:37.928 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:37 smithi045 ceph-mon[31585]: osdmap e62: 9 total, 9 up, 9 in 2024-01-31T05:18:37.928 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:18:37.928 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:18:37.929 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:37.929 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:37 smithi045 ceph-mon[31585]: osdmap e63: 9 total, 9 up, 9 in 2024-01-31T05:18:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:37 smithi156 ceph-mon[35720]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:37 smithi156 ceph-mon[35720]: osdmap e62: 9 total, 9 up, 9 in 2024-01-31T05:18:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:18:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:18:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:37 smithi156 ceph-mon[35720]: osdmap e63: 9 total, 9 up, 9 in 2024-01-31T05:18:38.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:38 smithi156 ceph-mon[35720]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:18:38.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:38 smithi156 ceph-mon[35720]: osdmap e64: 9 total, 9 up, 9 in 2024-01-31T05:18:38.991 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:38 smithi045 ceph-mon[31585]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:18:38.992 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:38 smithi045 ceph-mon[31585]: osdmap e64: 9 total, 9 up, 9 in 2024-01-31T05:18:39.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:38 smithi186 ceph-mon[35659]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:18:39.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:38 smithi186 ceph-mon[35659]: osdmap e64: 9 total, 9 up, 9 in 2024-01-31T05:18:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:39 smithi156 ceph-mon[35720]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:40.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:39 smithi186 ceph-mon[35659]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:40.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:39 smithi045 ceph-mon[31585]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:40.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:40 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3617677862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfaec4e4-25e7-40b8-a571-0b8a0bc40e75"}]: dispatch 2024-01-31T05:18:40.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:40 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3617677862' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfaec4e4-25e7-40b8-a571-0b8a0bc40e75"}]': finished 2024-01-31T05:18:40.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:40 smithi045 ceph-mon[31585]: osdmap e65: 10 total, 9 up, 10 in 2024-01-31T05:18:40.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:40 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:18:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:40 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3617677862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfaec4e4-25e7-40b8-a571-0b8a0bc40e75"}]: dispatch 2024-01-31T05:18:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:40 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3617677862' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfaec4e4-25e7-40b8-a571-0b8a0bc40e75"}]': finished 2024-01-31T05:18:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:40 smithi156 ceph-mon[35720]: osdmap e65: 10 total, 9 up, 10 in 2024-01-31T05:18:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:40 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:18:41.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:40 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3617677862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfaec4e4-25e7-40b8-a571-0b8a0bc40e75"}]: dispatch 2024-01-31T05:18:41.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:40 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3617677862' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfaec4e4-25e7-40b8-a571-0b8a0bc40e75"}]': finished 2024-01-31T05:18:41.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:40 smithi186 ceph-mon[35659]: osdmap e65: 10 total, 9 up, 10 in 2024-01-31T05:18:41.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:40 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:18:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:41 smithi156 ceph-mon[35720]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:41 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3135146603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:18:42.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:41 smithi186 ceph-mon[35659]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:42.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:41 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3135146603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:18:42.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:41 smithi045 ceph-mon[31585]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:42.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:41 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3135146603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:18:43.718 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:43 smithi045 ceph-mon[31585]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-01-31T05:18:43.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:43 smithi156 ceph-mon[35720]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-01-31T05:18:43.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:43 smithi186 ceph-mon[35659]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-01-31T05:18:45.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:45 smithi045 ceph-mon[31585]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-01-31T05:18:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:45 smithi156 ceph-mon[35720]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-01-31T05:18:46.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:45 smithi186 ceph-mon[35659]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-01-31T05:18:47.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:47 smithi045 ceph-mon[31585]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-01-31T05:18:47.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-01-31T05:18:47.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:47 smithi156 ceph-mon[35720]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-01-31T05:18:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-01-31T05:18:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:48.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:47 smithi186 ceph-mon[35659]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-01-31T05:18:48.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-01-31T05:18:48.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:48.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:48 smithi045 ceph-mon[31585]: Deploying daemon osd.9 on smithi045 2024-01-31T05:18:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:48 smithi156 ceph-mon[35720]: Deploying daemon osd.9 on smithi045 2024-01-31T05:18:49.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:48 smithi186 ceph-mon[35659]: Deploying daemon osd.9 on smithi045 2024-01-31T05:18:49.850 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:49 smithi045 ceph-mon[31585]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-31T05:18:49.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:49 smithi156 ceph-mon[35720]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-31T05:18:50.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:49 smithi186 ceph-mon[35659]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-31T05:18:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:51 smithi045 ceph-mon[31585]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-01-31T05:18:51.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:51 smithi156 ceph-mon[35720]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-01-31T05:18:52.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:51 smithi186 ceph-mon[35659]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-01-31T05:18:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:53 smithi156 ceph-mon[35720]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-31T05:18:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:53.764 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:53 smithi045 ceph-mon[31585]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-31T05:18:53.764 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:53.764 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:53.764 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:53.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:53 smithi186 ceph-mon[35659]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-31T05:18:53.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:53.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:53.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:55.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:55 smithi045 ceph-mon[31585]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:55.913 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:55 smithi186 ceph-mon[35659]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:55 smithi156 ceph-mon[35720]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:57.107 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 9 on host 'smithi045' 2024-01-31T05:18:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.107+0000 7f8df6ffd700 1 -- 172.21.15.45:0/1424942277 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8e0002d540 con 0x7f8dec0a13f0 2024-01-31T05:18:57.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.109+0000 7f8e07e26700 1 -- 172.21.15.45:0/1424942277 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8dec0a13f0 msgr2=0x7f8dec0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:18:57.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.109+0000 7f8e07e26700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8dec0a13f0 0x7f8dec0a38b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e00100bc0 tx=0x7f8dfc00a380 comp rx=0 tx=0).stop 2024-01-31T05:18:57.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.109+0000 7f8e07e26700 1 -- 172.21.15.45:0/1424942277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e00101a10 msgr2=0x7f8e0019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:18:57.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.109+0000 7f8e07e26700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e00101a10 0x7f8e0019ed30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8df800ef60 tx=0x7f8df800ef90 comp rx=0 tx=0).stop 2024-01-31T05:18:57.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.110+0000 7f8e07e26700 1 -- 172.21.15.45:0/1424942277 shutdown_connections 2024-01-31T05:18:57.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.110+0000 7f8e07e26700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8dec0a13f0 0x7f8dec0a38b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:57.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.110+0000 7f8e07e26700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e00101a10 0x7f8e0019ed30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:57.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.110+0000 7f8e07e26700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e000ffe60 0x7f8e00197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:57.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.110+0000 7f8e07e26700 1 --2- 172.21.15.45:0/1424942277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e00101050 0x7f8e00197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:18:57.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.110+0000 7f8e07e26700 1 -- 172.21.15.45:0/1424942277 >> 172.21.15.45:0/1424942277 conn(0x7f8e000fb430 msgr2=0x7f8e00104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:18:57.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.110+0000 7f8e07e26700 1 -- 172.21.15.45:0/1424942277 shutdown_connections 2024-01-31T05:18:57.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:18:57.110+0000 7f8e07e26700 1 -- 172.21.15.45:0/1424942277 wait complete. 2024-01-31T05:18:57.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:57 smithi045 ceph-mon[31585]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:57 smithi045 ceph-mon[31585]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-01-31T05:18:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:57 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:57.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:57.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:57.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:57 smithi156 ceph-mon[35720]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:57.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:57 smithi156 ceph-mon[35720]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-01-31T05:18:57.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:57 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:57.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:18:57.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:18:57.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:57 smithi186 ceph-mon[35659]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:57.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:57 smithi186 ceph-mon[35659]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-01-31T05:18:57.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:18:57.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:57 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:18:57.695 DEBUG:teuthology.orchestra.run.smithi045:osd.9> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.9.service 2024-01-31T05:18:57.697 INFO:tasks.cephadm:Deploying osd.10 on smithi156 with /dev/vg_nvme/lv_1... 2024-01-31T05:18:57.698 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-01-31T05:18:57.944 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.b/config 2024-01-31T05:18:58.165 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:58 smithi045 ceph-mon[31585]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-01-31T05:18:58.165 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:58 smithi045 ceph-mon[31585]: osdmap e66: 10 total, 9 up, 10 in 2024-01-31T05:18:58.165 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:58 smithi045 ceph-mon[31585]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-01-31T05:18:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:58 smithi156 ceph-mon[35720]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-01-31T05:18:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:58 smithi156 ceph-mon[35720]: osdmap e66: 10 total, 9 up, 10 in 2024-01-31T05:18:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:58 smithi156 ceph-mon[35720]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-01-31T05:18:58.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:58 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:18:58.454 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:58 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:18:58.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:58 smithi186 ceph-mon[35659]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-01-31T05:18:58.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:58 smithi186 ceph-mon[35659]: osdmap e66: 10 total, 9 up, 10 in 2024-01-31T05:18:58.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:58 smithi186 ceph-mon[35659]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-01-31T05:18:58.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:58 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:18:59.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:18:58 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:18:58.864+0000 7f9f68a11700 -1 osd.9 0 waiting for initial osdmap 2024-01-31T05:18:59.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:18:58 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:18:58.878+0000 7f9f61004700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:18:59.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:59 smithi156 ceph-mon[35720]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-01-31T05:18:59.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:59 smithi156 ceph-mon[35720]: osdmap e67: 10 total, 9 up, 10 in 2024-01-31T05:18:59.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:18:59.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:59 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:18:59.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:59 smithi156 ceph-mon[35720]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:59.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:18:59 smithi156 ceph-mon[35720]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' 2024-01-31T05:18:59.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:59 smithi186 ceph-mon[35659]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-01-31T05:18:59.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:59 smithi186 ceph-mon[35659]: osdmap e67: 10 total, 9 up, 10 in 2024-01-31T05:18:59.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:18:59.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:59 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:18:59.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:59 smithi186 ceph-mon[35659]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:59.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:18:59 smithi186 ceph-mon[35659]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' 2024-01-31T05:18:59.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:59 smithi045 ceph-mon[31585]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-01-31T05:18:59.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:59 smithi045 ceph-mon[31585]: osdmap e67: 10 total, 9 up, 10 in 2024-01-31T05:18:59.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:18:59.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:59 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:18:59.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:59 smithi045 ceph-mon[31585]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-01-31T05:18:59.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:18:59 smithi045 ceph-mon[31585]: from='osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931]' entity='osd.9' 2024-01-31T05:18:59.792 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-01-31T05:18:59.792 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-01-31T05:18:59.792 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-01-31T05:18:59.793 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-01-31T05:18:59.793 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00858197 s, 1.2 GB/s 2024-01-31T05:18:59.793 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-01-31T05:19:00.253 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:00 smithi045 ceph-mon[31585]: purged_snaps scrub starts 2024-01-31T05:19:00.253 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:00 smithi045 ceph-mon[31585]: purged_snaps scrub ok 2024-01-31T05:19:00.253 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:00 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:19:00.254 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:00 smithi045 ceph-mon[31585]: osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931] boot 2024-01-31T05:19:00.254 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:00 smithi045 ceph-mon[31585]: osdmap e68: 10 total, 10 up, 10 in 2024-01-31T05:19:00.254 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:00 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:19:00.374 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:00 smithi156 ceph-mon[35720]: purged_snaps scrub starts 2024-01-31T05:19:00.375 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:00 smithi156 ceph-mon[35720]: purged_snaps scrub ok 2024-01-31T05:19:00.375 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:00 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:19:00.375 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:00 smithi156 ceph-mon[35720]: osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931] boot 2024-01-31T05:19:00.375 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:00 smithi156 ceph-mon[35720]: osdmap e68: 10 total, 10 up, 10 in 2024-01-31T05:19:00.375 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:00 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:19:00.401 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_1 2024-01-31T05:19:00.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:00 smithi186 ceph-mon[35659]: purged_snaps scrub starts 2024-01-31T05:19:00.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:00 smithi186 ceph-mon[35659]: purged_snaps scrub ok 2024-01-31T05:19:00.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:00 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:19:00.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:00 smithi186 ceph-mon[35659]: osd.9 [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931] boot 2024-01-31T05:19:00.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:00 smithi186 ceph-mon[35659]: osdmap e68: 10 total, 10 up, 10 in 2024-01-31T05:19:00.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:00 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:19:00.651 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.b/config 2024-01-31T05:19:01.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:01.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: Detected new or changed devices on smithi045 2024-01-31T05:19:01.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:01.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:01.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:01.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:01.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:01.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:01.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:01.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:01.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:01.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:01.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:01 smithi156 ceph-mon[35720]: osdmap e69: 10 total, 10 up, 10 in 2024-01-31T05:19:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: Detected new or changed devices on smithi045 2024-01-31T05:19:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:02.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:01 smithi186 ceph-mon[35659]: osdmap e69: 10 total, 10 up, 10 in 2024-01-31T05:19:02.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: Detected new or changed devices on smithi045 2024-01-31T05:19:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:01 smithi045 ceph-mon[31585]: osdmap e69: 10 total, 10 up, 10 in 2024-01-31T05:19:02.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.413+0000 7f01aa91d700 1 -- 172.21.15.156:0/4244074312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01a4101810 msgr2=0x7f01a4101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:02.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.413+0000 7f01aa91d700 1 --2- 172.21.15.156:0/4244074312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01a4101810 0x7f01a4101c30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f01980099c0 tx=0x7f0198009cd0 comp rx=0 tx=0).stop 2024-01-31T05:19:02.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.413+0000 7f01aa91d700 1 -- 172.21.15.156:0/4244074312 shutdown_connections 2024-01-31T05:19:02.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.413+0000 7f01aa91d700 1 --2- 172.21.15.156:0/4244074312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a41033b0 0x7f01a4109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:02.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.413+0000 7f01aa91d700 1 --2- 172.21.15.156:0/4244074312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01a4101810 0x7f01a4101c30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:02.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.413+0000 7f01aa91d700 1 --2- 172.21.15.156:0/4244074312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a4102a10 0x7f01a4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.413+0000 7f01aa91d700 1 -- 172.21.15.156:0/4244074312 >> 172.21.15.156:0/4244074312 conn(0x7f01a40fcdf0 msgr2=0x7f01a40ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.413+0000 7f01aa91d700 1 -- 172.21.15.156:0/4244074312 shutdown_connections 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.413+0000 7f01aa91d700 1 -- 172.21.15.156:0/4244074312 wait complete. 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01aa91d700 1 Processor -- start 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01aa91d700 1 -- start start 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01aa91d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01a4101810 0x7f01a41977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01aa91d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a4102a10 0x7f01a4197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01aa91d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a41033b0 0x7f01a419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01aa91d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a419f350 con 0x7f01a4102a10 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01aa91d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a419f490 con 0x7f01a4101810 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01aa91d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a419f620 con 0x7f01a41033b0 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01a4101810 0x7f01a41977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01a4101810 0x7f01a41977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:55600/0 (socket says 172.21.15.156:55600) 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01a3fff700 1 -- 172.21.15.156:0/3493452160 learned_addr learned my addr 172.21.15.156:0/3493452160 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-01-31T05:19:02.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01a8eba700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a41033b0 0x7f01a419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.414+0000 7f01a37fe700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a4102a10 0x7f01a4197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.415+0000 7f01a3fff700 1 -- 172.21.15.156:0/3493452160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a41033b0 msgr2=0x7f01a419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.415+0000 7f01a3fff700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a41033b0 0x7f01a419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.415+0000 7f01a3fff700 1 -- 172.21.15.156:0/3493452160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a4102a10 msgr2=0x7f01a4197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.415+0000 7f01a3fff700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a4102a10 0x7f01a4197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.415+0000 7f01a3fff700 1 -- 172.21.15.156:0/3493452160 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01980095f0 con 0x7f01a4101810 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.415+0000 7f01a8eba700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a41033b0 0x7f01a419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.415+0000 7f01a3fff700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01a4101810 0x7f01a41977c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0198009f50 tx=0x7f01980047a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.415+0000 7f01a17fa700 1 -- 172.21.15.156:0/3493452160 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019801d070 con 0x7f01a4101810 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.415+0000 7f01aa91d700 1 -- 172.21.15.156:0/3493452160 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a419f8a0 con 0x7f01a4101810 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.415+0000 7f01a17fa700 1 -- 172.21.15.156:0/3493452160 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0198004200 con 0x7f01a4101810 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.415+0000 7f01a17fa700 1 -- 172.21.15.156:0/3493452160 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019800f460 con 0x7f01a4101810 2024-01-31T05:19:02.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.415+0000 7f01aa91d700 1 -- 172.21.15.156:0/3493452160 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a419fdc0 con 0x7f01a4101810 2024-01-31T05:19:02.417 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.416+0000 7f01aa91d700 1 -- 172.21.15.156:0/3493452160 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a4108ca0 con 0x7f01a4101810 2024-01-31T05:19:02.417 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.417+0000 7f01a17fa700 1 -- 172.21.15.156:0/3493452160 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f0198003d40 con 0x7f01a4101810 2024-01-31T05:19:02.417 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.418+0000 7f01a17fa700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f018c0a1370 0x7f018c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:02.418 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.418+0000 7f01a17fa700 1 -- 172.21.15.156:0/3493452160 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f01980cdb80 con 0x7f01a4101810 2024-01-31T05:19:02.418 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.418+0000 7f01a37fe700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f018c0a1370 0x7f018c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:02.418 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.419+0000 7f01a37fe700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f018c0a1370 0x7f018c0a3830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0194009a10 tx=0x7f0194009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:19:02.420 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.422+0000 7f01a17fa700 1 -- 172.21.15.156:0/3493452160 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f019809b320 con 0x7f01a4101810 2024-01-31T05:19:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:02.620+0000 7f01aa91d700 1 -- 172.21.15.156:0/3493452160 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f01a402d5a0 con 0x7f018c0a1370 2024-01-31T05:19:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:02 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi045 to 4273M 2024-01-31T05:19:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:19:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:19:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:03.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:02 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi045 to 4273M 2024-01-31T05:19:03.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:19:03.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:19:03.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:03.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:02 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi045 to 4273M 2024-01-31T05:19:03.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:19:03.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:19:03.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:03.833 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:03 smithi156 ceph-mon[35720]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:03.833 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:03 smithi156 ceph-mon[35720]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:19:04.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:03 smithi186 ceph-mon[35659]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:04.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:03 smithi186 ceph-mon[35659]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:19:04.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:03 smithi045 ceph-mon[31585]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:04.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:03 smithi045 ceph-mon[31585]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:19:05.944 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:05 smithi045 ceph-mon[31585]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:05.945 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:05 smithi045 ceph-mon[31585]: from='client.? 172.21.15.156:0/3616931358' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc"}]: dispatch 2024-01-31T05:19:05.945 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:05 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc"}]: dispatch 2024-01-31T05:19:05.945 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:05 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc"}]': finished 2024-01-31T05:19:05.945 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:05 smithi045 ceph-mon[31585]: osdmap e70: 11 total, 10 up, 11 in 2024-01-31T05:19:05.945 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:05 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:05.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:05 smithi156 ceph-mon[35720]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:05.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:05 smithi156 ceph-mon[35720]: from='client.? 172.21.15.156:0/3616931358' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc"}]: dispatch 2024-01-31T05:19:05.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:05 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc"}]: dispatch 2024-01-31T05:19:05.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:05 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc"}]': finished 2024-01-31T05:19:05.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:05 smithi156 ceph-mon[35720]: osdmap e70: 11 total, 10 up, 11 in 2024-01-31T05:19:05.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:05 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:06.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:05 smithi186 ceph-mon[35659]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:06.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:05 smithi186 ceph-mon[35659]: from='client.? 172.21.15.156:0/3616931358' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc"}]: dispatch 2024-01-31T05:19:06.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:05 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc"}]: dispatch 2024-01-31T05:19:06.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:05 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc"}]': finished 2024-01-31T05:19:06.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:05 smithi186 ceph-mon[35659]: osdmap e70: 11 total, 10 up, 11 in 2024-01-31T05:19:06.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:05 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:06.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:06 smithi156 ceph-mon[35720]: from='client.? 172.21.15.156:0/4273000162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:19:06.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:06 smithi156 ceph-mon[35720]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:07.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:06 smithi186 ceph-mon[35659]: from='client.? 172.21.15.156:0/4273000162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:19:07.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:06 smithi186 ceph-mon[35659]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:07.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:06 smithi045 ceph-mon[31585]: from='client.? 172.21.15.156:0/4273000162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:19:07.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:06 smithi045 ceph-mon[31585]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:09.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:09 smithi045 ceph-mon[31585]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:09.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:09 smithi156 ceph-mon[35720]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:10.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:09 smithi186 ceph-mon[35659]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:11.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:11 smithi045 ceph-mon[31585]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:11 smithi156 ceph-mon[35720]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:12.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:11 smithi186 ceph-mon[35659]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-01-31T05:19:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:13 smithi156 ceph-mon[35720]: Deploying daemon osd.10 on smithi156 2024-01-31T05:19:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:13 smithi156 ceph-mon[35720]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-01-31T05:19:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:13 smithi186 ceph-mon[35659]: Deploying daemon osd.10 on smithi156 2024-01-31T05:19:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:13 smithi186 ceph-mon[35659]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:13.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-01-31T05:19:13.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:13.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:13 smithi045 ceph-mon[31585]: Deploying daemon osd.10 on smithi156 2024-01-31T05:19:13.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:13 smithi045 ceph-mon[31585]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:15.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:15 smithi045 ceph-mon[31585]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:15 smithi156 ceph-mon[35720]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:16.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:15 smithi186 ceph-mon[35659]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:17.888 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:17 smithi156 ceph-mon[35720]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:17.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:17 smithi045 ceph-mon[31585]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:18.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:17 smithi186 ceph-mon[35659]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:19.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:18 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:19.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:18 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:19.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:18 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:19:19.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:18 smithi156 ceph-mon[35720]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:19.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:18 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:19.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:18 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:19.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:18 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:19:19.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:18 smithi186 ceph-mon[35659]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:19.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:18 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:19.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:18 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:19.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:18 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:19:19.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:18 smithi045 ceph-mon[31585]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:21.655 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:21 smithi156 ceph-mon[35720]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:21.655 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:21.655 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:21.655 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:21.655 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:21.655 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:21.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.736+0000 7f01a17fa700 1 -- 172.21.15.156:0/3493452160 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f01a402d5a0 con 0x7f018c0a1370 2024-01-31T05:19:21.736 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 10 on host 'smithi156' 2024-01-31T05:19:21.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.739+0000 7f01aa91d700 1 -- 172.21.15.156:0/3493452160 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f018c0a1370 msgr2=0x7f018c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:21.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.739+0000 7f01aa91d700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f018c0a1370 0x7f018c0a3830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0194009a10 tx=0x7f0194009190 comp rx=0 tx=0).stop 2024-01-31T05:19:21.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.739+0000 7f01aa91d700 1 -- 172.21.15.156:0/3493452160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01a4101810 msgr2=0x7f01a41977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:21.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.739+0000 7f01aa91d700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01a4101810 0x7f01a41977c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0198009f50 tx=0x7f01980047a0 comp rx=0 tx=0).stop 2024-01-31T05:19:21.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.739+0000 7f01aa91d700 1 -- 172.21.15.156:0/3493452160 shutdown_connections 2024-01-31T05:19:21.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.739+0000 7f01aa91d700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a41033b0 0x7f01a419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:21.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.739+0000 7f01aa91d700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01a4101810 0x7f01a41977c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:21.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.739+0000 7f01aa91d700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f018c0a1370 0x7f018c0a3830 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:21.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.739+0000 7f01aa91d700 1 --2- 172.21.15.156:0/3493452160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a4102a10 0x7f01a4197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:21.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.739+0000 7f01aa91d700 1 -- 172.21.15.156:0/3493452160 >> 172.21.15.156:0/3493452160 conn(0x7f01a40fcdf0 msgr2=0x7f01a4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:19:21.739 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.739+0000 7f01aa91d700 1 -- 172.21.15.156:0/3493452160 shutdown_connections 2024-01-31T05:19:21.739 INFO:teuthology.orchestra.run.smithi156.stderr:2024-01-31T05:19:21.739+0000 7f01aa91d700 1 -- 172.21.15.156:0/3493452160 wait complete. 2024-01-31T05:19:21.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:21 smithi045 ceph-mon[31585]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:21.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:21.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:21.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:21.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:21.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:22.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:21 smithi186 ceph-mon[35659]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:22.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:22.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:22.418 DEBUG:teuthology.orchestra.run.smithi156:osd.10> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.10.service 2024-01-31T05:19:22.420 INFO:tasks.cephadm:Deploying osd.11 on smithi186 with /dev/vg_nvme/lv_1... 2024-01-31T05:19:22.421 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-01-31T05:19:22.671 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.c/config 2024-01-31T05:19:22.794 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:22.794 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:22.794 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:19:22.794 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:22 smithi186 ceph-mon[35659]: from='osd.10 [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-31T05:19:22.794 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:22 smithi186 ceph-mon[35659]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-31T05:19:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:19:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:22 smithi156 ceph-mon[35720]: from='osd.10 [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-31T05:19:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:22 smithi156 ceph-mon[35720]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-31T05:19:23.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:23.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:23.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:19:23.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:22 smithi045 ceph-mon[31585]: from='osd.10 [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-31T05:19:23.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:22 smithi045 ceph-mon[31585]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-31T05:19:23.942 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:23 smithi156 ceph-mon[35720]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:23.942 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:23 smithi156 ceph-mon[35720]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-01-31T05:19:23.943 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:23 smithi156 ceph-mon[35720]: osdmap e71: 11 total, 10 up, 11 in 2024-01-31T05:19:23.943 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:23 smithi156 ceph-mon[35720]: from='osd.10 [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:19:23.943 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:23 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:23.943 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:23 smithi156 ceph-mon[35720]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:19:23.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:23 smithi186 ceph-mon[35659]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:23.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:23 smithi186 ceph-mon[35659]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-01-31T05:19:23.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:23 smithi186 ceph-mon[35659]: osdmap e71: 11 total, 10 up, 11 in 2024-01-31T05:19:23.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:23 smithi186 ceph-mon[35659]: from='osd.10 [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:19:23.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:23 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:23.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:23 smithi186 ceph-mon[35659]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:19:24.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:23 smithi045 ceph-mon[31585]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:23 smithi045 ceph-mon[31585]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-01-31T05:19:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:23 smithi045 ceph-mon[31585]: osdmap e71: 11 total, 10 up, 11 in 2024-01-31T05:19:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:23 smithi045 ceph-mon[31585]: from='osd.10 [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:19:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:23 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:23 smithi045 ceph-mon[31585]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-01-31T05:19:24.567 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-01-31T05:19:24.567 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-01-31T05:19:24.567 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-01-31T05:19:24.567 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-01-31T05:19:24.567 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134255 s, 781 MB/s 2024-01-31T05:19:24.567 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-01-31T05:19:24.706 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:19:24 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:19:24.389+0000 7fc56fd30700 -1 osd.10 0 waiting for initial osdmap 2024-01-31T05:19:24.706 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:19:24 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:19:24.404+0000 7fc566b20700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:19:24.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:24 smithi186 ceph-mon[35659]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-01-31T05:19:24.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:24 smithi186 ceph-mon[35659]: osdmap e72: 11 total, 10 up, 11 in 2024-01-31T05:19:24.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:24 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:24.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:24 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:24.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:24 smithi186 ceph-mon[35659]: from='osd.10 ' entity='osd.10' 2024-01-31T05:19:24.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:24 smithi186 ceph-mon[35659]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:24 smithi156 ceph-mon[35720]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-01-31T05:19:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:24 smithi156 ceph-mon[35720]: osdmap e72: 11 total, 10 up, 11 in 2024-01-31T05:19:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:24 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:24 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:24 smithi156 ceph-mon[35720]: from='osd.10 ' entity='osd.10' 2024-01-31T05:19:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:24 smithi156 ceph-mon[35720]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:25.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:24 smithi045 ceph-mon[31585]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-01-31T05:19:25.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:24 smithi045 ceph-mon[31585]: osdmap e72: 11 total, 10 up, 11 in 2024-01-31T05:19:25.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:24 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:25.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:24 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:25.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:24 smithi045 ceph-mon[31585]: from='osd.10 ' entity='osd.10' 2024-01-31T05:19:25.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:24 smithi045 ceph-mon[31585]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-01-31T05:19:25.314 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch daemon add osd smithi186:vg_nvme/lv_1 2024-01-31T05:19:25.556 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.c/config 2024-01-31T05:19:25.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: purged_snaps scrub starts 2024-01-31T05:19:25.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: purged_snaps scrub ok 2024-01-31T05:19:25.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: osd.10 [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526] boot 2024-01-31T05:19:25.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: osdmap e73: 11 total, 11 up, 11 in 2024-01-31T05:19:25.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:25.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: Detected new or changed devices on smithi156 2024-01-31T05:19:25.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:25.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:25.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:25.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:25.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:25.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:25.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi156 to 4270M 2024-01-31T05:19:25.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:25.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:25.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:25.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: purged_snaps scrub starts 2024-01-31T05:19:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: purged_snaps scrub ok 2024-01-31T05:19:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: osd.10 [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526] boot 2024-01-31T05:19:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: osdmap e73: 11 total, 11 up, 11 in 2024-01-31T05:19:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: Detected new or changed devices on smithi156 2024-01-31T05:19:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:25.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:25.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:25.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:25.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:25.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:25.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi156 to 4270M 2024-01-31T05:19:25.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:25.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:25.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:25.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: purged_snaps scrub starts 2024-01-31T05:19:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: purged_snaps scrub ok 2024-01-31T05:19:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: osd.10 [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526] boot 2024-01-31T05:19:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: osdmap e73: 11 total, 11 up, 11 in 2024-01-31T05:19:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:19:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: Detected new or changed devices on smithi156 2024-01-31T05:19:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:26.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:26.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi156 to 4270M 2024-01-31T05:19:26.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:26.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:26.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:26.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:26.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:26 smithi186 ceph-mon[35659]: osdmap e74: 11 total, 11 up, 11 in 2024-01-31T05:19:26.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:26 smithi186 ceph-mon[35659]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:26 smithi156 ceph-mon[35720]: osdmap e74: 11 total, 11 up, 11 in 2024-01-31T05:19:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:26 smithi156 ceph-mon[35720]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:27.037 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.034+0000 7f576f1c2700 1 -- 172.21.15.186:0/2696523543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100710 msgr2=0x7f5768100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:27.037 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.034+0000 7f576f1c2700 1 --2- 172.21.15.186:0/2696523543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100710 0x7f5768100b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f575c009990 tx=0x7f575c009ca0 comp rx=0 tx=0).stop 2024-01-31T05:19:27.037 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.034+0000 7f576f1c2700 1 -- 172.21.15.186:0/2696523543 shutdown_connections 2024-01-31T05:19:27.037 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.034+0000 7f576f1c2700 1 --2- 172.21.15.186:0/2696523543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57681022b0 0x7f5768108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:27.037 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.034+0000 7f576f1c2700 1 --2- 172.21.15.186:0/2696523543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100710 0x7f5768100b30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:27.037 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.034+0000 7f576f1c2700 1 --2- 172.21.15.186:0/2696523543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5768101910 0x7f5768101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:27.037 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.034+0000 7f576f1c2700 1 -- 172.21.15.186:0/2696523543 >> 172.21.15.186:0/2696523543 conn(0x7f57680fbcd0 msgr2=0x7f57680fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:19:27.038 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.034+0000 7f576f1c2700 1 -- 172.21.15.186:0/2696523543 shutdown_connections 2024-01-31T05:19:27.038 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.034+0000 7f576f1c2700 1 -- 172.21.15.186:0/2696523543 wait complete. 2024-01-31T05:19:27.038 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.034+0000 7f576f1c2700 1 Processor -- start 2024-01-31T05:19:27.038 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.034+0000 7f576f1c2700 1 -- start start 2024-01-31T05:19:27.038 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.034+0000 7f576f1c2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100710 0x7f576806aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:27.038 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576f1c2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5768101910 0x7f576819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:27.038 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576f1c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57681022b0 0x7f576819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:27.038 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576f1c2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576819f280 con 0x7f57681022b0 2024-01-31T05:19:27.038 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576f1c2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576819f3f0 con 0x7f5768101910 2024-01-31T05:19:27.038 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576f1c2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576819f5b0 con 0x7f5768100710 2024-01-31T05:19:27.038 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576cf5e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100710 0x7f576806aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:27.039 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576cf5e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100710 0x7f576806aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47854/0 (socket says 172.21.15.186:47854) 2024-01-31T05:19:27.039 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576cf5e700 1 -- 172.21.15.186:0/436688633 learned_addr learned my addr 172.21.15.186:0/436688633 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-01-31T05:19:27.039 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576d75f700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57681022b0 0x7f576819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:27.039 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f5767fff700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5768101910 0x7f576819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:27.039 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576cf5e700 1 -- 172.21.15.186:0/436688633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5768101910 msgr2=0x7f576819c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:27.039 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576cf5e700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5768101910 0x7f576819c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:27.039 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576cf5e700 1 -- 172.21.15.186:0/436688633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57681022b0 msgr2=0x7f576819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:27.039 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576cf5e700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57681022b0 0x7f576819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:27.039 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576cf5e700 1 -- 172.21.15.186:0/436688633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c0095f0 con 0x7f5768100710 2024-01-31T05:19:27.039 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f5767fff700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5768101910 0x7f576819c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:19:27.039 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576d75f700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57681022b0 0x7f576819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:19:27.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576cf5e700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100710 0x7f576806aa70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f575c000c00 tx=0x7f575c00f720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:19:27.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f5765ffb700 1 -- 172.21.15.186:0/436688633 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c01d070 con 0x7f5768100710 2024-01-31T05:19:27.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576f1c2700 1 -- 172.21.15.186:0/436688633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576819f830 con 0x7f5768100710 2024-01-31T05:19:27.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f5765ffb700 1 -- 172.21.15.186:0/436688633 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f575c00fd00 con 0x7f5768100710 2024-01-31T05:19:27.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f5765ffb700 1 -- 172.21.15.186:0/436688633 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0176c0 con 0x7f5768100710 2024-01-31T05:19:27.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.035+0000 7f576f1c2700 1 -- 172.21.15.186:0/436688633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576819fcd0 con 0x7f5768100710 2024-01-31T05:19:27.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.037+0000 7f576f1c2700 1 -- 172.21.15.186:0/436688633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5768107bf0 con 0x7f5768100710 2024-01-31T05:19:27.040 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.038+0000 7f5765ffb700 1 -- 172.21.15.186:0/436688633 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f575c022470 con 0x7f5768100710 2024-01-31T05:19:27.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.038+0000 7f5765ffb700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f57580a1370 0x7f57580a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:27.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.039+0000 7f5765ffb700 1 -- 172.21.15.186:0/436688633 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f575c0cddd0 con 0x7f5768100710 2024-01-31T05:19:27.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.039+0000 7f5767fff700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f57580a1370 0x7f57580a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:27.041 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.039+0000 7f5767fff700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f57580a1370 0x7f57580a3830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5754005fd0 tx=0x7f5754005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:19:27.042 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.042+0000 7f5765ffb700 1 -- 172.21.15.186:0/436688633 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f575c097910 con 0x7f5768100710 2024-01-31T05:19:27.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:26 smithi045 ceph-mon[31585]: osdmap e74: 11 total, 11 up, 11 in 2024-01-31T05:19:27.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:26 smithi045 ceph-mon[31585]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:27.243 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:27.243+0000 7f576f1c2700 1 -- 172.21.15.186:0/436688633 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f576802d570 con 0x7f57580a1370 2024-01-31T05:19:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:27 smithi156 ceph-mon[35720]: osdmap e75: 11 total, 11 up, 11 in 2024-01-31T05:19:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:27 smithi156 ceph-mon[35720]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:19:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:27 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:19:27.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:27 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:19:27.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:27 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:27.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:27 smithi156 ceph-mon[35720]: osdmap e76: 11 total, 11 up, 11 in 2024-01-31T05:19:28.055 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:27 smithi186 ceph-mon[35659]: osdmap e75: 11 total, 11 up, 11 in 2024-01-31T05:19:28.055 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:27 smithi186 ceph-mon[35659]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:19:28.055 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:27 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:19:28.056 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:27 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:19:28.056 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:27 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:28.056 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:27 smithi186 ceph-mon[35659]: osdmap e76: 11 total, 11 up, 11 in 2024-01-31T05:19:28.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:27 smithi045 ceph-mon[31585]: osdmap e75: 11 total, 11 up, 11 in 2024-01-31T05:19:28.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:27 smithi045 ceph-mon[31585]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:19:28.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:27 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-31T05:19:28.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:27 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-31T05:19:28.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:27 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:28.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:27 smithi045 ceph-mon[31585]: osdmap e76: 11 total, 11 up, 11 in 2024-01-31T05:19:29.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:29 smithi156 ceph-mon[35720]: osdmap e77: 11 total, 11 up, 11 in 2024-01-31T05:19:29.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:29 smithi156 ceph-mon[35720]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:29.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:29 smithi186 ceph-mon[35659]: osdmap e77: 11 total, 11 up, 11 in 2024-01-31T05:19:29.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:29 smithi186 ceph-mon[35659]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:29.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:29 smithi045 ceph-mon[31585]: osdmap e77: 11 total, 11 up, 11 in 2024-01-31T05:19:29.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:29 smithi045 ceph-mon[31585]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:30.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:30 smithi156 ceph-mon[35720]: from='client.? 172.21.15.186:0/1560217002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26f7c8a-679e-43d2-b923-01f3358ffcc7"}]: dispatch 2024-01-31T05:19:30.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:30 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26f7c8a-679e-43d2-b923-01f3358ffcc7"}]: dispatch 2024-01-31T05:19:30.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:30 smithi156 ceph-mon[35720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26f7c8a-679e-43d2-b923-01f3358ffcc7"}]': finished 2024-01-31T05:19:30.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:30 smithi156 ceph-mon[35720]: osdmap e78: 12 total, 11 up, 12 in 2024-01-31T05:19:30.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:30 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:30.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:30 smithi186 ceph-mon[35659]: from='client.? 172.21.15.186:0/1560217002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26f7c8a-679e-43d2-b923-01f3358ffcc7"}]: dispatch 2024-01-31T05:19:30.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:30 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26f7c8a-679e-43d2-b923-01f3358ffcc7"}]: dispatch 2024-01-31T05:19:30.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:30 smithi186 ceph-mon[35659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26f7c8a-679e-43d2-b923-01f3358ffcc7"}]': finished 2024-01-31T05:19:30.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:30 smithi186 ceph-mon[35659]: osdmap e78: 12 total, 11 up, 12 in 2024-01-31T05:19:30.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:30 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:30.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:30 smithi045 ceph-mon[31585]: from='client.? 172.21.15.186:0/1560217002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26f7c8a-679e-43d2-b923-01f3358ffcc7"}]: dispatch 2024-01-31T05:19:30.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:30 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26f7c8a-679e-43d2-b923-01f3358ffcc7"}]: dispatch 2024-01-31T05:19:30.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:30 smithi045 ceph-mon[31585]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26f7c8a-679e-43d2-b923-01f3358ffcc7"}]': finished 2024-01-31T05:19:30.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:30 smithi045 ceph-mon[31585]: osdmap e78: 12 total, 11 up, 12 in 2024-01-31T05:19:30.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:30 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:31.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:31 smithi156 ceph-mon[35720]: from='client.? 172.21.15.186:0/1768722114' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:19:31.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:31 smithi156 ceph-mon[35720]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:31.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:31 smithi186 ceph-mon[35659]: from='client.? 172.21.15.186:0/1768722114' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:19:31.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:31 smithi186 ceph-mon[35659]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:31.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:31 smithi045 ceph-mon[31585]: from='client.? 172.21.15.186:0/1768722114' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-31T05:19:31.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:31 smithi045 ceph-mon[31585]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:33.713 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:33 smithi186 ceph-mon[35659]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-01-31T05:19:33.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:33 smithi156 ceph-mon[35720]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-01-31T05:19:33.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:33 smithi045 ceph-mon[31585]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-01-31T05:19:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:35 smithi045 ceph-mon[31585]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-01-31T05:19:35.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:35 smithi186 ceph-mon[35659]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-01-31T05:19:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:35 smithi156 ceph-mon[35720]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-01-31T05:19:37.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:37 smithi186 ceph-mon[35659]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-01-31T05:19:37.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-01-31T05:19:37.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:37.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:37 smithi045 ceph-mon[31585]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-01-31T05:19:37.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-01-31T05:19:37.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:37 smithi156 ceph-mon[35720]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-01-31T05:19:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-01-31T05:19:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:38.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:38 smithi045 ceph-mon[31585]: Deploying daemon osd.11 on smithi186 2024-01-31T05:19:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:38 smithi156 ceph-mon[35720]: Deploying daemon osd.11 on smithi186 2024-01-31T05:19:38.978 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:38 smithi186 ceph-mon[35659]: Deploying daemon osd.11 on smithi186 2024-01-31T05:19:39.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:39 smithi045 ceph-mon[31585]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-31T05:19:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:39 smithi156 ceph-mon[35720]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-31T05:19:40.008 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:39 smithi186 ceph-mon[35659]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-31T05:19:41.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:41 smithi045 ceph-mon[31585]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-01-31T05:19:41.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:41 smithi156 ceph-mon[35720]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-01-31T05:19:42.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:41 smithi186 ceph-mon[35659]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-01-31T05:19:43.646 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:43 smithi186 ceph-mon[35659]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-31T05:19:43.646 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:43 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:43.646 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:43 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:43.646 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:43 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:19:43.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:43 smithi156 ceph-mon[35720]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-31T05:19:43.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:43 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:43.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:43 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:43.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:43 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:19:43.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:43 smithi045 ceph-mon[31585]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-31T05:19:43.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:43 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:43.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:43 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:43.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:43 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:19:45.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:45 smithi186 ceph-mon[35659]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:45.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:45 smithi045 ceph-mon[31585]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:45 smithi156 ceph-mon[35720]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:46.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.715+0000 7f5765ffb700 1 -- 172.21.15.186:0/436688633 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f576802d570 con 0x7f57580a1370 2024-01-31T05:19:46.714 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 11 on host 'smithi186' 2024-01-31T05:19:46.716 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.717+0000 7f576f1c2700 1 -- 172.21.15.186:0/436688633 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f57580a1370 msgr2=0x7f57580a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:46.716 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.717+0000 7f576f1c2700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f57580a1370 0x7f57580a3830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5754005fd0 tx=0x7f5754005dc0 comp rx=0 tx=0).stop 2024-01-31T05:19:46.716 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.717+0000 7f576f1c2700 1 -- 172.21.15.186:0/436688633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100710 msgr2=0x7f576806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:46.717 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.717+0000 7f576f1c2700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100710 0x7f576806aa70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f575c000c00 tx=0x7f575c00f720 comp rx=0 tx=0).stop 2024-01-31T05:19:46.717 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.717+0000 7f576f1c2700 1 -- 172.21.15.186:0/436688633 shutdown_connections 2024-01-31T05:19:46.717 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.717+0000 7f576f1c2700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f57580a1370 0x7f57580a3830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:46.717 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.717+0000 7f576f1c2700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57681022b0 0x7f576819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:46.717 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.717+0000 7f576f1c2700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100710 0x7f576806aa70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:46.717 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.717+0000 7f576f1c2700 1 --2- 172.21.15.186:0/436688633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5768101910 0x7f576819c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:46.717 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.717+0000 7f576f1c2700 1 -- 172.21.15.186:0/436688633 >> 172.21.15.186:0/436688633 conn(0x7f57680fbcd0 msgr2=0x7f5768104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:19:46.717 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.717+0000 7f576f1c2700 1 -- 172.21.15.186:0/436688633 shutdown_connections 2024-01-31T05:19:46.717 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:19:46.717+0000 7f576f1c2700 1 -- 172.21.15.186:0/436688633 wait complete. 2024-01-31T05:19:46.889 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:46.889 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:46.889 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:46.889 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:46.889 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:46.889 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:46 smithi186 ceph-mon[35659]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:46.889 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:46.889 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:46.889 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:19:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:46 smithi156 ceph-mon[35720]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:19:47.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:46 smithi045 ceph-mon[31585]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:19:47.347 DEBUG:teuthology.orchestra.run.smithi186:osd.11> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.11.service 2024-01-31T05:19:47.349 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-01-31T05:19:47.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd stat -f json 2024-01-31T05:19:47.604 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:19:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:47 smithi156 ceph-mon[35720]: from='osd.11 [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-31T05:19:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:47 smithi156 ceph-mon[35720]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-31T05:19:48.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:47 smithi045 ceph-mon[31585]: from='osd.11 [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-31T05:19:48.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:47 smithi045 ceph-mon[31585]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-31T05:19:48.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:47 smithi186 ceph-mon[35659]: from='osd.11 [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-31T05:19:48.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:47 smithi186 ceph-mon[35659]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-31T05:19:48.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:48 smithi186 ceph-mon[35659]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-01-31T05:19:48.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:48 smithi186 ceph-mon[35659]: osdmap e79: 12 total, 11 up, 12 in 2024-01-31T05:19:48.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:48 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:48.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:48 smithi186 ceph-mon[35659]: from='osd.11 [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:19:48.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:48 smithi186 ceph-mon[35659]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:19:48.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:48 smithi186 ceph-mon[35659]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:48.933 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:19:48 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:19:48.734+0000 7f1ec8f9a700 -1 osd.11 0 waiting for initial osdmap 2024-01-31T05:19:48.933 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:19:48 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:19:48.750+0000 7f1ec5595700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:19:48.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:48 smithi045 ceph-mon[31585]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-01-31T05:19:48.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:48 smithi045 ceph-mon[31585]: osdmap e79: 12 total, 11 up, 12 in 2024-01-31T05:19:48.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:48 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:48.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:48 smithi045 ceph-mon[31585]: from='osd.11 [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:19:48.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:48 smithi045 ceph-mon[31585]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:19:48.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:48 smithi045 ceph-mon[31585]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:49.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:48 smithi156 ceph-mon[35720]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-01-31T05:19:49.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:48 smithi156 ceph-mon[35720]: osdmap e79: 12 total, 11 up, 12 in 2024-01-31T05:19:49.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:48 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:49.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:48 smithi156 ceph-mon[35720]: from='osd.11 [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:19:49.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:48 smithi156 ceph-mon[35720]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-01-31T05:19:49.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:48 smithi156 ceph-mon[35720]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.337+0000 7f82911b6700 1 -- 172.21.15.45:0/3923178800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c102a10 msgr2=0x7f828c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.337+0000 7f82911b6700 1 --2- 172.21.15.45:0/3923178800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c102a10 0x7f828c102e70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f827c0099c0 tx=0x7f827c009cd0 comp rx=0 tx=0).stop 2024-01-31T05:19:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.337+0000 7f82911b6700 1 -- 172.21.15.45:0/3923178800 shutdown_connections 2024-01-31T05:19:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.337+0000 7f82911b6700 1 --2- 172.21.15.45:0/3923178800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c1033b0 0x7f828c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.337+0000 7f82911b6700 1 --2- 172.21.15.45:0/3923178800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f828c101810 0x7f828c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.337+0000 7f82911b6700 1 --2- 172.21.15.45:0/3923178800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c102a10 0x7f828c102e70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.337+0000 7f82911b6700 1 -- 172.21.15.45:0/3923178800 >> 172.21.15.45:0/3923178800 conn(0x7f828c0fcdf0 msgr2=0x7f828c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.337+0000 7f82911b6700 1 -- 172.21.15.45:0/3923178800 shutdown_connections 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.337+0000 7f82911b6700 1 -- 172.21.15.45:0/3923178800 wait complete. 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f82911b6700 1 Processor -- start 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f82911b6700 1 -- start start 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f82911b6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c101810 0x7f828c075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f82911b6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f828c102a10 0x7f828c073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f82911b6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1033b0 0x7f828c074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f82911b6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c074890 con 0x7f828c1033b0 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f82911b6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c074a00 con 0x7f828c102a10 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f82911b6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c1a1770 con 0x7f828c101810 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f828b59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1033b0 0x7f828c074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f828b59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1033b0 0x7f828c074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40404/0 (socket says 172.21.15.45:40404) 2024-01-31T05:19:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f828b59e700 1 -- 172.21.15.45:0/242748702 learned_addr learned my addr 172.21.15.45:0/242748702 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f828ad9d700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c101810 0x7f828c075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f828a59c700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f828c102a10 0x7f828c073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f828b59e700 1 -- 172.21.15.45:0/242748702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c101810 msgr2=0x7f828c075580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f828b59e700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c101810 0x7f828c075580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f828b59e700 1 -- 172.21.15.45:0/242748702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f828c102a10 msgr2=0x7f828c073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f828b59e700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f828c102a10 0x7f828c073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.338+0000 7f828b59e700 1 -- 172.21.15.45:0/242748702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0095f0 con 0x7f828c1033b0 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.339+0000 7f828b59e700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1033b0 0x7f828c074110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f828400ef30 tx=0x7f828400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.339+0000 7f8273fff700 1 -- 172.21.15.45:0/242748702 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284018070 con 0x7f828c1033b0 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.339+0000 7f82911b6700 1 -- 172.21.15.45:0/242748702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c1a19f0 con 0x7f828c1033b0 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.339+0000 7f8273fff700 1 -- 172.21.15.45:0/242748702 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f828400f040 con 0x7f828c1033b0 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.339+0000 7f8273fff700 1 -- 172.21.15.45:0/242748702 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828400cd00 con 0x7f828c1033b0 2024-01-31T05:19:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.339+0000 7f82911b6700 1 -- 172.21.15.45:0/242748702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c1a1ef0 con 0x7f828c1033b0 2024-01-31T05:19:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.340+0000 7f82911b6700 1 -- 172.21.15.45:0/242748702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828c108ca0 con 0x7f828c1033b0 2024-01-31T05:19:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.341+0000 7f8273fff700 1 -- 172.21.15.45:0/242748702 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f8284003a00 con 0x7f828c1033b0 2024-01-31T05:19:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.342+0000 7f8273fff700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f82740a1370 0x7f82740a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:49.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.342+0000 7f828ad9d700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f82740a1370 0x7f82740a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:49.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.342+0000 7f8273fff700 1 -- 172.21.15.45:0/242748702 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f82840cc130 con 0x7f828c1033b0 2024-01-31T05:19:49.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.342+0000 7f828ad9d700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f82740a1370 0x7f82740a3830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f828c102870 tx=0x7f8280009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:19:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.347+0000 7f8273fff700 1 -- 172.21.15.45:0/242748702 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8284095b60 con 0x7f828c1033b0 2024-01-31T05:19:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.548+0000 7f82911b6700 1 -- 172.21.15.45:0/242748702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f828c0684a0 con 0x7f828c1033b0 2024-01-31T05:19:49.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.549+0000 7f8273fff700 1 -- 172.21.15.45:0/242748702 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f8284099180 con 0x7f828c1033b0 2024-01-31T05:19:49.548 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:19:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.551+0000 7f82911b6700 1 -- 172.21.15.45:0/242748702 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f82740a1370 msgr2=0x7f82740a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.551+0000 7f82911b6700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f82740a1370 0x7f82740a3830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f828c102870 tx=0x7f8280009260 comp rx=0 tx=0).stop 2024-01-31T05:19:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.551+0000 7f82911b6700 1 -- 172.21.15.45:0/242748702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1033b0 msgr2=0x7f828c074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.551+0000 7f82911b6700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1033b0 0x7f828c074110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f828400ef30 tx=0x7f828400c3a0 comp rx=0 tx=0).stop 2024-01-31T05:19:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.551+0000 7f82911b6700 1 -- 172.21.15.45:0/242748702 shutdown_connections 2024-01-31T05:19:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.551+0000 7f82911b6700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f82740a1370 0x7f82740a3830 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.551+0000 7f82911b6700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828c1033b0 0x7f828c074110 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.551+0000 7f82911b6700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c101810 0x7f828c075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.551+0000 7f82911b6700 1 --2- 172.21.15.45:0/242748702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f828c102a10 0x7f828c073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.551+0000 7f82911b6700 1 -- 172.21.15.45:0/242748702 >> 172.21.15.45:0/242748702 conn(0x7f828c0fcdf0 msgr2=0x7f828c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:19:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.551+0000 7f82911b6700 1 -- 172.21.15.45:0/242748702 shutdown_connections 2024-01-31T05:19:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:49.551+0000 7f82911b6700 1 -- 172.21.15.45:0/242748702 wait complete. 2024-01-31T05:19:49.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:49 smithi186 ceph-mon[35659]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-01-31T05:19:49.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:49 smithi186 ceph-mon[35659]: osdmap e80: 12 total, 11 up, 12 in 2024-01-31T05:19:49.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:49 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:49.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:49 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:49.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:49 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/242748702' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-31T05:19:49.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:49 smithi045 ceph-mon[31585]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-01-31T05:19:49.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:49 smithi045 ceph-mon[31585]: osdmap e80: 12 total, 11 up, 12 in 2024-01-31T05:19:49.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:49 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:49.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:49 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:49.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:49 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/242748702' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-31T05:19:50.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:49 smithi156 ceph-mon[35720]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-01-31T05:19:50.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:49 smithi156 ceph-mon[35720]: osdmap e80: 12 total, 11 up, 12 in 2024-01-31T05:19:50.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:49 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:50.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:49 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:50.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:49 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/242748702' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-31T05:19:50.244 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1706678364,"num_in_osds":12,"osd_in_since":1706678369,"num_remapped_pgs":0} 2024-01-31T05:19:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: purged_snaps scrub starts 2024-01-31T05:19:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: purged_snaps scrub ok 2024-01-31T05:19:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: osd.11 [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910] boot 2024-01-31T05:19:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: osdmap e81: 12 total, 12 up, 12 in 2024-01-31T05:19:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: Detected new or changed devices on smithi186 2024-01-31T05:19:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:50.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:50.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:50.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi186 to 4270M 2024-01-31T05:19:50.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:50.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:50.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:50.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:50.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:50 smithi045 ceph-mon[31585]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:51.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: purged_snaps scrub starts 2024-01-31T05:19:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: purged_snaps scrub ok 2024-01-31T05:19:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: osd.11 [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910] boot 2024-01-31T05:19:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: osdmap e81: 12 total, 12 up, 12 in 2024-01-31T05:19:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: Detected new or changed devices on smithi186 2024-01-31T05:19:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi186 to 4270M 2024-01-31T05:19:51.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:51.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:51.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:51.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:51.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:50 smithi186 ceph-mon[35659]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:51.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: purged_snaps scrub starts 2024-01-31T05:19:51.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: purged_snaps scrub ok 2024-01-31T05:19:51.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: osd.11 [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910] boot 2024-01-31T05:19:51.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: osdmap e81: 12 total, 12 up, 12 in 2024-01-31T05:19:51.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:19:51.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: Detected new or changed devices on smithi186 2024-01-31T05:19:51.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:19:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi186 to 4270M 2024-01-31T05:19:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:19:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:19:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:19:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:50 smithi156 ceph-mon[35720]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-01-31T05:19:51.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd stat -f json 2024-01-31T05:19:51.500 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:19:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:51 smithi045 ceph-mon[31585]: osdmap e82: 12 total, 12 up, 12 in 2024-01-31T05:19:52.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:51 smithi186 ceph-mon[35659]: osdmap e82: 12 total, 12 up, 12 in 2024-01-31T05:19:52.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:51 smithi156 ceph-mon[35720]: osdmap e82: 12 total, 12 up, 12 in 2024-01-31T05:19:53.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.088+0000 7f2eb88f9700 1 -- 172.21.15.45:0/1057884772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eb4101910 msgr2=0x7f2eb4101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:53.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.088+0000 7f2eb88f9700 1 --2- 172.21.15.45:0/1057884772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eb4101910 0x7f2eb4101d70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0099c0 tx=0x7f2e9c009cd0 comp rx=0 tx=0).stop 2024-01-31T05:19:53.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.088+0000 7f2eb88f9700 1 -- 172.21.15.45:0/1057884772 shutdown_connections 2024-01-31T05:19:53.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.088+0000 7f2eb88f9700 1 --2- 172.21.15.45:0/1057884772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb41022b0 0x7f2eb4108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:53.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.088+0000 7f2eb88f9700 1 --2- 172.21.15.45:0/1057884772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb4100710 0x7f2eb4100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:53.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.088+0000 7f2eb88f9700 1 --2- 172.21.15.45:0/1057884772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eb4101910 0x7f2eb4101d70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:53.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.088+0000 7f2eb88f9700 1 -- 172.21.15.45:0/1057884772 >> 172.21.15.45:0/1057884772 conn(0x7f2eb40fbcd0 msgr2=0x7f2eb40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:19:53.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.088+0000 7f2eb88f9700 1 -- 172.21.15.45:0/1057884772 shutdown_connections 2024-01-31T05:19:53.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.088+0000 7f2eb88f9700 1 -- 172.21.15.45:0/1057884772 wait complete. 2024-01-31T05:19:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb88f9700 1 Processor -- start 2024-01-31T05:19:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb88f9700 1 -- start start 2024-01-31T05:19:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb88f9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb4100710 0x7f2eb41976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb88f9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb4101910 0x7f2eb4197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb88f9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eb41022b0 0x7f2eb419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb88f9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb419f280 con 0x7f2eb41022b0 2024-01-31T05:19:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb88f9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb419f3c0 con 0x7f2eb4100710 2024-01-31T05:19:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb88f9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb419f550 con 0x7f2eb4101910 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eb41022b0 0x7f2eb419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eb41022b0 0x7f2eb419ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40428/0 (socket says 172.21.15.45:40428) 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb2d9d700 1 -- 172.21.15.45:0/2472119039 learned_addr learned my addr 172.21.15.45:0/2472119039 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb259c700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb4100710 0x7f2eb41976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb1d9b700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb4101910 0x7f2eb4197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb2d9d700 1 -- 172.21.15.45:0/2472119039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb4101910 msgr2=0x7f2eb4197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb2d9d700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb4101910 0x7f2eb4197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb2d9d700 1 -- 172.21.15.45:0/2472119039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb4100710 msgr2=0x7f2eb41976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb2d9d700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb4100710 0x7f2eb41976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb2d9d700 1 -- 172.21.15.45:0/2472119039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0095f0 con 0x7f2eb41022b0 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.089+0000 7f2eb1d9b700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb4101910 0x7f2eb4197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.090+0000 7f2eb2d9d700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eb41022b0 0x7f2eb419ecb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00b580 tx=0x7f2eac00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:19:53.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.090+0000 7f2eab7fe700 1 -- 172.21.15.45:0/2472119039 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac011830 con 0x7f2eb41022b0 2024-01-31T05:19:53.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.090+0000 7f2eab7fe700 1 -- 172.21.15.45:0/2472119039 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eac011e50 con 0x7f2eb41022b0 2024-01-31T05:19:53.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.090+0000 7f2eab7fe700 1 -- 172.21.15.45:0/2472119039 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac010e70 con 0x7f2eb41022b0 2024-01-31T05:19:53.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.090+0000 7f2eb88f9700 1 -- 172.21.15.45:0/2472119039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb419f830 con 0x7f2eb41022b0 2024-01-31T05:19:53.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.090+0000 7f2eb88f9700 1 -- 172.21.15.45:0/2472119039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb419fd30 con 0x7f2eb41022b0 2024-01-31T05:19:53.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.091+0000 7f2eb88f9700 1 -- 172.21.15.45:0/2472119039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb40684a0 con 0x7f2eb41022b0 2024-01-31T05:19:53.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.092+0000 7f2eab7fe700 1 -- 172.21.15.45:0/2472119039 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f2eac010510 con 0x7f2eb41022b0 2024-01-31T05:19:53.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.092+0000 7f2eab7fe700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2ea00a13f0 0x7f2ea00a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:53.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.092+0000 7f2eb259c700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2ea00a13f0 0x7f2ea00a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:53.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.092+0000 7f2eab7fe700 1 -- 172.21.15.45:0/2472119039 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2eac0cbd50 con 0x7f2eb41022b0 2024-01-31T05:19:53.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.093+0000 7f2eb259c700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2ea00a13f0 0x7f2ea00a38b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4007720 tx=0x7f2ea40076b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:19:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.097+0000 7f2eab7fe700 1 -- 172.21.15.45:0/2472119039 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f2eac0cf030 con 0x7f2eb41022b0 2024-01-31T05:19:53.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.299+0000 7f2eb88f9700 1 -- 172.21.15.45:0/2472119039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f2eb410bf90 con 0x7f2eb41022b0 2024-01-31T05:19:53.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.299+0000 7f2eab7fe700 1 -- 172.21.15.45:0/2472119039 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) v1 ==== 74+0+133 (secure 0 0 0) 0x7f2eac095670 con 0x7f2eb41022b0 2024-01-31T05:19:53.299 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:19:53.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.301+0000 7f2eb88f9700 1 -- 172.21.15.45:0/2472119039 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2ea00a13f0 msgr2=0x7f2ea00a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.301+0000 7f2eb88f9700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2ea00a13f0 0x7f2ea00a38b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4007720 tx=0x7f2ea40076b0 comp rx=0 tx=0).stop 2024-01-31T05:19:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.301+0000 7f2eb88f9700 1 -- 172.21.15.45:0/2472119039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eb41022b0 msgr2=0x7f2eb419ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.301+0000 7f2eb88f9700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eb41022b0 0x7f2eb419ecb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00b580 tx=0x7f2eac00b940 comp rx=0 tx=0).stop 2024-01-31T05:19:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.301+0000 7f2eb88f9700 1 -- 172.21.15.45:0/2472119039 shutdown_connections 2024-01-31T05:19:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.301+0000 7f2eb88f9700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2ea00a13f0 0x7f2ea00a38b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.301+0000 7f2eb88f9700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eb41022b0 0x7f2eb419ecb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.301+0000 7f2eb88f9700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb4100710 0x7f2eb41976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.301+0000 7f2eb88f9700 1 --2- 172.21.15.45:0/2472119039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb4101910 0x7f2eb4197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.301+0000 7f2eb88f9700 1 -- 172.21.15.45:0/2472119039 >> 172.21.15.45:0/2472119039 conn(0x7f2eb40fbcd0 msgr2=0x7f2eb40fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:19:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.301+0000 7f2eb88f9700 1 -- 172.21.15.45:0/2472119039 shutdown_connections 2024-01-31T05:19:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:53.301+0000 7f2eb88f9700 1 -- 172.21.15.45:0/2472119039 wait complete. 2024-01-31T05:19:53.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:53 smithi045 ceph-mon[31585]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:19:53.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:53 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2472119039' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-31T05:19:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:53 smithi156 ceph-mon[35720]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:19:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:53 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2472119039' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-31T05:19:53.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:53 smithi186 ceph-mon[35659]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:19:53.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:53 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2472119039' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-31T05:19:54.073 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1706678389,"num_in_osds":12,"osd_in_since":1706678369,"num_remapped_pgs":0} 2024-01-31T05:19:54.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd dump --format=json 2024-01-31T05:19:54.321 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:19:55.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:55 smithi045 ceph-mon[31585]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:19:55.911 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:55 smithi186 ceph-mon[35659]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:19:55.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.969+0000 7f71f8ab7700 1 -- 172.21.15.45:0/1785846661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f4106690 msgr2=0x7f71f4106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:55.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.969+0000 7f71f8ab7700 1 --2- 172.21.15.45:0/1785846661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f4106690 0x7f71f4106a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00b0a0 tx=0x7f71ec00b3b0 comp rx=0 tx=0).stop 2024-01-31T05:19:55.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.969+0000 7f71f8ab7700 1 -- 172.21.15.45:0/1785846661 shutdown_connections 2024-01-31T05:19:55.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.969+0000 7f71f8ab7700 1 --2- 172.21.15.45:0/1785846661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71f4101080 0x7f71f410d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:55.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.969+0000 7f71f8ab7700 1 --2- 172.21.15.45:0/1785846661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f4106690 0x7f71f4106a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:55.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.969+0000 7f71f8ab7700 1 --2- 172.21.15.45:0/1785846661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71f41006e0 0x7f71f4100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:55.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.969+0000 7f71f8ab7700 1 -- 172.21.15.45:0/1785846661 >> 172.21.15.45:0/1785846661 conn(0x7f71f4077610 msgr2=0x7f71f4077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:19:55.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.969+0000 7f71f8ab7700 1 -- 172.21.15.45:0/1785846661 shutdown_connections 2024-01-31T05:19:55.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.969+0000 7f71f8ab7700 1 -- 172.21.15.45:0/1785846661 wait complete. 2024-01-31T05:19:55.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.970+0000 7f71f8ab7700 1 Processor -- start 2024-01-31T05:19:55.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.970+0000 7f71f8ab7700 1 -- start start 2024-01-31T05:19:55.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.970+0000 7f71f8ab7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71f41006e0 0x7f71f4102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:55.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.970+0000 7f71f8ab7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f4101080 0x7f71f41027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:55.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.970+0000 7f71f8ab7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71f4106690 0x7f71f4102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:55.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.970+0000 7f71f8ab7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f41a62f0 con 0x7f71f4101080 2024-01-31T05:19:55.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.970+0000 7f71f8ab7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f41a6460 con 0x7f71f41006e0 2024-01-31T05:19:55.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.970+0000 7f71f8ab7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f41a6620 con 0x7f71f4106690 2024-01-31T05:19:55.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f4101080 0x7f71f41027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f2d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71f4106690 0x7f71f4102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f4101080 0x7f71f41027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40444/0 (socket says 172.21.15.45:40444) 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f2d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71f4106690 0x7f71f4102ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:40772/0 (socket says 172.21.15.45:40772) 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f1d9b700 1 -- 172.21.15.45:0/2881385007 learned_addr learned my addr 172.21.15.45:0/2881385007 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f259c700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71f41006e0 0x7f71f4102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f1d9b700 1 -- 172.21.15.45:0/2881385007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71f4106690 msgr2=0x7f71f4102ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f1d9b700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71f4106690 0x7f71f4102ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f1d9b700 1 -- 172.21.15.45:0/2881385007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71f41006e0 msgr2=0x7f71f4102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f1d9b700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71f41006e0 0x7f71f4102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f1d9b700 1 -- 172.21.15.45:0/2881385007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ec009d00 con 0x7f71f4101080 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f2d9d700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71f4106690 0x7f71f4102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f1d9b700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f4101080 0x7f71f41027a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00b530 tx=0x7f71dc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71eb7fe700 1 -- 172.21.15.45:0/2881385007 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc011830 con 0x7f71f4101080 2024-01-31T05:19:55.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71eb7fe700 1 -- 172.21.15.45:0/2881385007 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71dc011e50 con 0x7f71f4101080 2024-01-31T05:19:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f8ab7700 1 -- 172.21.15.45:0/2881385007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f41a6900 con 0x7f71f4101080 2024-01-31T05:19:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71eb7fe700 1 -- 172.21.15.45:0/2881385007 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc00f2e0 con 0x7f71f4101080 2024-01-31T05:19:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.971+0000 7f71f8ab7700 1 -- 172.21.15.45:0/2881385007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f41a6e00 con 0x7f71f4101080 2024-01-31T05:19:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.972+0000 7f71f8ab7700 1 -- 172.21.15.45:0/2881385007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f4109c50 con 0x7f71f4101080 2024-01-31T05:19:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.974+0000 7f71eb7fe700 1 -- 172.21.15.45:0/2881385007 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f71dc010380 con 0x7f71f4101080 2024-01-31T05:19:55.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.974+0000 7f71eb7fe700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f71e00a1370 0x7f71e00a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:55.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.974+0000 7f71f259c700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f71e00a1370 0x7f71e00a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:55.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.974+0000 7f71eb7fe700 1 -- 172.21.15.45:0/2881385007 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f71dc0cc2b0 con 0x7f71f4101080 2024-01-31T05:19:55.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.975+0000 7f71f259c700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f71e00a1370 0x7f71e00a3830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0062a0 tx=0x7f71ec0061f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:19:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:55 smithi156 ceph-mon[35720]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:19:55.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:55.978+0000 7f71eb7fe700 1 -- 172.21.15.45:0/2881385007 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f71dc095b30 con 0x7f71f4101080 2024-01-31T05:19:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.181+0000 7f71f8ab7700 1 -- 172.21.15.45:0/2881385007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71f4103790 con 0x7f71f4101080 2024-01-31T05:19:56.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.182+0000 7f71eb7fe700 1 -- 172.21.15.45:0/2881385007 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18436 (secure 0 0 0) 0x7f71dc099150 con 0x7f71f4101080 2024-01-31T05:19:56.183 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:19:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:19:50.733440+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:19:56.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.185+0000 7f71f8ab7700 1 -- 172.21.15.45:0/2881385007 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f71e00a1370 msgr2=0x7f71e00a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:56.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.185+0000 7f71f8ab7700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f71e00a1370 0x7f71e00a3830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0062a0 tx=0x7f71ec0061f0 comp rx=0 tx=0).stop 2024-01-31T05:19:56.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.185+0000 7f71f8ab7700 1 -- 172.21.15.45:0/2881385007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f4101080 msgr2=0x7f71f41027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:56.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.185+0000 7f71f8ab7700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f4101080 0x7f71f41027a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00b530 tx=0x7f71dc00b840 comp rx=0 tx=0).stop 2024-01-31T05:19:56.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.185+0000 7f71f8ab7700 1 -- 172.21.15.45:0/2881385007 shutdown_connections 2024-01-31T05:19:56.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.185+0000 7f71f8ab7700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71f4106690 0x7f71f4102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:56.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.185+0000 7f71f8ab7700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71f41006e0 0x7f71f4102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:56.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.185+0000 7f71f8ab7700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f71e00a1370 0x7f71e00a3830 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:56.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.185+0000 7f71f8ab7700 1 --2- 172.21.15.45:0/2881385007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f4101080 0x7f71f41027a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:56.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.185+0000 7f71f8ab7700 1 -- 172.21.15.45:0/2881385007 >> 172.21.15.45:0/2881385007 conn(0x7f71f4077610 msgr2=0x7f71f40698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:19:56.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.185+0000 7f71f8ab7700 1 -- 172.21.15.45:0/2881385007 shutdown_connections 2024-01-31T05:19:56.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:56.185+0000 7f71f8ab7700 1 -- 172.21.15.45:0/2881385007 wait complete. 2024-01-31T05:19:56.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:56 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2881385007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:19:56.807 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:16:10.856581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:19:56.808 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-01-31T05:19:56.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:56 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2881385007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:19:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:19:57.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:56 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2881385007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:19:57.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:57 smithi045 ceph-mon[31585]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:19:57.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:57 smithi156 ceph-mon[35720]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:19:58.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:57 smithi186 ceph-mon[35659]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:19:58.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.656+0000 7f0552bb9700 1 -- 172.21.15.45:0/1121674148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c101290 msgr2=0x7f054c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:58.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.656+0000 7f0552bb9700 1 --2- 172.21.15.45:0/1121674148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c101290 0x7f054c101710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f05380099c0 tx=0x7f0538009cd0 comp rx=0 tx=0).stop 2024-01-31T05:19:58.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.656+0000 7f0552bb9700 1 -- 172.21.15.45:0/1121674148 shutdown_connections 2024-01-31T05:19:58.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.656+0000 7f0552bb9700 1 --2- 172.21.15.45:0/1121674148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f054c101c50 0x7f054c10d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.656+0000 7f0552bb9700 1 --2- 172.21.15.45:0/1121674148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f054c105710 0x7f054c105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.656+0000 7f0552bb9700 1 --2- 172.21.15.45:0/1121674148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c101290 0x7f054c101710 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.656+0000 7f0552bb9700 1 -- 172.21.15.45:0/1121674148 >> 172.21.15.45:0/1121674148 conn(0x7f054c075a60 msgr2=0x7f054c075e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.656+0000 7f0552bb9700 1 -- 172.21.15.45:0/1121674148 shutdown_connections 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.656+0000 7f0552bb9700 1 -- 172.21.15.45:0/1121674148 wait complete. 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.657+0000 7f0552bb9700 1 Processor -- start 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.657+0000 7f0552bb9700 1 -- start start 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.657+0000 7f0552bb9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f054c101290 0x7f054c1969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.657+0000 7f0552bb9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f054c101c50 0x7f054c196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.657+0000 7f0552bb9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c105710 0x7f054c197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.657+0000 7f0552bb9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c1a60e0 con 0x7f054c105710 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.657+0000 7f0552bb9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c1a6250 con 0x7f054c101c50 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.657+0000 7f0552bb9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c1a6410 con 0x7f054c101290 2024-01-31T05:19:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.657+0000 7f0551156700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c105710 0x7f054c197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.657+0000 7f0551156700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c105710 0x7f054c197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36490/0 (socket says 172.21.15.45:36490) 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.657+0000 7f0551156700 1 -- 172.21.15.45:0/2235689746 learned_addr learned my addr 172.21.15.45:0/2235689746 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.657+0000 7f054bfff700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f054c101c50 0x7f054c196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.658+0000 7f0551156700 1 -- 172.21.15.45:0/2235689746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f054c101290 msgr2=0x7f054c1969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.658+0000 7f0550955700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f054c101290 0x7f054c1969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.658+0000 7f0551156700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f054c101290 0x7f054c1969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.658+0000 7f0551156700 1 -- 172.21.15.45:0/2235689746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f054c101c50 msgr2=0x7f054c196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.658+0000 7f0551156700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f054c101c50 0x7f054c196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.658+0000 7f0551156700 1 -- 172.21.15.45:0/2235689746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05380095f0 con 0x7f054c105710 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.658+0000 7f0551156700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c105710 0x7f054c197420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f054400ada0 tx=0x7f054400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.658+0000 7f0549ffb700 1 -- 172.21.15.45:0/2235689746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054400f070 con 0x7f054c105710 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.658+0000 7f0552bb9700 1 -- 172.21.15.45:0/2235689746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c1a66f0 con 0x7f054c105710 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.658+0000 7f0549ffb700 1 -- 172.21.15.45:0/2235689746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0544007b30 con 0x7f054c105710 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.658+0000 7f0549ffb700 1 -- 172.21.15.45:0/2235689746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544005440 con 0x7f054c105710 2024-01-31T05:19:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.658+0000 7f0552bb9700 1 -- 172.21.15.45:0/2235689746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c1a6bf0 con 0x7f054c105710 2024-01-31T05:19:58.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.659+0000 7f0552bb9700 1 -- 172.21.15.45:0/2235689746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c108ce0 con 0x7f054c105710 2024-01-31T05:19:58.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.660+0000 7f0549ffb700 1 -- 172.21.15.45:0/2235689746 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f0544004ad0 con 0x7f054c105710 2024-01-31T05:19:58.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.661+0000 7f0549ffb700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f053c0a1370 0x7f053c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:19:58.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.661+0000 7f0550955700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f053c0a1370 0x7f053c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:19:58.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.661+0000 7f0549ffb700 1 -- 172.21.15.45:0/2235689746 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f05440cbd20 con 0x7f054c105710 2024-01-31T05:19:58.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.661+0000 7f0550955700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f053c0a1370 0x7f053c0a3830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0540006fd0 tx=0x7f0540008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:19:58.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.665+0000 7f0549ffb700 1 -- 172.21.15.45:0/2235689746 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f05440956d0 con 0x7f054c105710 2024-01-31T05:19:58.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.864+0000 7f0552bb9700 1 -- 172.21.15.45:0/2235689746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f054c100770 con 0x7f054c105710 2024-01-31T05:19:58.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.864+0000 7f0549ffb700 1 -- 172.21.15.45:0/2235689746 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0544098cf0 con 0x7f054c105710 2024-01-31T05:19:58.863 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-01-31T05:19:58.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.866+0000 7f0552bb9700 1 -- 172.21.15.45:0/2235689746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f053c0a1370 msgr2=0x7f053c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:58.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.866+0000 7f0552bb9700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f053c0a1370 0x7f053c0a3830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0540006fd0 tx=0x7f0540008040 comp rx=0 tx=0).stop 2024-01-31T05:19:58.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.866+0000 7f0552bb9700 1 -- 172.21.15.45:0/2235689746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c105710 msgr2=0x7f054c197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:19:58.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.866+0000 7f0552bb9700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c105710 0x7f054c197420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f054400ada0 tx=0x7f054400c330 comp rx=0 tx=0).stop 2024-01-31T05:19:58.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.866+0000 7f0552bb9700 1 -- 172.21.15.45:0/2235689746 shutdown_connections 2024-01-31T05:19:58.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.866+0000 7f0552bb9700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f053c0a1370 0x7f053c0a3830 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:58.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.866+0000 7f0552bb9700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c105710 0x7f054c197420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:58.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.866+0000 7f0552bb9700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f054c101290 0x7f054c1969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:58.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.866+0000 7f0552bb9700 1 --2- 172.21.15.45:0/2235689746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f054c101c50 0x7f054c196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:19:58.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.866+0000 7f0552bb9700 1 -- 172.21.15.45:0/2235689746 >> 172.21.15.45:0/2235689746 conn(0x7f054c075a60 msgr2=0x7f054c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:19:58.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.867+0000 7f0552bb9700 1 -- 172.21.15.45:0/2235689746 shutdown_connections 2024-01-31T05:19:58.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:19:58.867+0000 7f0552bb9700 1 -- 172.21.15.45:0/2235689746 wait complete. 2024-01-31T05:19:59.481 INFO:tasks.cephadm:Adding mds.a on smithi045 2024-01-31T05:19:59.481 INFO:tasks.cephadm:Adding mds.d on smithi045 2024-01-31T05:19:59.481 INFO:tasks.cephadm:Adding mds.g on smithi045 2024-01-31T05:19:59.482 INFO:tasks.cephadm:Adding mds.j on smithi045 2024-01-31T05:19:59.482 INFO:tasks.cephadm:Adding mds.b on smithi156 2024-01-31T05:19:59.482 INFO:tasks.cephadm:Adding mds.e on smithi156 2024-01-31T05:19:59.482 INFO:tasks.cephadm:Adding mds.h on smithi156 2024-01-31T05:19:59.482 INFO:tasks.cephadm:Adding mds.k on smithi156 2024-01-31T05:19:59.482 INFO:tasks.cephadm:Adding mds.c on smithi186 2024-01-31T05:19:59.482 INFO:tasks.cephadm:Adding mds.f on smithi186 2024-01-31T05:19:59.482 INFO:tasks.cephadm:Adding mds.i on smithi186 2024-01-31T05:19:59.482 INFO:tasks.cephadm:Adding mds.l on smithi186 2024-01-31T05:19:59.482 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph orch apply mds all '12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l' 2024-01-31T05:19:59.737 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.c/config 2024-01-31T05:19:59.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:59 smithi186 ceph-mon[35659]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:19:59.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:19:59 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2235689746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:19:59.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:59 smithi045 ceph-mon[31585]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:19:59.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:19:59 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2235689746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:19:59.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:59 smithi156 ceph-mon[35720]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:19:59.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:19:59 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2235689746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:20:00.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:00 smithi045 ceph-mon[31585]: overall HEALTH_OK 2024-01-31T05:20:00.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:00 smithi156 ceph-mon[35720]: overall HEALTH_OK 2024-01-31T05:20:01.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:00 smithi186 ceph-mon[35659]: overall HEALTH_OK 2024-01-31T05:20:01.570 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.569+0000 7f183bb50700 1 -- 172.21.15.186:0/3176918724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1834102340 msgr2=0x7f1834104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.569+0000 7f183bb50700 1 --2- 172.21.15.186:0/3176918724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1834102340 0x7f1834104730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1824009990 tx=0x7f1824009ca0 comp rx=0 tx=0).stop 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.569+0000 7f183bb50700 1 -- 172.21.15.186:0/3176918724 shutdown_connections 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.569+0000 7f183bb50700 1 --2- 172.21.15.186:0/3176918724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18341075a0 0x7f1834109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.569+0000 7f183bb50700 1 --2- 172.21.15.186:0/3176918724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1834102340 0x7f1834104730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.569+0000 7f183bb50700 1 --2- 172.21.15.186:0/3176918724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1834104c70 0x7f1834107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.569+0000 7f183bb50700 1 -- 172.21.15.186:0/3176918724 >> 172.21.15.186:0/3176918724 conn(0x7f18340fbcb0 msgr2=0x7f18340fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.569+0000 7f183bb50700 1 -- 172.21.15.186:0/3176918724 shutdown_connections 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.569+0000 7f183bb50700 1 -- 172.21.15.186:0/3176918724 wait complete. 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f183bb50700 1 Processor -- start 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f183bb50700 1 -- start start 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f183bb50700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1834102340 0x7f1834101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f183bb50700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1834104c70 0x7f18341002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f183bb50700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18341075a0 0x7f1834100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:01.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f183bb50700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834100fb0 con 0x7f1834104c70 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f183bb50700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834101120 con 0x7f18341075a0 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f183bb50700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18341a1770 con 0x7f1834102340 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f18398ec700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1834102340 0x7f1834101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f18398ec700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1834102340 0x7f1834101ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:38796/0 (socket says 172.21.15.186:38796) 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f18398ec700 1 -- 172.21.15.186:0/1125803698 learned_addr learned my addr 172.21.15.186:0/1125803698 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f18390eb700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1834104c70 0x7f18341002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.570+0000 7f183a0ed700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18341075a0 0x7f1834100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f18398ec700 1 -- 172.21.15.186:0/1125803698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18341075a0 msgr2=0x7f1834100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f18398ec700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18341075a0 0x7f1834100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f18398ec700 1 -- 172.21.15.186:0/1125803698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1834104c70 msgr2=0x7f18341002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f18398ec700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1834104c70 0x7f18341002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f18398ec700 1 -- 172.21.15.186:0/1125803698 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18240095f0 con 0x7f1834102340 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f183a0ed700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18341075a0 0x7f1834100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f18390eb700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1834104c70 0x7f18341002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:01.572 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f18398ec700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1834102340 0x7f1834101ca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f18240098e0 tx=0x7f182400f7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:01.573 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f182affd700 1 -- 172.21.15.186:0/1125803698 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182401d070 con 0x7f1834102340 2024-01-31T05:20:01.573 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f183bb50700 1 -- 172.21.15.186:0/1125803698 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18341a19f0 con 0x7f1834102340 2024-01-31T05:20:01.573 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f182affd700 1 -- 172.21.15.186:0/1125803698 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f182400fca0 con 0x7f1834102340 2024-01-31T05:20:01.573 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f182affd700 1 -- 172.21.15.186:0/1125803698 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824017810 con 0x7f1834102340 2024-01-31T05:20:01.573 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.571+0000 7f183bb50700 1 -- 172.21.15.186:0/1125803698 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18341a1e00 con 0x7f1834102340 2024-01-31T05:20:01.574 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.572+0000 7f183bb50700 1 -- 172.21.15.186:0/1125803698 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18340fd830 con 0x7f1834102340 2024-01-31T05:20:01.574 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.573+0000 7f182affd700 1 -- 172.21.15.186:0/1125803698 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f1824022ac0 con 0x7f1834102340 2024-01-31T05:20:01.574 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.573+0000 7f182affd700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f18200a1370 0x7f18200a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:01.574 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.574+0000 7f182affd700 1 -- 172.21.15.186:0/1125803698 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f18240ce0f0 con 0x7f1834102340 2024-01-31T05:20:01.574 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.574+0000 7f18390eb700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f18200a1370 0x7f18200a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:01.574 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.574+0000 7f18390eb700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f18200a1370 0x7f18200a3830 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1830005950 tx=0x7f1830005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:01.576 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.577+0000 7f182affd700 1 -- 172.21.15.186:0/1125803698 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1824097aa0 con 0x7f1834102340 2024-01-31T05:20:01.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:01 smithi156 ceph-mon[35720]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:01.781 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.782+0000 7f183bb50700 1 -- 172.21.15.186:0/1125803698 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f1834061d20 con 0x7f18200a1370 2024-01-31T05:20:01.789 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.790+0000 7f182affd700 1 -- 172.21.15.186:0/1125803698 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f1834061d20 con 0x7f18200a1370 2024-01-31T05:20:01.789 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mds.all update... 2024-01-31T05:20:01.791 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.792+0000 7f183bb50700 1 -- 172.21.15.186:0/1125803698 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f18200a1370 msgr2=0x7f18200a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:01.792 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.792+0000 7f183bb50700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f18200a1370 0x7f18200a3830 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1830005950 tx=0x7f1830005f50 comp rx=0 tx=0).stop 2024-01-31T05:20:01.792 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.792+0000 7f183bb50700 1 -- 172.21.15.186:0/1125803698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1834102340 msgr2=0x7f1834101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:01.792 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.792+0000 7f183bb50700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1834102340 0x7f1834101ca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f18240098e0 tx=0x7f182400f7b0 comp rx=0 tx=0).stop 2024-01-31T05:20:01.792 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.792+0000 7f183bb50700 1 -- 172.21.15.186:0/1125803698 shutdown_connections 2024-01-31T05:20:01.792 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.792+0000 7f183bb50700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18341075a0 0x7f1834100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:01.792 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.792+0000 7f183bb50700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1834102340 0x7f1834101ca0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:01.792 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.792+0000 7f183bb50700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f18200a1370 0x7f18200a3830 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:01.792 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.792+0000 7f183bb50700 1 --2- 172.21.15.186:0/1125803698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1834104c70 0x7f18341002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:01.792 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.792+0000 7f183bb50700 1 -- 172.21.15.186:0/1125803698 >> 172.21.15.186:0/1125803698 conn(0x7f18340fbcb0 msgr2=0x7f1834102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:01.792 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.792+0000 7f183bb50700 1 -- 172.21.15.186:0/1125803698 shutdown_connections 2024-01-31T05:20:01.792 INFO:teuthology.orchestra.run.smithi186.stderr:2024-01-31T05:20:01.792+0000 7f183bb50700 1 -- 172.21.15.186:0/1125803698 wait complete. 2024-01-31T05:20:02.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:01 smithi186 ceph-mon[35659]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:02.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:01 smithi045 ceph-mon[31585]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:02.400 DEBUG:teuthology.orchestra.run.smithi045:mds.a> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.a.service 2024-01-31T05:20:02.402 DEBUG:teuthology.orchestra.run.smithi045:mds.d> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.d.service 2024-01-31T05:20:02.445 DEBUG:teuthology.orchestra.run.smithi045:mds.g> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.g.service 2024-01-31T05:20:02.487 DEBUG:teuthology.orchestra.run.smithi045:mds.j> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.j.service 2024-01-31T05:20:02.530 DEBUG:teuthology.orchestra.run.smithi156:mds.b> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.b.service 2024-01-31T05:20:02.533 DEBUG:teuthology.orchestra.run.smithi156:mds.e> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.e.service 2024-01-31T05:20:02.575 DEBUG:teuthology.orchestra.run.smithi156:mds.h> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service 2024-01-31T05:20:02.617 DEBUG:teuthology.orchestra.run.smithi156:mds.k> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.k.service 2024-01-31T05:20:02.661 DEBUG:teuthology.orchestra.run.smithi186:mds.c> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.c.service 2024-01-31T05:20:02.663 DEBUG:teuthology.orchestra.run.smithi186:mds.f> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.f.service 2024-01-31T05:20:02.706 DEBUG:teuthology.orchestra.run.smithi186:mds.i> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.i.service 2024-01-31T05:20:02.749 DEBUG:teuthology.orchestra.run.smithi186:mds.l> sudo journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.l.service 2024-01-31T05:20:02.791 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-01-31T05:20:02.792 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:20:02.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.903+0000 7f2011e78640 1 -- 172.21.15.45:0/3862476247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f200c0fedb0 msgr2=0x7f200c0ff230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:02.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.903+0000 7f2011e78640 1 --2- 172.21.15.45:0/3862476247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f200c0fedb0 0x7f200c0ff230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2000009900 tx=0x7f200002bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.903+0000 7f2011e78640 1 -- 172.21.15.45:0/3862476247 shutdown_connections 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.903+0000 7f2011e78640 1 --2- 172.21.15.45:0/3862476247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f200c0fedb0 0x7f200c0ff230 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.903+0000 7f2011e78640 1 --2- 172.21.15.45:0/3862476247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f200c0ff770 0x7f200c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.903+0000 7f2011e78640 1 --2- 172.21.15.45:0/3862476247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f200c102340 0x7f200c102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.903+0000 7f2011e78640 1 -- 172.21.15.45:0/3862476247 >> 172.21.15.45:0/3862476247 conn(0x7f200c0fa930 msgr2=0x7f200c0fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.903+0000 7f2011e78640 1 -- 172.21.15.45:0/3862476247 shutdown_connections 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.903+0000 7f2011e78640 1 -- 172.21.15.45:0/3862476247 wait complete. 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f2011e78640 1 Processor -- start 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f2011e78640 1 -- start start 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f2011e78640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f200c0fedb0 0x7f200c101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f2011e78640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f200c0ff770 0x7f200c100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f2011e78640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f200c102340 0x7f200c1005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f2011e78640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c100d20 con 0x7f200c102340 2024-01-31T05:20:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f2011e78640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c100e90 con 0x7f200c0ff770 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f2011e78640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c1a0d10 con 0x7f200c0fedb0 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f200bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f200c102340 0x7f200c1005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f200bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f200c102340 0x7f200c1005c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36518/0 (socket says 172.21.15.45:36518) 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f200bfff640 1 -- 172.21.15.45:0/1757246736 learned_addr learned my addr 172.21.15.45:0/1757246736 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f200affd640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f200c0ff770 0x7f200c100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.904+0000 7f200b7fe640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f200c0fedb0 0x7f200c101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.905+0000 7f200bfff640 1 -- 172.21.15.45:0/1757246736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f200c0fedb0 msgr2=0x7f200c101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.905+0000 7f200bfff640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f200c0fedb0 0x7f200c101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.905+0000 7f200bfff640 1 -- 172.21.15.45:0/1757246736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f200c0ff770 msgr2=0x7f200c100080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.905+0000 7f200bfff640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f200c0ff770 0x7f200c100080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.905+0000 7f200bfff640 1 -- 172.21.15.45:0/1757246736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2000009560 con 0x7f200c102340 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.905+0000 7f200bfff640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f200c102340 0x7f200c1005c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f200400db20 tx=0x7f200400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.905+0000 7f2008ff9640 1 -- 172.21.15.45:0/1757246736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200400bea0 con 0x7f200c102340 2024-01-31T05:20:02.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.905+0000 7f2008ff9640 1 -- 172.21.15.45:0/1757246736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2004009940 con 0x7f200c102340 2024-01-31T05:20:02.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.905+0000 7f2011e78640 1 -- 172.21.15.45:0/1757246736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200c1a0fc0 con 0x7f200c102340 2024-01-31T05:20:02.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.905+0000 7f2008ff9640 1 -- 172.21.15.45:0/1757246736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20040106c0 con 0x7f200c102340 2024-01-31T05:20:02.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.905+0000 7f2011e78640 1 -- 172.21.15.45:0/1757246736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200c1a13f0 con 0x7f200c102340 2024-01-31T05:20:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.906+0000 7f2011e78640 1 -- 172.21.15.45:0/1757246736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200c108200 con 0x7f200c102340 2024-01-31T05:20:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.907+0000 7f2008ff9640 1 -- 172.21.15.45:0/1757246736 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f20040041b0 con 0x7f200c102340 2024-01-31T05:20:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.907+0000 7f2008ff9640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ff40a12e0 0x7f1ff40a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.908+0000 7f2008ff9640 1 -- 172.21.15.45:0/1757246736 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f20040cc340 con 0x7f200c102340 2024-01-31T05:20:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.908+0000 7f200b7fe640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ff40a12e0 0x7f1ff40a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.908+0000 7f200b7fe640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ff40a12e0 0x7f1ff40a37a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc005fd0 tx=0x7f1ffc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:02.912+0000 7f2008ff9640 1 -- 172.21.15.45:0/1757246736 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f2004095d10 con 0x7f200c102340 2024-01-31T05:20:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:02 smithi156 ceph-mon[35720]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:02 smithi156 ceph-mon[35720]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-01-31T05:20:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:20:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:02 smithi156 ceph-mon[35720]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:02.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:02.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:20:02.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:02.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:02.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:02.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:02 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:02.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:02 smithi156 ceph-mon[35720]: Deploying daemon mds.l on smithi186 2024-01-31T05:20:03.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:02 smithi186 ceph-mon[35659]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:03.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:02 smithi186 ceph-mon[35659]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-01-31T05:20:03.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:20:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:02 smithi186 ceph-mon[35659]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:20:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:02 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:02 smithi186 ceph-mon[35659]: Deploying daemon mds.l on smithi186 2024-01-31T05:20:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.097+0000 7f2011e78640 1 -- 172.21.15.45:0/1757246736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f200c02cad0 con 0x7f200c102340 2024-01-31T05:20:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.098+0000 7f2008ff9640 1 -- 172.21.15.45:0/1757246736 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18436 (secure 0 0 0) 0x7f2004099330 con 0x7f200c102340 2024-01-31T05:20:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:03.098 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:19:50.733440+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:20:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.100+0000 7f2011e78640 1 -- 172.21.15.45:0/1757246736 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ff40a12e0 msgr2=0x7f1ff40a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.100+0000 7f2011e78640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ff40a12e0 0x7f1ff40a37a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc005fd0 tx=0x7f1ffc00b410 comp rx=0 tx=0).stop 2024-01-31T05:20:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.100+0000 7f2011e78640 1 -- 172.21.15.45:0/1757246736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f200c102340 msgr2=0x7f200c1005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.100+0000 7f2011e78640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f200c102340 0x7f200c1005c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f200400db20 tx=0x7f200400b260 comp rx=0 tx=0).stop 2024-01-31T05:20:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.100+0000 7f2011e78640 1 -- 172.21.15.45:0/1757246736 shutdown_connections 2024-01-31T05:20:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.100+0000 7f2011e78640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ff40a12e0 0x7f1ff40a37a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.100+0000 7f2011e78640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f200c102340 0x7f200c1005c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.100+0000 7f2011e78640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f200c0ff770 0x7f200c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.100+0000 7f2011e78640 1 --2- 172.21.15.45:0/1757246736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f200c0fedb0 0x7f200c101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.100+0000 7f2011e78640 1 -- 172.21.15.45:0/1757246736 >> 172.21.15.45:0/1757246736 conn(0x7f200c0fa930 msgr2=0x7f200c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.101+0000 7f2011e78640 1 -- 172.21.15.45:0/1757246736 shutdown_connections 2024-01-31T05:20:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.101+0000 7f2011e78640 1 -- 172.21.15.45:0/1757246736 wait complete. 2024-01-31T05:20:03.110 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:02 smithi045 ceph-mon[31585]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:03.110 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:02 smithi045 ceph-mon[31585]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-01-31T05:20:03.110 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:03.110 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:20:03.110 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:02 smithi045 ceph-mon[31585]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:03.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:03.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:20:03.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:03.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:03.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:03.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:02 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:03.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:02 smithi045 ceph-mon[31585]: Deploying daemon mds.l on smithi186 2024-01-31T05:20:03.112 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:16:10.856581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:20:03.113 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:20:03.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.219+0000 7fe684406640 1 -- 172.21.15.45:0/3012154116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe67c0ffc10 msgr2=0x7fe67c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:03.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.219+0000 7fe684406640 1 --2- 172.21.15.45:0/3012154116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe67c0ffc10 0x7fe67c100030 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe678009900 tx=0x7fe67802bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:03.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.219+0000 7fe684406640 1 -- 172.21.15.45:0/3012154116 shutdown_connections 2024-01-31T05:20:03.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.219+0000 7fe684406640 1 --2- 172.21.15.45:0/3012154116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1017c0 0x7fe67c108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.219+0000 7fe684406640 1 --2- 172.21.15.45:0/3012154116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c100e00 0x7fe67c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.219+0000 7fe684406640 1 --2- 172.21.15.45:0/3012154116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe67c0ffc10 0x7fe67c100030 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.219+0000 7fe684406640 1 -- 172.21.15.45:0/3012154116 >> 172.21.15.45:0/3012154116 conn(0x7fe67c0fb1a0 msgr2=0x7fe67c0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:03.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.219+0000 7fe684406640 1 -- 172.21.15.45:0/3012154116 shutdown_connections 2024-01-31T05:20:03.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.219+0000 7fe684406640 1 -- 172.21.15.45:0/3012154116 wait complete. 2024-01-31T05:20:03.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe684406640 1 Processor -- start 2024-01-31T05:20:03.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe684406640 1 -- start start 2024-01-31T05:20:03.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe684406640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe67c0ffc10 0x7fe67c196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:03.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe684406640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c100e00 0x7fe67c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe684406640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1017c0 0x7fe67c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe684406640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c19e7d0 con 0x7fe67c0ffc10 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe684406640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c19e940 con 0x7fe67c100e00 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe684406640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c19eb00 con 0x7fe67c1017c0 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6821a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe67c0ffc10 0x7fe67c196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6821a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe67c0ffc10 0x7fe67c196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36530/0 (socket says 172.21.15.45:36530) 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6821a2640 1 -- 172.21.15.45:0/2168135884 learned_addr learned my addr 172.21.15.45:0/2168135884 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6829a3640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1017c0 0x7fe67c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6819a1640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c100e00 0x7fe67c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6821a2640 1 -- 172.21.15.45:0/2168135884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1017c0 msgr2=0x7fe67c19e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6821a2640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1017c0 0x7fe67c19e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6821a2640 1 -- 172.21.15.45:0/2168135884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c100e00 msgr2=0x7fe67c197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6821a2640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c100e00 0x7fe67c197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6821a2640 1 -- 172.21.15.45:0/2168135884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe678009560 con 0x7fe67c0ffc10 2024-01-31T05:20:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6829a3640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1017c0 0x7fe67c19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6821a2640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe67c0ffc10 0x7fe67c196cf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe678009230 tx=0x7fe678004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.220+0000 7fe6737fe640 1 -- 172.21.15.45:0/2168135884 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6780043d0 con 0x7fe67c0ffc10 2024-01-31T05:20:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.221+0000 7fe6737fe640 1 -- 172.21.15.45:0/2168135884 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6780054c0 con 0x7fe67c0ffc10 2024-01-31T05:20:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.221+0000 7fe684406640 1 -- 172.21.15.45:0/2168135884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c19ed80 con 0x7fe67c0ffc10 2024-01-31T05:20:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.221+0000 7fe6737fe640 1 -- 172.21.15.45:0/2168135884 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678003e60 con 0x7fe67c0ffc10 2024-01-31T05:20:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.221+0000 7fe684406640 1 -- 172.21.15.45:0/2168135884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c19f1d0 con 0x7fe67c0ffc10 2024-01-31T05:20:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.221+0000 7fe684406640 1 -- 172.21.15.45:0/2168135884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c107000 con 0x7fe67c0ffc10 2024-01-31T05:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.223+0000 7fe6737fe640 1 -- 172.21.15.45:0/2168135884 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe6780035f0 con 0x7fe67c0ffc10 2024-01-31T05:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.224+0000 7fe6737fe640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6680a12e0 0x7fe6680a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.224+0000 7fe6819a1640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6680a12e0 0x7fe6680a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.224+0000 7fe6737fe640 1 -- 172.21.15.45:0/2168135884 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe6780ef570 con 0x7fe67c0ffc10 2024-01-31T05:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.224+0000 7fe6819a1640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6680a12e0 0x7fe6680a37a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe66c005d00 tx=0x7fe66c005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.227+0000 7fe6737fe640 1 -- 172.21.15.45:0/2168135884 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe6780b8e00 con 0x7fe67c0ffc10 2024-01-31T05:20:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.405+0000 7fe684406640 1 -- 172.21.15.45:0/2168135884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe67c10c600 con 0x7fe67c0ffc10 2024-01-31T05:20:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.405+0000 7fe6737fe640 1 -- 172.21.15.45:0/2168135884 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6780bad30 con 0x7fe67c0ffc10 2024-01-31T05:20:03.405 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-01-31T05:20:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.407+0000 7fe684406640 1 -- 172.21.15.45:0/2168135884 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6680a12e0 msgr2=0x7fe6680a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.407+0000 7fe684406640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6680a12e0 0x7fe6680a37a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe66c005d00 tx=0x7fe66c005c50 comp rx=0 tx=0).stop 2024-01-31T05:20:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.407+0000 7fe684406640 1 -- 172.21.15.45:0/2168135884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe67c0ffc10 msgr2=0x7fe67c196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.407+0000 7fe684406640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe67c0ffc10 0x7fe67c196cf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe678009230 tx=0x7fe678004f40 comp rx=0 tx=0).stop 2024-01-31T05:20:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.407+0000 7fe684406640 1 -- 172.21.15.45:0/2168135884 shutdown_connections 2024-01-31T05:20:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.407+0000 7fe684406640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6680a12e0 0x7fe6680a37a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.407+0000 7fe684406640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1017c0 0x7fe67c19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.407+0000 7fe684406640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c100e00 0x7fe67c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.407+0000 7fe684406640 1 --2- 172.21.15.45:0/2168135884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe67c0ffc10 0x7fe67c196cf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.407+0000 7fe684406640 1 -- 172.21.15.45:0/2168135884 >> 172.21.15.45:0/2168135884 conn(0x7fe67c0fb1a0 msgr2=0x7fe67c1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.407+0000 7fe684406640 1 -- 172.21.15.45:0/2168135884 shutdown_connections 2024-01-31T05:20:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.407+0000 7fe684406640 1 -- 172.21.15.45:0/2168135884 wait complete. 2024-01-31T05:20:03.417 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:03.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.520+0000 7fe4b5619640 1 -- 172.21.15.45:0/4275129999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b0100d10 msgr2=0x7fe4b0101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:03.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.520+0000 7fe4b5619640 1 --2- 172.21.15.45:0/4275129999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b0100d10 0x7fe4b0101130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009900 tx=0x7fe4a002bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:03.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.520+0000 7fe4b5619640 1 -- 172.21.15.45:0/4275129999 shutdown_connections 2024-01-31T05:20:03.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.520+0000 7fe4b5619640 1 --2- 172.21.15.45:0/4275129999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4b01028c0 0x7fe4b0109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.520+0000 7fe4b5619640 1 --2- 172.21.15.45:0/4275129999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4b0101f00 0x7fe4b0102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.520+0000 7fe4b5619640 1 --2- 172.21.15.45:0/4275129999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b0100d10 0x7fe4b0101130 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.520+0000 7fe4b5619640 1 -- 172.21.15.45:0/4275129999 >> 172.21.15.45:0/4275129999 conn(0x7fe4b00fc2a0 msgr2=0x7fe4b00fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.520+0000 7fe4b5619640 1 -- 172.21.15.45:0/4275129999 shutdown_connections 2024-01-31T05:20:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.520+0000 7fe4b5619640 1 -- 172.21.15.45:0/4275129999 wait complete. 2024-01-31T05:20:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.520+0000 7fe4b5619640 1 Processor -- start 2024-01-31T05:20:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4b5619640 1 -- start start 2024-01-31T05:20:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4b5619640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4b0100d10 0x7fe4b0196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4b5619640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4b0101f00 0x7fe4b0197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4b5619640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b01028c0 0x7fe4b019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4b5619640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b019e7e0 con 0x7fe4b01028c0 2024-01-31T05:20:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4b5619640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b019e950 con 0x7fe4b0101f00 2024-01-31T05:20:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4b5619640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b019eb10 con 0x7fe4b0100d10 2024-01-31T05:20:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4af7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b01028c0 0x7fe4b019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4af7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b01028c0 0x7fe4b019e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36540/0 (socket says 172.21.15.45:36540) 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4af7fe640 1 -- 172.21.15.45:0/2117348551 learned_addr learned my addr 172.21.15.45:0/2117348551 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4aeffd640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4b0100d10 0x7fe4b0196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4ae7fc640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4b0101f00 0x7fe4b0197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4af7fe640 1 -- 172.21.15.45:0/2117348551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4b0100d10 msgr2=0x7fe4b0196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4af7fe640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4b0100d10 0x7fe4b0196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4af7fe640 1 -- 172.21.15.45:0/2117348551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4b0101f00 msgr2=0x7fe4b0197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4af7fe640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4b0101f00 0x7fe4b0197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4af7fe640 1 -- 172.21.15.45:0/2117348551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a0009560 con 0x7fe4b01028c0 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.521+0000 7fe4af7fe640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b01028c0 0x7fe4b019e270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800eaa0 tx=0x7fe4a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.522+0000 7fe497fff640 1 -- 172.21.15.45:0/2117348551 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a800cdb0 con 0x7fe4b01028c0 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.522+0000 7fe4b5619640 1 -- 172.21.15.45:0/2117348551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b019edf0 con 0x7fe4b01028c0 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.522+0000 7fe497fff640 1 -- 172.21.15.45:0/2117348551 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4a8005020 con 0x7fe4b01028c0 2024-01-31T05:20:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.522+0000 7fe497fff640 1 -- 172.21.15.45:0/2117348551 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a80116d0 con 0x7fe4b01028c0 2024-01-31T05:20:03.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.522+0000 7fe4b5619640 1 -- 172.21.15.45:0/2117348551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b019f210 con 0x7fe4b01028c0 2024-01-31T05:20:03.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.523+0000 7fe4b5619640 1 -- 172.21.15.45:0/2117348551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b0108100 con 0x7fe4b01028c0 2024-01-31T05:20:03.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.524+0000 7fe497fff640 1 -- 172.21.15.45:0/2117348551 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe4a8005190 con 0x7fe4b01028c0 2024-01-31T05:20:03.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.524+0000 7fe497fff640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe4980a12e0 0x7fe4980a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.524+0000 7fe4aeffd640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe4980a12e0 0x7fe4980a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.524+0000 7fe497fff640 1 -- 172.21.15.45:0/2117348551 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe4a80ccd40 con 0x7fe4b01028c0 2024-01-31T05:20:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.525+0000 7fe4aeffd640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe4980a12e0 0x7fe4980a37a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0000c00 tx=0x7fe4a002c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:03.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.528+0000 7fe497fff640 1 -- 172.21.15.45:0/2117348551 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe4a80965d0 con 0x7fe4b01028c0 2024-01-31T05:20:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.744+0000 7fe4b5619640 1 -- 172.21.15.45:0/2117348551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4b010c5a0 con 0x7fe4b01028c0 2024-01-31T05:20:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.745+0000 7fe497fff640 1 -- 172.21.15.45:0/2117348551 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fe4a8098500 con 0x7fe4b01028c0 2024-01-31T05:20:03.744 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:03.744 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-01-31T05:20:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 1 2024-01-31T05:20:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.747+0000 7fe4b5619640 1 -- 172.21.15.45:0/2117348551 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe4980a12e0 msgr2=0x7fe4980a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.747+0000 7fe4b5619640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe4980a12e0 0x7fe4980a37a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0000c00 tx=0x7fe4a002c5d0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.747+0000 7fe4b5619640 1 -- 172.21.15.45:0/2117348551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b01028c0 msgr2=0x7fe4b019e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.747+0000 7fe4b5619640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b01028c0 0x7fe4b019e270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800eaa0 tx=0x7fe4a800ef60 comp rx=0 tx=0).stop 2024-01-31T05:20:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.747+0000 7fe4b5619640 1 -- 172.21.15.45:0/2117348551 shutdown_connections 2024-01-31T05:20:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.747+0000 7fe4b5619640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe4980a12e0 0x7fe4980a37a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.747+0000 7fe4b5619640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b01028c0 0x7fe4b019e270 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.747+0000 7fe4b5619640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4b0101f00 0x7fe4b0197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.747+0000 7fe4b5619640 1 --2- 172.21.15.45:0/2117348551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4b0100d10 0x7fe4b0196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.747+0000 7fe4b5619640 1 -- 172.21.15.45:0/2117348551 >> 172.21.15.45:0/2117348551 conn(0x7fe4b00fc2a0 msgr2=0x7fe4b01052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.747+0000 7fe4b5619640 1 -- 172.21.15.45:0/2117348551 shutdown_connections 2024-01-31T05:20:03.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:03.747+0000 7fe4b5619640 1 -- 172.21.15.45:0/2117348551 wait complete. 2024-01-31T05:20:04.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:03 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1757246736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:04.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:03 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2168135884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:20:04.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:03 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2117348551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:04.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:03 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1757246736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:04.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:03 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2168135884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:20:04.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:03 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2117348551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:04.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:03 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1757246736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:04.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:03 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2168135884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:20:04.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:03 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2117348551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:04.795 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:20:04 smithi186 systemd[1]: Starting Ceph mds.l for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:20:05.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:04 smithi045 ceph-mon[31585]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:05.061 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:04 smithi186 ceph-mon[35659]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:05.062 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:20:04 smithi186 podman[66034]: 2024-01-31 05:20:04.729170795 +0000 UTC m=+0.018349673 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:20:05.062 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:20:04 smithi186 podman[66034]: 2024-01-31 05:20:04.835233021 +0000 UTC m=+0.124411918 container create ab8db6740b0a6ede1119f591d4bb07229546071db57634f6c40003321c5bd783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-01-31T05:20:05.062 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:20:05 smithi186 podman[66034]: 2024-01-31 05:20:05.063644542 +0000 UTC m=+0.352823419 container init ab8db6740b0a6ede1119f591d4bb07229546071db57634f6c40003321c5bd783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS) 2024-01-31T05:20:05.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:04 smithi156 ceph-mon[35720]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:05.341 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:20:05 smithi186 podman[66034]: 2024-01-31 05:20:05.06647783 +0000 UTC m=+0.355656721 container start ab8db6740b0a6ede1119f591d4bb07229546071db57634f6c40003321c5bd783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-31T05:20:05.341 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:20:05 smithi186 ceph-mds[66053]: -- 172.21.15.186:0/3436639380 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5615b2207520 con 0x5615b21dc800 2024-01-31T05:20:05.341 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:20:05 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l[66049]: starting mds.l at 2024-01-31T05:20:05.341 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:20:05 smithi186 bash[66034]: ab8db6740b0a6ede1119f591d4bb07229546071db57634f6c40003321c5bd783 2024-01-31T05:20:05.341 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:20:05 smithi186 systemd[1]: Started Ceph mds.l for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:20:05.758 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.863+0000 7faeb8342640 1 -- 172.21.15.45:0/3465717130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb01069f0 msgr2=0x7faeb0108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.863+0000 7faeb8342640 1 --2- 172.21.15.45:0/3465717130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb01069f0 0x7faeb0108e40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faea8009ca0 tx=0x7faea802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.863+0000 7faeb8342640 1 -- 172.21.15.45:0/3465717130 shutdown_connections 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.863+0000 7faeb8342640 1 --2- 172.21.15.45:0/3465717130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb01069f0 0x7faeb0108e40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.863+0000 7faeb8342640 1 --2- 172.21.15.45:0/3465717130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeb01040c0 0x7faeb01064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.863+0000 7faeb8342640 1 --2- 172.21.15.45:0/3465717130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb0101790 0x7faeb0103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.863+0000 7faeb8342640 1 -- 172.21.15.45:0/3465717130 >> 172.21.15.45:0/3465717130 conn(0x7faeb00fb180 msgr2=0x7faeb00fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.863+0000 7faeb8342640 1 -- 172.21.15.45:0/3465717130 shutdown_connections 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.863+0000 7faeb8342640 1 -- 172.21.15.45:0/3465717130 wait complete. 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb8342640 1 Processor -- start 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb8342640 1 -- start start 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb8342640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb0101790 0x7faeb0101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb8342640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeb01040c0 0x7faeb00ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb8342640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb01069f0 0x7faeb00ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb8342640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb01003e0 con 0x7faeb0101790 2024-01-31T05:20:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb8342640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb0100550 con 0x7faeb01040c0 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb8342640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb01a0d30 con 0x7faeb01069f0 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb60de640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb0101790 0x7faeb0101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb60de640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb0101790 0x7faeb0101100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36558/0 (socket says 172.21.15.45:36558) 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb60de640 1 -- 172.21.15.45:0/2826981894 learned_addr learned my addr 172.21.15.45:0/2826981894 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb58dd640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeb01040c0 0x7faeb00ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb68df640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb01069f0 0x7faeb00ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb60de640 1 -- 172.21.15.45:0/2826981894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb01069f0 msgr2=0x7faeb00ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb60de640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb01069f0 0x7faeb00ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb60de640 1 -- 172.21.15.45:0/2826981894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeb01040c0 msgr2=0x7faeb00ff740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb60de640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeb01040c0 0x7faeb00ff740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb60de640 1 -- 172.21.15.45:0/2826981894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea8009900 con 0x7faeb0101790 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.864+0000 7faeb58dd640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeb01040c0 0x7faeb00ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.865+0000 7faeb60de640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb0101790 0x7faeb0101100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faea000c9a0 tx=0x7faea000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.865+0000 7faea77fe640 1 -- 172.21.15.45:0/2826981894 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0007bb0 con 0x7faeb0101790 2024-01-31T05:20:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.865+0000 7faeb8342640 1 -- 172.21.15.45:0/2826981894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb01a0fe0 con 0x7faeb0101790 2024-01-31T05:20:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.865+0000 7faea77fe640 1 -- 172.21.15.45:0/2826981894 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faea0007d10 con 0x7faeb0101790 2024-01-31T05:20:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.865+0000 7faea77fe640 1 -- 172.21.15.45:0/2826981894 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea000f450 con 0x7faeb0101790 2024-01-31T05:20:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.865+0000 7faeb8342640 1 -- 172.21.15.45:0/2826981894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb01a1400 con 0x7faeb0101790 2024-01-31T05:20:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.866+0000 7faeb8342640 1 -- 172.21.15.45:0/2826981894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb00fcc70 con 0x7faeb0101790 2024-01-31T05:20:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.868+0000 7faea77fe640 1 -- 172.21.15.45:0/2826981894 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7faea0004ef0 con 0x7faeb0101790 2024-01-31T05:20:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.868+0000 7faea77fe640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fae9c0a12e0 0x7fae9c0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.868+0000 7faeb58dd640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fae9c0a12e0 0x7fae9c0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.868+0000 7faea77fe640 1 -- 172.21.15.45:0/2826981894 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faea00cc8a0 con 0x7faeb0101790 2024-01-31T05:20:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.869+0000 7faeb58dd640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fae9c0a12e0 0x7fae9c0a37a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faeac0058c0 tx=0x7faeac005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:05.872+0000 7faea77fe640 1 -- 172.21.15.45:0/2826981894 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7faea00961c0 con 0x7faeb0101790 2024-01-31T05:20:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.088+0000 7faeb8342640 1 -- 172.21.15.45:0/2826981894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faeb010c850 con 0x7faeb0101790 2024-01-31T05:20:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.088+0000 7faea77fe640 1 -- 172.21.15.45:0/2826981894 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7faea001d090 con 0x7faeb0101790 2024-01-31T05:20:06.088 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:06.088 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-31T05:20:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 2 2024-01-31T05:20:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.090+0000 7faeb8342640 1 -- 172.21.15.45:0/2826981894 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fae9c0a12e0 msgr2=0x7fae9c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.090+0000 7faeb8342640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fae9c0a12e0 0x7fae9c0a37a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faeac0058c0 tx=0x7faeac005850 comp rx=0 tx=0).stop 2024-01-31T05:20:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.090+0000 7faeb8342640 1 -- 172.21.15.45:0/2826981894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb0101790 msgr2=0x7faeb0101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.090+0000 7faeb8342640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb0101790 0x7faeb0101100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faea000c9a0 tx=0x7faea000ce60 comp rx=0 tx=0).stop 2024-01-31T05:20:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.090+0000 7faeb8342640 1 -- 172.21.15.45:0/2826981894 shutdown_connections 2024-01-31T05:20:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.090+0000 7faeb8342640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fae9c0a12e0 0x7fae9c0a37a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.090+0000 7faeb8342640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb01069f0 0x7faeb00ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.090+0000 7faeb8342640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeb01040c0 0x7faeb00ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.090+0000 7faeb8342640 1 --2- 172.21.15.45:0/2826981894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb0101790 0x7faeb0101100 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.090+0000 7faeb8342640 1 -- 172.21.15.45:0/2826981894 >> 172.21.15.45:0/2826981894 conn(0x7faeb00fb180 msgr2=0x7faeb01020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.090+0000 7faeb8342640 1 -- 172.21.15.45:0/2826981894 shutdown_connections 2024-01-31T05:20:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:06.090+0000 7faeb8342640 1 -- 172.21.15.45:0/2826981894 wait complete. 2024-01-31T05:20:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:06 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.186:6832/3707036230,v1:172.21.15.186:6833/3707036230] up:boot 2024-01-31T05:20:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:06 smithi045 ceph-mon[31585]: fsmap 1 up:standby 2024-01-31T05:20:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:06 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-31T05:20:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:06 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:06 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:06 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:06 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:06 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:06 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:06 smithi045 ceph-mon[31585]: Deploying daemon mds.k on smithi156 2024-01-31T05:20:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:06 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2826981894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:06.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:06 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.186:6832/3707036230,v1:172.21.15.186:6833/3707036230] up:boot 2024-01-31T05:20:06.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:06 smithi156 ceph-mon[35720]: fsmap 1 up:standby 2024-01-31T05:20:06.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:06 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-31T05:20:06.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:06 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:06.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:06 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:06.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:06 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:06.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:06 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:06.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:06 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:06.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:06 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:06.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:06 smithi156 ceph-mon[35720]: Deploying daemon mds.k on smithi156 2024-01-31T05:20:06.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:06 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2826981894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:06.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:06 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.186:6832/3707036230,v1:172.21.15.186:6833/3707036230] up:boot 2024-01-31T05:20:06.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:06 smithi186 ceph-mon[35659]: fsmap 1 up:standby 2024-01-31T05:20:06.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:06 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-31T05:20:06.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:06 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:06.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:06 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:06.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:06 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:06.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:06 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:06.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:06 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:06.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:06 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:06.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:06 smithi186 ceph-mon[35659]: Deploying daemon mds.k on smithi156 2024-01-31T05:20:06.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:06 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2826981894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:07.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:07 smithi045 ceph-mon[31585]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:07.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:07 smithi156 ceph-mon[35720]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:07.476 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:20:07 smithi156 systemd[1]: Starting Ceph mds.k for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:20:07.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:07 smithi186 ceph-mon[35659]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:07.842 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:20:07 smithi156 podman[65636]: 2024-01-31 05:20:07.453313867 +0000 UTC m=+0.017795651 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:20:07.842 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:20:07 smithi156 podman[65636]: 2024-01-31 05:20:07.5574404 +0000 UTC m=+0.121922216 container create 6ee9d87f4ebf0e4dd5e470a266d9d360745b12d0e80b25687103d58f94086268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-k, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.31.3) 2024-01-31T05:20:07.842 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:20:07 smithi156 podman[65636]: 2024-01-31 05:20:07.759943618 +0000 UTC m=+0.324425400 container init 6ee9d87f4ebf0e4dd5e470a266d9d360745b12d0e80b25687103d58f94086268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-k, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, ceph=True) 2024-01-31T05:20:07.842 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:20:07 smithi156 podman[65636]: 2024-01-31 05:20:07.76269236 +0000 UTC m=+0.327174142 container start 6ee9d87f4ebf0e4dd5e470a266d9d360745b12d0e80b25687103d58f94086268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-k, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-01-31T05:20:07.842 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:20:07 smithi156 ceph-mds[65655]: -- 172.21.15.156:0/2996843443 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bbe3291520 con 0x55bbe3267000 2024-01-31T05:20:07.842 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:20:07 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-k[65651]: starting mds.k at 2024-01-31T05:20:07.842 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:20:07 smithi156 bash[65636]: 6ee9d87f4ebf0e4dd5e470a266d9d360745b12d0e80b25687103d58f94086268 2024-01-31T05:20:08.100 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:08.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.205+0000 7fe6d671f640 1 -- 172.21.15.45:0/1566858785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d01017c0 msgr2=0x7fe6d0108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:08.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.205+0000 7fe6d671f640 1 --2- 172.21.15.45:0/1566858785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d01017c0 0x7fe6d0108050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009ca0 tx=0x7fe6c802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.205+0000 7fe6d671f640 1 -- 172.21.15.45:0/1566858785 shutdown_connections 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.205+0000 7fe6d671f640 1 --2- 172.21.15.45:0/1566858785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d01017c0 0x7fe6d0108050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.205+0000 7fe6d671f640 1 --2- 172.21.15.45:0/1566858785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6d0100e00 0x7fe6d0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.205+0000 7fe6d671f640 1 --2- 172.21.15.45:0/1566858785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d00ffc10 0x7fe6d0100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.205+0000 7fe6d671f640 1 -- 172.21.15.45:0/1566858785 >> 172.21.15.45:0/1566858785 conn(0x7fe6d00fb180 msgr2=0x7fe6d00fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.205+0000 7fe6d671f640 1 -- 172.21.15.45:0/1566858785 shutdown_connections 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.205+0000 7fe6d671f640 1 -- 172.21.15.45:0/1566858785 wait complete. 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6d671f640 1 Processor -- start 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6d671f640 1 -- start start 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6d671f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d00ffc10 0x7fe6d0196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6d671f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d0100e00 0x7fe6d0197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6d671f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6d01017c0 0x7fe6d019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6d671f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d019e790 con 0x7fe6d0100e00 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6d671f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d019e900 con 0x7fe6d01017c0 2024-01-31T05:20:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6d671f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d019eac0 con 0x7fe6d00ffc10 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6cf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d0100e00 0x7fe6d0197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6cf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d0100e00 0x7fe6d0197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59026/0 (socket says 172.21.15.45:59026) 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6cf7fe640 1 -- 172.21.15.45:0/3356157567 learned_addr learned my addr 172.21.15.45:0/3356157567 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6cffff640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d00ffc10 0x7fe6d0196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6d4cbc640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6d01017c0 0x7fe6d019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6cf7fe640 1 -- 172.21.15.45:0/3356157567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d00ffc10 msgr2=0x7fe6d0196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6cf7fe640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d00ffc10 0x7fe6d0196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6cf7fe640 1 -- 172.21.15.45:0/3356157567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6d01017c0 msgr2=0x7fe6d019e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6cf7fe640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6d01017c0 0x7fe6d019e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6cf7fe640 1 -- 172.21.15.45:0/3356157567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c8009900 con 0x7fe6d0100e00 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.206+0000 7fe6cf7fe640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d0100e00 0x7fe6d0197210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400e9f0 tx=0x7fe6c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.207+0000 7fe6cd7fa640 1 -- 172.21.15.45:0/3356157567 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4009700 con 0x7fe6d0100e00 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.207+0000 7fe6d671f640 1 -- 172.21.15.45:0/3356157567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d019eda0 con 0x7fe6d0100e00 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.207+0000 7fe6cd7fa640 1 -- 172.21.15.45:0/3356157567 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6c400cd80 con 0x7fe6d0100e00 2024-01-31T05:20:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.207+0000 7fe6d671f640 1 -- 172.21.15.45:0/3356157567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d019f270 con 0x7fe6d0100e00 2024-01-31T05:20:08.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.207+0000 7fe6cd7fa640 1 -- 172.21.15.45:0/3356157567 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4018860 con 0x7fe6d0100e00 2024-01-31T05:20:08.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.208+0000 7fe6d671f640 1 -- 172.21.15.45:0/3356157567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d010b4d0 con 0x7fe6d0100e00 2024-01-31T05:20:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.209+0000 7fe6cd7fa640 1 -- 172.21.15.45:0/3356157567 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe6c40189c0 con 0x7fe6d0100e00 2024-01-31T05:20:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.210+0000 7fe6cd7fa640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6b80a12e0 0x7fe6b80a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.210+0000 7fe6cffff640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6b80a12e0 0x7fe6b80a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.210+0000 7fe6cd7fa640 1 -- 172.21.15.45:0/3356157567 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe6c4014070 con 0x7fe6d0100e00 2024-01-31T05:20:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.210+0000 7fe6cffff640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6b80a12e0 0x7fe6b80a37a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0068e30 tx=0x7fe6c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:08.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.213+0000 7fe6cd7fa640 1 -- 172.21.15.45:0/3356157567 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe6c40967f0 con 0x7fe6d0100e00 2024-01-31T05:20:08.226 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:20:07 smithi156 systemd[1]: Started Ceph mds.k for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:20:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.434+0000 7fe6d671f640 1 -- 172.21.15.45:0/3356157567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6d019f7a0 con 0x7fe6d0100e00 2024-01-31T05:20:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.435+0000 7fe6cd7fa640 1 -- 172.21.15.45:0/3356157567 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fe6c4099e10 con 0x7fe6d0100e00 2024-01-31T05:20:08.434 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:08.434 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-31T05:20:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 2 2024-01-31T05:20:08.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.438+0000 7fe6d671f640 1 -- 172.21.15.45:0/3356157567 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6b80a12e0 msgr2=0x7fe6b80a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:08.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.438+0000 7fe6d671f640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6b80a12e0 0x7fe6b80a37a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0068e30 tx=0x7fe6c0005f90 comp rx=0 tx=0).stop 2024-01-31T05:20:08.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.438+0000 7fe6d671f640 1 -- 172.21.15.45:0/3356157567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d0100e00 msgr2=0x7fe6d0197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:08.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.438+0000 7fe6d671f640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d0100e00 0x7fe6d0197210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400e9f0 tx=0x7fe6c400eeb0 comp rx=0 tx=0).stop 2024-01-31T05:20:08.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.438+0000 7fe6d671f640 1 -- 172.21.15.45:0/3356157567 shutdown_connections 2024-01-31T05:20:08.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.438+0000 7fe6d671f640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6b80a12e0 0x7fe6b80a37a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:08.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.438+0000 7fe6d671f640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d0100e00 0x7fe6d0197210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:08.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.438+0000 7fe6d671f640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6d01017c0 0x7fe6d019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:08.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.438+0000 7fe6d671f640 1 --2- 172.21.15.45:0/3356157567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d00ffc10 0x7fe6d0196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:08.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.438+0000 7fe6d671f640 1 -- 172.21.15.45:0/3356157567 >> 172.21.15.45:0/3356157567 conn(0x7fe6d00fb180 msgr2=0x7fe6d0067e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:08.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.438+0000 7fe6d671f640 1 -- 172.21.15.45:0/3356157567 shutdown_connections 2024-01-31T05:20:08.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:08.438+0000 7fe6d671f640 1 -- 172.21.15.45:0/3356157567 wait complete. 2024-01-31T05:20:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:08 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:08 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:08 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:08 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:08 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:08 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:08 smithi045 ceph-mon[31585]: Deploying daemon mds.j on smithi045 2024-01-31T05:20:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:08 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3356157567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:08 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.156:6832/2627326005,v1:172.21.15.156:6833/2627326005] up:boot 2024-01-31T05:20:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:08 smithi045 ceph-mon[31585]: fsmap 2 up:standby 2024-01-31T05:20:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:08 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-31T05:20:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:08 smithi045 ceph-mon[31585]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:08 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:08 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:08 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:08 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:08 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:08 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:08 smithi156 ceph-mon[35720]: Deploying daemon mds.j on smithi045 2024-01-31T05:20:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:08 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3356157567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:08 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.156:6832/2627326005,v1:172.21.15.156:6833/2627326005] up:boot 2024-01-31T05:20:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:08 smithi156 ceph-mon[35720]: fsmap 2 up:standby 2024-01-31T05:20:09.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:08 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-31T05:20:09.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:08 smithi156 ceph-mon[35720]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:09.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:08 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:09.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:08 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:08 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:08 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:08 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:08 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:08 smithi186 ceph-mon[35659]: Deploying daemon mds.j on smithi045 2024-01-31T05:20:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:08 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3356157567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:08 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.156:6832/2627326005,v1:172.21.15.156:6833/2627326005] up:boot 2024-01-31T05:20:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:08 smithi186 ceph-mon[35659]: fsmap 2 up:standby 2024-01-31T05:20:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:08 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-31T05:20:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:08 smithi186 ceph-mon[35659]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:10.095 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:20:09 smithi045 systemd[1]: Starting Ceph mds.j for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:20:10.348 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:20:10 smithi045 podman[72098]: 2024-01-31 05:20:10.000285026 +0000 UTC m=+0.018760218 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:20:10.348 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:20:10 smithi045 podman[72098]: 2024-01-31 05:20:10.104793479 +0000 UTC m=+0.123268651 container create aa11c2272224b2e3f2e5008710ffd818b484f3da8d95fa2874a1b9d47eaec252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-01-31T05:20:10.348 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:20:10 smithi045 podman[72098]: 2024-01-31 05:20:10.324379822 +0000 UTC m=+0.342854997 container init aa11c2272224b2e3f2e5008710ffd818b484f3da8d95fa2874a1b9d47eaec252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-j, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-31T05:20:10.349 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:20:10 smithi045 podman[72098]: 2024-01-31 05:20:10.327044287 +0000 UTC m=+0.345519468 container start aa11c2272224b2e3f2e5008710ffd818b484f3da8d95fa2874a1b9d47eaec252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-j, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-01-31T05:20:10.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:10.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.559+0000 7f3825692640 1 -- 172.21.15.45:0/1215811316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820104120 msgr2=0x7f3820106510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:10.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.559+0000 7f3825692640 1 --2- 172.21.15.45:0/1215811316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820104120 0x7f3820106510 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3814009900 tx=0x7f381402bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.559+0000 7f3825692640 1 -- 172.21.15.45:0/1215811316 shutdown_connections 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.559+0000 7f3825692640 1 --2- 172.21.15.45:0/1215811316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3820104120 0x7f3820106510 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.559+0000 7f3825692640 1 --2- 172.21.15.45:0/1215811316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3820106a50 0x7f3820108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.559+0000 7f3825692640 1 --2- 172.21.15.45:0/1215811316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38201017f0 0x7f3820103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.559+0000 7f3825692640 1 -- 172.21.15.45:0/1215811316 >> 172.21.15.45:0/1215811316 conn(0x7f38200fb180 msgr2=0x7f38200fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.559+0000 7f3825692640 1 -- 172.21.15.45:0/1215811316 shutdown_connections 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.559+0000 7f3825692640 1 -- 172.21.15.45:0/1215811316 wait complete. 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.560+0000 7f3825692640 1 Processor -- start 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.560+0000 7f3825692640 1 -- start start 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.560+0000 7f3825692640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38201017f0 0x7f3820101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.560+0000 7f3825692640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3820104120 0x7f38200ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.560+0000 7f3825692640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3820106a50 0x7f38200ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.560+0000 7f3825692640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3820100440 con 0x7f38201017f0 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.560+0000 7f3825692640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38201005b0 con 0x7f3820106a50 2024-01-31T05:20:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.560+0000 7f3825692640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38201a0d10 con 0x7f3820104120 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.560+0000 7f381effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38201017f0 0x7f3820101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.560+0000 7f381effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38201017f0 0x7f3820101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59056/0 (socket says 172.21.15.45:59056) 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.560+0000 7f381effd640 1 -- 172.21.15.45:0/3480889376 learned_addr learned my addr 172.21.15.45:0/3480889376 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.560+0000 7f381f7fe640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3820106a50 0x7f38200ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.561+0000 7f381e7fc640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3820104120 0x7f38200ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.561+0000 7f381effd640 1 -- 172.21.15.45:0/3480889376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3820104120 msgr2=0x7f38200ff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.561+0000 7f381effd640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3820104120 0x7f38200ff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.561+0000 7f381effd640 1 -- 172.21.15.45:0/3480889376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3820106a50 msgr2=0x7f38200ffce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.561+0000 7f381effd640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3820106a50 0x7f38200ffce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.561+0000 7f381effd640 1 -- 172.21.15.45:0/3480889376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3814009560 con 0x7f38201017f0 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.561+0000 7f381effd640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38201017f0 0x7f3820101160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f381000c950 tx=0x7f381000ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.561+0000 7f3807fff640 1 -- 172.21.15.45:0/3480889376 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810007c70 con 0x7f38201017f0 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.561+0000 7f3825692640 1 -- 172.21.15.45:0/3480889376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38201a0fc0 con 0x7f38201017f0 2024-01-31T05:20:10.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.561+0000 7f3807fff640 1 -- 172.21.15.45:0/3480889376 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3810007dd0 con 0x7f38201017f0 2024-01-31T05:20:10.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.561+0000 7f3807fff640 1 -- 172.21.15.45:0/3480889376 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810018800 con 0x7f38201017f0 2024-01-31T05:20:10.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.561+0000 7f3825692640 1 -- 172.21.15.45:0/3480889376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38201a13f0 con 0x7f38201017f0 2024-01-31T05:20:10.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.562+0000 7f3825692640 1 -- 172.21.15.45:0/3480889376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38200fcc70 con 0x7f38201017f0 2024-01-31T05:20:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.563+0000 7f3807fff640 1 -- 172.21.15.45:0/3480889376 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f381000faa0 con 0x7f38201017f0 2024-01-31T05:20:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.564+0000 7f3807fff640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f38080a12e0 0x7f38080a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.564+0000 7f381e7fc640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f38080a12e0 0x7f38080a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.564+0000 7f3807fff640 1 -- 172.21.15.45:0/3480889376 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f38100cc9d0 con 0x7f38201017f0 2024-01-31T05:20:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.564+0000 7f381e7fc640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f38080a12e0 0x7f38080a37a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3814000c00 tx=0x7f381402c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.568+0000 7f3807fff640 1 -- 172.21.15.45:0/3480889376 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f3810096380 con 0x7f38201017f0 2024-01-31T05:20:10.642 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:20:10 smithi045 ceph-mds[72117]: -- 172.21.15.45:0/529885366 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556e7a73b520 con 0x556e7a711000 2024-01-31T05:20:10.642 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:20:10 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-j[72113]: starting mds.j at 2024-01-31T05:20:10.642 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:20:10 smithi045 bash[72098]: aa11c2272224b2e3f2e5008710ffd818b484f3da8d95fa2874a1b9d47eaec252 2024-01-31T05:20:10.643 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:20:10 smithi045 systemd[1]: Started Ceph mds.j for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:20:10.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.786+0000 7f3825692640 1 -- 172.21.15.45:0/3480889376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f382010c5a0 con 0x7f38201017f0 2024-01-31T05:20:10.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.787+0000 7f3807fff640 1 -- 172.21.15.45:0/3480889376 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f38100999a0 con 0x7f38201017f0 2024-01-31T05:20:10.786 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:10.786 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-31T05:20:10.787 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 4 2024-01-31T05:20:10.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.789+0000 7f3825692640 1 -- 172.21.15.45:0/3480889376 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f38080a12e0 msgr2=0x7f38080a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:10.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.789+0000 7f3825692640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f38080a12e0 0x7f38080a37a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3814000c00 tx=0x7f381402c5d0 comp rx=0 tx=0).stop 2024-01-31T05:20:10.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.789+0000 7f3825692640 1 -- 172.21.15.45:0/3480889376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38201017f0 msgr2=0x7f3820101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:10.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.789+0000 7f3825692640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38201017f0 0x7f3820101160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f381000c950 tx=0x7f381000ce10 comp rx=0 tx=0).stop 2024-01-31T05:20:10.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.789+0000 7f3825692640 1 -- 172.21.15.45:0/3480889376 shutdown_connections 2024-01-31T05:20:10.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.789+0000 7f3825692640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f38080a12e0 0x7f38080a37a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:10.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.789+0000 7f3825692640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3820106a50 0x7f38200ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:10.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.789+0000 7f3825692640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3820104120 0x7f38200ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:10.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.789+0000 7f3825692640 1 --2- 172.21.15.45:0/3480889376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38201017f0 0x7f3820101160 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:10.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.789+0000 7f3825692640 1 -- 172.21.15.45:0/3480889376 >> 172.21.15.45:0/3480889376 conn(0x7f38200fb180 msgr2=0x7f3820102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:10.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.789+0000 7f3825692640 1 -- 172.21.15.45:0/3480889376 shutdown_connections 2024-01-31T05:20:10.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:10.789+0000 7f3825692640 1 -- 172.21.15.45:0/3480889376 wait complete. 2024-01-31T05:20:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:11 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.45:6834/363421179,v1:172.21.15.45:6835/363421179] up:boot 2024-01-31T05:20:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:11 smithi156 ceph-mon[35720]: fsmap 3 up:standby 2024-01-31T05:20:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-31T05:20:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:11.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:11 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:11 smithi156 ceph-mon[35720]: Deploying daemon mds.d on smithi045 2024-01-31T05:20:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:11 smithi156 ceph-mon[35720]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:11.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:11 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3480889376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:11.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:11 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.45:6834/363421179,v1:172.21.15.45:6835/363421179] up:boot 2024-01-31T05:20:11.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:11 smithi186 ceph-mon[35659]: fsmap 3 up:standby 2024-01-31T05:20:11.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-31T05:20:11.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:11.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:11.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:11.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:11.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:11.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:11 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:11.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:11 smithi186 ceph-mon[35659]: Deploying daemon mds.d on smithi045 2024-01-31T05:20:11.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:11 smithi186 ceph-mon[35659]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:11.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:11 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3480889376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:11 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.45:6834/363421179,v1:172.21.15.45:6835/363421179] up:boot 2024-01-31T05:20:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:11 smithi045 ceph-mon[31585]: fsmap 3 up:standby 2024-01-31T05:20:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-31T05:20:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:11 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:11 smithi045 ceph-mon[31585]: Deploying daemon mds.d on smithi045 2024-01-31T05:20:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:11 smithi045 ceph-mon[31585]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:11.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:11 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3480889376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:12.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.916+0000 7f16c7ffb640 1 -- 172.21.15.45:0/2512040388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00ff580 msgr2=0x7f16c01019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.916+0000 7f16c7ffb640 1 --2- 172.21.15.45:0/2512040388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00ff580 0x7f16c01019b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009900 tx=0x7f16bc02bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.916+0000 7f16c7ffb640 1 -- 172.21.15.45:0/2512040388 shutdown_connections 2024-01-31T05:20:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.916+0000 7f16c7ffb640 1 --2- 172.21.15.45:0/2512040388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c01048a0 0x7f16c0104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.916+0000 7f16c7ffb640 1 --2- 172.21.15.45:0/2512040388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c0101ef0 0x7f16c0104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.916+0000 7f16c7ffb640 1 --2- 172.21.15.45:0/2512040388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00ff580 0x7f16c01019b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.916+0000 7f16c7ffb640 1 -- 172.21.15.45:0/2512040388 >> 172.21.15.45:0/2512040388 conn(0x7f16c00fb180 msgr2=0x7f16c00fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.916+0000 7f16c7ffb640 1 -- 172.21.15.45:0/2512040388 shutdown_connections 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.916+0000 7f16c7ffb640 1 -- 172.21.15.45:0/2512040388 wait complete. 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.916+0000 7f16c7ffb640 1 Processor -- start 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.916+0000 7f16c7ffb640 1 -- start start 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c7ffb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00ff580 0x7f16c019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c7ffb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0101ef0 0x7f16c019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c7ffb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c01048a0 0x7f16c01a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c7ffb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c01a4dc0 con 0x7f16c00ff580 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c7ffb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c01a4f30 con 0x7f16c01048a0 2024-01-31T05:20:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c7ffb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c01a50f0 con 0x7f16c0101ef0 2024-01-31T05:20:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c5d97640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00ff580 0x7f16c019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c5d97640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00ff580 0x7f16c019d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59068/0 (socket says 172.21.15.45:59068) 2024-01-31T05:20:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c5d97640 1 -- 172.21.15.45:0/1019754771 learned_addr learned my addr 172.21.15.45:0/1019754771 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c5596640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0101ef0 0x7f16c019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c6598640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c01048a0 0x7f16c01a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c5d97640 1 -- 172.21.15.45:0/1019754771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0101ef0 msgr2=0x7f16c019d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c5d97640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0101ef0 0x7f16c019d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c5d97640 1 -- 172.21.15.45:0/1019754771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c01048a0 msgr2=0x7f16c01a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c5d97640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c01048a0 0x7f16c01a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c5d97640 1 -- 172.21.15.45:0/1019754771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc009560 con 0x7f16c00ff580 2024-01-31T05:20:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c5596640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0101ef0 0x7f16c019d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16c5d97640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00ff580 0x7f16c019d320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16bc000c00 tx=0x7f16bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.917+0000 7f16b6ffd640 1 -- 172.21.15.45:0/1019754771 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc0044e0 con 0x7f16c00ff580 2024-01-31T05:20:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.918+0000 7f16c7ffb640 1 -- 172.21.15.45:0/1019754771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c01a5370 con 0x7f16c00ff580 2024-01-31T05:20:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.918+0000 7f16c7ffb640 1 -- 172.21.15.45:0/1019754771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c01a58d0 con 0x7f16c00ff580 2024-01-31T05:20:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.918+0000 7f16b6ffd640 1 -- 172.21.15.45:0/1019754771 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16bc003cc0 con 0x7f16c00ff580 2024-01-31T05:20:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.918+0000 7f16b6ffd640 1 -- 172.21.15.45:0/1019754771 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc03e450 con 0x7f16c00ff580 2024-01-31T05:20:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.919+0000 7f16c7ffb640 1 -- 172.21.15.45:0/1019754771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c00fcc70 con 0x7f16c00ff580 2024-01-31T05:20:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.920+0000 7f16b6ffd640 1 -- 172.21.15.45:0/1019754771 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f16bc004dd0 con 0x7f16c00ff580 2024-01-31T05:20:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.921+0000 7f16b6ffd640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f16ac0a12e0 0x7f16ac0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.921+0000 7f16b6ffd640 1 -- 172.21.15.45:0/1019754771 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f16bc0eeae0 con 0x7f16c00ff580 2024-01-31T05:20:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.921+0000 7f16c5596640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f16ac0a12e0 0x7f16ac0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.921+0000 7f16c5596640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f16ac0a12e0 0x7f16ac0a37a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f16b0005d00 tx=0x7f16b0005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:12.925+0000 7f16b6ffd640 1 -- 172.21.15.45:0/1019754771 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f16bc0b8400 con 0x7f16c00ff580 2024-01-31T05:20:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.145+0000 7f16c7ffb640 1 -- 172.21.15.45:0/1019754771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16c0112c80 con 0x7f16c00ff580 2024-01-31T05:20:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.145+0000 7f16b6ffd640 1 -- 172.21.15.45:0/1019754771 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f16bc043b70 con 0x7f16c00ff580 2024-01-31T05:20:13.145 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:13.145 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-31T05:20:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 4 2024-01-31T05:20:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.147+0000 7f16c7ffb640 1 -- 172.21.15.45:0/1019754771 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f16ac0a12e0 msgr2=0x7f16ac0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.147+0000 7f16c7ffb640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f16ac0a12e0 0x7f16ac0a37a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f16b0005d00 tx=0x7f16b0005c70 comp rx=0 tx=0).stop 2024-01-31T05:20:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.147+0000 7f16c7ffb640 1 -- 172.21.15.45:0/1019754771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00ff580 msgr2=0x7f16c019d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.147+0000 7f16c7ffb640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00ff580 0x7f16c019d320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16bc000c00 tx=0x7f16bc0042f0 comp rx=0 tx=0).stop 2024-01-31T05:20:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.147+0000 7f16c7ffb640 1 -- 172.21.15.45:0/1019754771 shutdown_connections 2024-01-31T05:20:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.147+0000 7f16c7ffb640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f16ac0a12e0 0x7f16ac0a37a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.147+0000 7f16c7ffb640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c01048a0 0x7f16c01a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.148+0000 7f16c7ffb640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c0101ef0 0x7f16c019d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.148+0000 7f16c7ffb640 1 --2- 172.21.15.45:0/1019754771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00ff580 0x7f16c019d320 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.148+0000 7f16c7ffb640 1 -- 172.21.15.45:0/1019754771 >> 172.21.15.45:0/1019754771 conn(0x7f16c00fb180 msgr2=0x7f16c01028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.148+0000 7f16c7ffb640 1 -- 172.21.15.45:0/1019754771 shutdown_connections 2024-01-31T05:20:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:13.148+0000 7f16c7ffb640 1 -- 172.21.15.45:0/1019754771 wait complete. 2024-01-31T05:20:13.278 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jan 31 05:20:13 smithi045 podman[72470]: 2024-01-31 05:20:13.256550481 +0000 UTC m=+0.351758475 container start 0500c71b12ff184f78c6313980d5abfde785f374c9c060091fabafe902911d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-d, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-01-31T05:20:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:13 smithi045 ceph-mon[31585]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:13 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1019754771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:13.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:13 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:13.644 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jan 31 05:20:13 smithi045 ceph-mds[72510]: -- 172.21.15.45:0/693296973 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55555aae1520 con 0x55555aab6800 2024-01-31T05:20:13.644 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jan 31 05:20:13 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-d[72504]: starting mds.d at 2024-01-31T05:20:13.644 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jan 31 05:20:13 smithi045 bash[72470]: 0500c71b12ff184f78c6313980d5abfde785f374c9c060091fabafe902911d22 2024-01-31T05:20:13.644 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jan 31 05:20:13 smithi045 systemd[1]: Started Ceph mds.d for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:20:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:13 smithi156 ceph-mon[35720]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:13 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1019754771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:13 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:13 smithi186 ceph-mon[35659]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:13 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1019754771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:13.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:13.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:13.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:13.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:13 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:14.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:14 smithi156 ceph-mon[35720]: Deploying daemon mds.e on smithi156 2024-01-31T05:20:14.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:14 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.45:6836/176161833,v1:172.21.15.45:6837/176161833] up:boot 2024-01-31T05:20:14.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:14 smithi156 ceph-mon[35720]: fsmap 4 up:standby 2024-01-31T05:20:14.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:14 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-31T05:20:14.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:14 smithi186 ceph-mon[35659]: Deploying daemon mds.e on smithi156 2024-01-31T05:20:14.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:14 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.45:6836/176161833,v1:172.21.15.45:6837/176161833] up:boot 2024-01-31T05:20:14.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:14 smithi186 ceph-mon[35659]: fsmap 4 up:standby 2024-01-31T05:20:14.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:14 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-31T05:20:14.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:14 smithi045 ceph-mon[31585]: Deploying daemon mds.e on smithi156 2024-01-31T05:20:14.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:14 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.45:6836/176161833,v1:172.21.15.45:6837/176161833] up:boot 2024-01-31T05:20:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:14 smithi045 ceph-mon[31585]: fsmap 4 up:standby 2024-01-31T05:20:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:14 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-31T05:20:15.159 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:15.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.275+0000 7fc9ce46e640 1 -- 172.21.15.45:0/1299233450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9c8100d10 msgr2=0x7fc9c8101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:15.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.275+0000 7fc9ce46e640 1 --2- 172.21.15.45:0/1299233450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9c8100d10 0x7fc9c8101130 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009900 tx=0x7fc9b802bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:15.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.276+0000 7fc9ce46e640 1 -- 172.21.15.45:0/1299233450 shutdown_connections 2024-01-31T05:20:15.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.276+0000 7fc9ce46e640 1 --2- 172.21.15.45:0/1299233450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9c8101f00 0x7fc9c8102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:15.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.276+0000 7fc9ce46e640 1 --2- 172.21.15.45:0/1299233450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9c81028c0 0x7fc9c8109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:15.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.276+0000 7fc9ce46e640 1 --2- 172.21.15.45:0/1299233450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9c8100d10 0x7fc9c8101130 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:15.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.276+0000 7fc9ce46e640 1 -- 172.21.15.45:0/1299233450 >> 172.21.15.45:0/1299233450 conn(0x7fc9c80fc2a0 msgr2=0x7fc9c80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:15.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.276+0000 7fc9ce46e640 1 -- 172.21.15.45:0/1299233450 shutdown_connections 2024-01-31T05:20:15.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.276+0000 7fc9ce46e640 1 -- 172.21.15.45:0/1299233450 wait complete. 2024-01-31T05:20:15.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.276+0000 7fc9ce46e640 1 Processor -- start 2024-01-31T05:20:15.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.276+0000 7fc9ce46e640 1 -- start start 2024-01-31T05:20:15.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.276+0000 7fc9ce46e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9c8100d10 0x7fc9c8196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:15.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.276+0000 7fc9ce46e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9c8101f00 0x7fc9c8197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9ce46e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9c81028c0 0x7fc9c819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9ce46e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c819e800 con 0x7fc9c8100d10 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9ce46e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c819e970 con 0x7fc9c8101f00 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9ce46e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c819eb30 con 0x7fc9c81028c0 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9c8100d10 0x7fc9c8196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9c8100d10 0x7fc9c8196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59114/0 (socket says 172.21.15.45:59114) 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c7fff640 1 -- 172.21.15.45:0/2324874277 learned_addr learned my addr 172.21.15.45:0/2324874277 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c77fe640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9c8101f00 0x7fc9c8197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9cca0b640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9c81028c0 0x7fc9c819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c7fff640 1 -- 172.21.15.45:0/2324874277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9c81028c0 msgr2=0x7fc9c819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c7fff640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9c81028c0 0x7fc9c819e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c7fff640 1 -- 172.21.15.45:0/2324874277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9c8101f00 msgr2=0x7fc9c8197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c7fff640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9c8101f00 0x7fc9c8197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c7fff640 1 -- 172.21.15.45:0/2324874277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc009490 con 0x7fc9c8100d10 2024-01-31T05:20:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c7fff640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9c8100d10 0x7fc9c8196d20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009230 tx=0x7fc9b8004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c57fa640 1 -- 172.21.15.45:0/2324874277 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b80043d0 con 0x7fc9c8100d10 2024-01-31T05:20:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9ce46e640 1 -- 172.21.15.45:0/2324874277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b8009560 con 0x7fc9c8100d10 2024-01-31T05:20:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c57fa640 1 -- 172.21.15.45:0/2324874277 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9b80054c0 con 0x7fc9c8100d10 2024-01-31T05:20:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9c57fa640 1 -- 172.21.15.45:0/2324874277 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8003e50 con 0x7fc9c8100d10 2024-01-31T05:20:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.277+0000 7fc9ce46e640 1 -- 172.21.15.45:0/2324874277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c819f180 con 0x7fc9c8100d10 2024-01-31T05:20:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.278+0000 7fc9ce46e640 1 -- 172.21.15.45:0/2324874277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c8067f30 con 0x7fc9c8100d10 2024-01-31T05:20:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.279+0000 7fc9c57fa640 1 -- 172.21.15.45:0/2324874277 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fc9b80035f0 con 0x7fc9c8100d10 2024-01-31T05:20:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.280+0000 7fc9c57fa640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc9b00a12e0 0x7fc9b00a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.280+0000 7fc9c77fe640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc9b00a12e0 0x7fc9b00a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.280+0000 7fc9c57fa640 1 -- 172.21.15.45:0/2324874277 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc9b80ef3d0 con 0x7fc9c8100d10 2024-01-31T05:20:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.280+0000 7fc9c77fe640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc9b00a12e0 0x7fc9b00a37a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009d60 tx=0x7fc9bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.283+0000 7fc9c57fa640 1 -- 172.21.15.45:0/2324874277 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fc9b80b8c60 con 0x7fc9c8100d10 2024-01-31T05:20:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.503+0000 7fc9ce46e640 1 -- 172.21.15.45:0/2324874277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9c810c750 con 0x7fc9c8100d10 2024-01-31T05:20:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.503+0000 7fc9c57fa640 1 -- 172.21.15.45:0/2324874277 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3698 (secure 0 0 0) 0x7fc9b8043e90 con 0x7fc9c8100d10 2024-01-31T05:20:15.503 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:15.503 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-31T05:20:15.503 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 5 2024-01-31T05:20:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.505+0000 7fc9ce46e640 1 -- 172.21.15.45:0/2324874277 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc9b00a12e0 msgr2=0x7fc9b00a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.505+0000 7fc9ce46e640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc9b00a12e0 0x7fc9b00a37a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009d60 tx=0x7fc9bc005f90 comp rx=0 tx=0).stop 2024-01-31T05:20:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.505+0000 7fc9ce46e640 1 -- 172.21.15.45:0/2324874277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9c8100d10 msgr2=0x7fc9c8196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.505+0000 7fc9ce46e640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9c8100d10 0x7fc9c8196d20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009230 tx=0x7fc9b8004ed0 comp rx=0 tx=0).stop 2024-01-31T05:20:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.505+0000 7fc9ce46e640 1 -- 172.21.15.45:0/2324874277 shutdown_connections 2024-01-31T05:20:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.505+0000 7fc9ce46e640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc9b00a12e0 0x7fc9b00a37a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.505+0000 7fc9ce46e640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9c81028c0 0x7fc9c819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.505+0000 7fc9ce46e640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9c8101f00 0x7fc9c8197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.505+0000 7fc9ce46e640 1 --2- 172.21.15.45:0/2324874277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9c8100d10 0x7fc9c8196d20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.505+0000 7fc9ce46e640 1 -- 172.21.15.45:0/2324874277 >> 172.21.15.45:0/2324874277 conn(0x7fc9c80fc2a0 msgr2=0x7fc9c8105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.505+0000 7fc9ce46e640 1 -- 172.21.15.45:0/2324874277 shutdown_connections 2024-01-31T05:20:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:15.505+0000 7fc9ce46e640 1 -- 172.21.15.45:0/2324874277 wait complete. 2024-01-31T05:20:15.649 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:15 smithi156 ceph-mon[35720]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:15.780 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:15 smithi045 ceph-mon[31585]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:15.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:15 smithi186 ceph-mon[35659]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:16.226 INFO:journalctl@ceph.mds.e.smithi156.stdout:Jan 31 05:20:15 smithi156 systemd[1]: Started Ceph mds.e for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:20:16.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:16 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2324874277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:16.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:16 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:16.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:16 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:16.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:16 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:16.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:16 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:16.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:16 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:16.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:16 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:16.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:16 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2324874277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:16.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:16 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:16.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:16 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:16.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:16 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:16.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:16 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:16.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:16 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:16.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:16 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:16.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:16 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2324874277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:16.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:16 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:16.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:16 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:16.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:16 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:16.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:16 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:16.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:16 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:16.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:16 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:17.516 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:17.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.633+0000 7f3d7e089640 1 -- 172.21.15.45:0/158406993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d780ffc10 msgr2=0x7f3d78100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.633+0000 7f3d7e089640 1 --2- 172.21.15.45:0/158406993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d780ffc10 0x7f3d78100030 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009900 tx=0x7f3d6802bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.633+0000 7f3d7e089640 1 -- 172.21.15.45:0/158406993 shutdown_connections 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.633+0000 7f3d7e089640 1 --2- 172.21.15.45:0/158406993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d781017c0 0x7f3d78108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.633+0000 7f3d7e089640 1 --2- 172.21.15.45:0/158406993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d78100e00 0x7f3d78101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.633+0000 7f3d7e089640 1 --2- 172.21.15.45:0/158406993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d780ffc10 0x7f3d78100030 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.633+0000 7f3d7e089640 1 -- 172.21.15.45:0/158406993 >> 172.21.15.45:0/158406993 conn(0x7f3d780fb180 msgr2=0x7f3d780fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.633+0000 7f3d7e089640 1 -- 172.21.15.45:0/158406993 shutdown_connections 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.633+0000 7f3d7e089640 1 -- 172.21.15.45:0/158406993 wait complete. 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.633+0000 7f3d7e089640 1 Processor -- start 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d7e089640 1 -- start start 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d7e089640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d780ffc10 0x7f3d78196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d7e089640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d78100e00 0x7f3d781972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d7e089640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d781017c0 0x7f3d7819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d7e089640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7819e880 con 0x7f3d78100e00 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d7e089640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7819e9f0 con 0x7f3d781017c0 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d7e089640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7819ebb0 con 0x7f3d780ffc10 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d76ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d78100e00 0x7f3d781972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d76ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d78100e00 0x7f3d781972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33042/0 (socket says 172.21.15.45:33042) 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d76ffd640 1 -- 172.21.15.45:0/918388893 learned_addr learned my addr 172.21.15.45:0/918388893 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d777fe640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d780ffc10 0x7f3d78196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d77fff640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d781017c0 0x7f3d7819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d76ffd640 1 -- 172.21.15.45:0/918388893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d780ffc10 msgr2=0x7f3d78196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d76ffd640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d780ffc10 0x7f3d78196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d76ffd640 1 -- 172.21.15.45:0/918388893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d781017c0 msgr2=0x7f3d7819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d76ffd640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d781017c0 0x7f3d7819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d76ffd640 1 -- 172.21.15.45:0/918388893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d68009560 con 0x7f3d78100e00 2024-01-31T05:20:17.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d76ffd640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d78100e00 0x7f3d781972e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00d950 tx=0x7f3d6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:17.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d74ff9640 1 -- 172.21.15.45:0/918388893 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c009c30 con 0x7f3d78100e00 2024-01-31T05:20:17.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d7e089640 1 -- 172.21.15.45:0/918388893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7819ee90 con 0x7f3d78100e00 2024-01-31T05:20:17.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d74ff9640 1 -- 172.21.15.45:0/918388893 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d6c009d90 con 0x7f3d78100e00 2024-01-31T05:20:17.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.634+0000 7f3d74ff9640 1 -- 172.21.15.45:0/918388893 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c00f740 con 0x7f3d78100e00 2024-01-31T05:20:17.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.635+0000 7f3d7e089640 1 -- 172.21.15.45:0/918388893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7819f310 con 0x7f3d78100e00 2024-01-31T05:20:17.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.635+0000 7f3d7e089640 1 -- 172.21.15.45:0/918388893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7810b4a0 con 0x7f3d78100e00 2024-01-31T05:20:17.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.637+0000 7f3d74ff9640 1 -- 172.21.15.45:0/918388893 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f3d6c010a20 con 0x7f3d78100e00 2024-01-31T05:20:17.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.637+0000 7f3d74ff9640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3d600a12e0 0x7f3d600a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:17.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.637+0000 7f3d777fe640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3d600a12e0 0x7f3d600a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:17.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.637+0000 7f3d74ff9640 1 -- 172.21.15.45:0/918388893 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3d6c0cc3e0 con 0x7f3d78100e00 2024-01-31T05:20:17.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.638+0000 7f3d777fe640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3d600a12e0 0x7f3d600a37a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3d78198270 tx=0x7f3d6802c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:17.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.641+0000 7f3d74ff9640 1 -- 172.21.15.45:0/918388893 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f3d6c095c70 con 0x7f3d78100e00 2024-01-31T05:20:17.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:17 smithi186 ceph-mon[35659]: Deploying daemon mds.f on smithi186 2024-01-31T05:20:17.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:17 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.156:6834/4045803513,v1:172.21.15.156:6835/4045803513] up:boot 2024-01-31T05:20:17.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:17 smithi186 ceph-mon[35659]: fsmap 5 up:standby 2024-01-31T05:20:17.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:17 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-31T05:20:17.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:17 smithi186 ceph-mon[35659]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:17.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:17 smithi156 ceph-mon[35720]: Deploying daemon mds.f on smithi186 2024-01-31T05:20:17.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:17 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.156:6834/4045803513,v1:172.21.15.156:6835/4045803513] up:boot 2024-01-31T05:20:17.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:17 smithi156 ceph-mon[35720]: fsmap 5 up:standby 2024-01-31T05:20:17.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:17 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-31T05:20:17.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:17 smithi156 ceph-mon[35720]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:17.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.857+0000 7f3d7e089640 1 -- 172.21.15.45:0/918388893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d7810b630 con 0x7f3d78100e00 2024-01-31T05:20:17.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.857+0000 7f3d74ff9640 1 -- 172.21.15.45:0/918388893 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7f3d6c097ba0 con 0x7f3d78100e00 2024-01-31T05:20:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:17.857 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-01-31T05:20:17.857 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 6 2024-01-31T05:20:17.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.859+0000 7f3d7e089640 1 -- 172.21.15.45:0/918388893 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3d600a12e0 msgr2=0x7f3d600a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:17.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.859+0000 7f3d7e089640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3d600a12e0 0x7f3d600a37a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3d78198270 tx=0x7f3d6802c5d0 comp rx=0 tx=0).stop 2024-01-31T05:20:17.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.859+0000 7f3d7e089640 1 -- 172.21.15.45:0/918388893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d78100e00 msgr2=0x7f3d781972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:17.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.859+0000 7f3d7e089640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d78100e00 0x7f3d781972e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00d950 tx=0x7f3d6c00de10 comp rx=0 tx=0).stop 2024-01-31T05:20:17.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.859+0000 7f3d7e089640 1 -- 172.21.15.45:0/918388893 shutdown_connections 2024-01-31T05:20:17.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.859+0000 7f3d7e089640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3d600a12e0 0x7f3d600a37a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:17.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.859+0000 7f3d7e089640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d78100e00 0x7f3d781972e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:17.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.859+0000 7f3d7e089640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d781017c0 0x7f3d7819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:17.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.859+0000 7f3d7e089640 1 --2- 172.21.15.45:0/918388893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d780ffc10 0x7f3d78196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:17.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.859+0000 7f3d7e089640 1 -- 172.21.15.45:0/918388893 >> 172.21.15.45:0/918388893 conn(0x7f3d780fb180 msgr2=0x7f3d78067ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:17.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.859+0000 7f3d7e089640 1 -- 172.21.15.45:0/918388893 shutdown_connections 2024-01-31T05:20:17.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:17.859+0000 7f3d7e089640 1 -- 172.21.15.45:0/918388893 wait complete. 2024-01-31T05:20:17.868 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:17 smithi045 ceph-mon[31585]: Deploying daemon mds.f on smithi186 2024-01-31T05:20:17.868 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:17 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.156:6834/4045803513,v1:172.21.15.156:6835/4045803513] up:boot 2024-01-31T05:20:17.868 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:17 smithi045 ceph-mon[31585]: fsmap 5 up:standby 2024-01-31T05:20:17.868 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:17 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-31T05:20:17.869 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:17 smithi045 ceph-mon[31585]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:18.466 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:20:18 smithi186 podman[66343]: 2024-01-31 05:20:18.371491726 +0000 UTC m=+0.135272991 container create c7d29b680cd7134ccf239ed653a97535ef164642e1096d6765fe089ef3623c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2) 2024-01-31T05:20:18.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:18 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/918388893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:18.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:18 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/918388893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:18.842 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:20:18 smithi186 podman[66343]: 2024-01-31 05:20:18.598671514 +0000 UTC m=+0.362452795 container init c7d29b680cd7134ccf239ed653a97535ef164642e1096d6765fe089ef3623c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-01-31T05:20:18.842 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:20:18 smithi186 podman[66343]: 2024-01-31 05:20:18.601357276 +0000 UTC m=+0.365138548 container start c7d29b680cd7134ccf239ed653a97535ef164642e1096d6765fe089ef3623c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD) 2024-01-31T05:20:18.842 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:20:18 smithi186 ceph-mds[66362]: -- 172.21.15.186:0/1275701802 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c12f861520 con 0x55c12f836800 2024-01-31T05:20:18.842 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:20:18 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f[66358]: starting mds.f at 2024-01-31T05:20:18.842 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:20:18 smithi186 bash[66343]: c7d29b680cd7134ccf239ed653a97535ef164642e1096d6765fe089ef3623c1e 2024-01-31T05:20:18.842 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:20:18 smithi186 systemd[1]: Started Ceph mds.f for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:20:18.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:18 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/918388893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:19.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:19 smithi186 ceph-mon[35659]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:19.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:19 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:19.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:19 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:19.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:19 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:19.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:19 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:19.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:19 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:19.726 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:19 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:19.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:19 smithi156 ceph-mon[35720]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:19.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:19 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:19.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:19 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:19.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:19 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:19.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:19 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:19.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:19 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:19.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:19 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:19.870 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:19.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:19 smithi045 ceph-mon[31585]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:19.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:19.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:19.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:19.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:19.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:19.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:19 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.986+0000 7f434cbab640 1 -- 172.21.15.45:0/2268560475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4348106a70 msgr2=0x7f4348108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.986+0000 7f434cbab640 1 --2- 172.21.15.45:0/2268560475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4348106a70 0x7f4348108ec0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4340009ca0 tx=0x7f434002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.986+0000 7f434cbab640 1 -- 172.21.15.45:0/2268560475 shutdown_connections 2024-01-31T05:20:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.986+0000 7f434cbab640 1 --2- 172.21.15.45:0/2268560475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4348106a70 0x7f4348108ec0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.986+0000 7f434cbab640 1 --2- 172.21.15.45:0/2268560475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4348104140 0x7f4348106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.986+0000 7f434cbab640 1 --2- 172.21.15.45:0/2268560475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4348101810 0x7f4348103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.986+0000 7f434cbab640 1 -- 172.21.15.45:0/2268560475 >> 172.21.15.45:0/2268560475 conn(0x7f43480fb220 msgr2=0x7f43480fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.986+0000 7f434cbab640 1 -- 172.21.15.45:0/2268560475 shutdown_connections 2024-01-31T05:20:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.986+0000 7f434cbab640 1 -- 172.21.15.45:0/2268560475 wait complete. 2024-01-31T05:20:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.987+0000 7f434cbab640 1 Processor -- start 2024-01-31T05:20:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.987+0000 7f434cbab640 1 -- start start 2024-01-31T05:20:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.987+0000 7f434cbab640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4348101810 0x7f4348101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.987+0000 7f434cbab640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4348104140 0x7f43480ff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.987+0000 7f434cbab640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4348106a70 0x7f43480ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.987+0000 7f434cbab640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4348100460 con 0x7f4348101810 2024-01-31T05:20:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.987+0000 7f434cbab640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43481005d0 con 0x7f4348104140 2024-01-31T05:20:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.987+0000 7f434cbab640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43481a0c70 con 0x7f4348106a70 2024-01-31T05:20:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.987+0000 7f434659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4348101810 0x7f4348101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f434659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4348101810 0x7f4348101180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33068/0 (socket says 172.21.15.45:33068) 2024-01-31T05:20:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f434659c640 1 -- 172.21.15.45:0/209724743 learned_addr learned my addr 172.21.15.45:0/209724743 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f4346d9d640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4348106a70 0x7f43480ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f4345d9b640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4348104140 0x7f43480ff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f434659c640 1 -- 172.21.15.45:0/209724743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4348106a70 msgr2=0x7f43480ffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f434659c640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4348106a70 0x7f43480ffd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f434659c640 1 -- 172.21.15.45:0/209724743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4348104140 msgr2=0x7f43480ff7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f434659c640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4348104140 0x7f43480ff7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f434659c640 1 -- 172.21.15.45:0/209724743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4340009900 con 0x7f4348101810 2024-01-31T05:20:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f434659c640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4348101810 0x7f4348101180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f433800d980 tx=0x7f433800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f43377fe640 1 -- 172.21.15.45:0/209724743 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338009c30 con 0x7f4348101810 2024-01-31T05:20:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f434cbab640 1 -- 172.21.15.45:0/209724743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43481a0f20 con 0x7f4348101810 2024-01-31T05:20:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f43377fe640 1 -- 172.21.15.45:0/209724743 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f433800b660 con 0x7f4348101810 2024-01-31T05:20:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f43377fe640 1 -- 172.21.15.45:0/209724743 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800f620 con 0x7f4348101810 2024-01-31T05:20:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.988+0000 7f434cbab640 1 -- 172.21.15.45:0/209724743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43481a1340 con 0x7f4348101810 2024-01-31T05:20:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.989+0000 7f434cbab640 1 -- 172.21.15.45:0/209724743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43480fcd10 con 0x7f4348101810 2024-01-31T05:20:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.990+0000 7f43377fe640 1 -- 172.21.15.45:0/209724743 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f4338010460 con 0x7f4348101810 2024-01-31T05:20:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.991+0000 7f43377fe640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f43300a12e0 0x7f43300a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.991+0000 7f4345d9b640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f43300a12e0 0x7f43300a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.991+0000 7f43377fe640 1 -- 172.21.15.45:0/209724743 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f43380cc260 con 0x7f4348101810 2024-01-31T05:20:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.991+0000 7f4345d9b640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f43300a12e0 0x7f43300a37a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f433c005e10 tx=0x7f433c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:19.994+0000 7f43377fe640 1 -- 172.21.15.45:0/209724743 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f4338095b80 con 0x7f4348101810 2024-01-31T05:20:20.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.212+0000 7f434cbab640 1 -- 172.21.15.45:0/209724743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f434810c870 con 0x7f4348101810 2024-01-31T05:20:20.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.212+0000 7f43377fe640 1 -- 172.21.15.45:0/209724743 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5283 (secure 0 0 0) 0x7f43380991a0 con 0x7f4348101810 2024-01-31T05:20:20.211 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:20.212 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-01-31T05:20:20.212 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 7 2024-01-31T05:20:20.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.214+0000 7f434cbab640 1 -- 172.21.15.45:0/209724743 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f43300a12e0 msgr2=0x7f43300a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:20.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.214+0000 7f434cbab640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f43300a12e0 0x7f43300a37a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f433c005e10 tx=0x7f433c005da0 comp rx=0 tx=0).stop 2024-01-31T05:20:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.214+0000 7f434cbab640 1 -- 172.21.15.45:0/209724743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4348101810 msgr2=0x7f4348101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.214+0000 7f434cbab640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4348101810 0x7f4348101180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f433800d980 tx=0x7f433800de40 comp rx=0 tx=0).stop 2024-01-31T05:20:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.214+0000 7f434cbab640 1 -- 172.21.15.45:0/209724743 shutdown_connections 2024-01-31T05:20:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.214+0000 7f434cbab640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f43300a12e0 0x7f43300a37a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.214+0000 7f434cbab640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4348106a70 0x7f43480ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.214+0000 7f434cbab640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4348104140 0x7f43480ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.214+0000 7f434cbab640 1 --2- 172.21.15.45:0/209724743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4348101810 0x7f4348101180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.214+0000 7f434cbab640 1 -- 172.21.15.45:0/209724743 >> 172.21.15.45:0/209724743 conn(0x7f43480fb220 msgr2=0x7f4348102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.214+0000 7f434cbab640 1 -- 172.21.15.45:0/209724743 shutdown_connections 2024-01-31T05:20:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:20.214+0000 7f434cbab640 1 -- 172.21.15.45:0/209724743 wait complete. 2024-01-31T05:20:20.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:20 smithi186 ceph-mon[35659]: Deploying daemon mds.c on smithi186 2024-01-31T05:20:20.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:20 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.186:6834/696958495,v1:172.21.15.186:6835/696958495] up:boot 2024-01-31T05:20:20.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:20 smithi186 ceph-mon[35659]: fsmap 6 up:standby 2024-01-31T05:20:20.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:20 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-31T05:20:20.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:20 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/209724743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:20.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:20 smithi045 ceph-mon[31585]: Deploying daemon mds.c on smithi186 2024-01-31T05:20:20.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:20 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.186:6834/696958495,v1:172.21.15.186:6835/696958495] up:boot 2024-01-31T05:20:20.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:20 smithi045 ceph-mon[31585]: fsmap 6 up:standby 2024-01-31T05:20:20.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:20 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-31T05:20:20.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:20 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/209724743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:20.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:20 smithi156 ceph-mon[35720]: Deploying daemon mds.c on smithi186 2024-01-31T05:20:20.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:20 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.186:6834/696958495,v1:172.21.15.186:6835/696958495] up:boot 2024-01-31T05:20:20.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:20 smithi156 ceph-mon[35720]: fsmap 6 up:standby 2024-01-31T05:20:20.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:20 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-31T05:20:20.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:20 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/209724743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:20.841 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:20:20 smithi186 systemd[1]: Starting Ceph mds.c for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:20:21.209 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:20:20 smithi186 podman[66658]: 2024-01-31 05:20:20.839784045 +0000 UTC m=+0.017523169 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:20:21.209 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:20:20 smithi186 podman[66658]: 2024-01-31 05:20:20.948927174 +0000 UTC m=+0.126666330 container create 3505dd9563489f0a7f8f57be0f49288b401c8d2de16f1c142924ac179ecb1f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-c, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-31T05:20:21.209 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:20:21 smithi186 podman[66658]: 2024-01-31 05:20:21.184662603 +0000 UTC m=+0.362401724 container init 3505dd9563489f0a7f8f57be0f49288b401c8d2de16f1c142924ac179ecb1f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-c, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-01-31T05:20:21.209 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:20:21 smithi186 podman[66658]: 2024-01-31 05:20:21.187272165 +0000 UTC m=+0.365011282 container start 3505dd9563489f0a7f8f57be0f49288b401c8d2de16f1c142924ac179ecb1f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-31T05:20:21.477 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:20:21 smithi186 ceph-mds[66677]: -- 172.21.15.186:0/2634707245 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557159289520 con 0x55715925ec00 2024-01-31T05:20:21.477 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:20:21 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-c[66673]: starting mds.c at 2024-01-31T05:20:21.478 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:20:21 smithi186 bash[66658]: 3505dd9563489f0a7f8f57be0f49288b401c8d2de16f1c142924ac179ecb1f42 2024-01-31T05:20:21.478 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:20:21 smithi186 systemd[1]: Started Ceph mds.c for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:20:21.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:21 smithi186 ceph-mon[35659]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:21.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:21.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:21.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:21.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:21.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:21.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:21 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:21.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:21 smithi045 ceph-mon[31585]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:21.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:21.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:21.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:21.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:21.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:21.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:21 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:21.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:21 smithi156 ceph-mon[35720]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:21.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:21.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:21.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:21.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:21.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:21.959 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:21 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:22.225 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:22.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.341+0000 7f2a9561d640 1 -- 172.21.15.45:0/2796003994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a901069f0 msgr2=0x7f2a90108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:22.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.341+0000 7f2a9561d640 1 --2- 172.21.15.45:0/2796003994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a901069f0 0x7f2a90108e40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009ca0 tx=0x7f2a8002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:22.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.341+0000 7f2a9561d640 1 -- 172.21.15.45:0/2796003994 shutdown_connections 2024-01-31T05:20:22.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.341+0000 7f2a9561d640 1 --2- 172.21.15.45:0/2796003994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a901069f0 0x7f2a90108e40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:22.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.341+0000 7f2a9561d640 1 --2- 172.21.15.45:0/2796003994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a901040c0 0x7f2a901064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:22.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.341+0000 7f2a9561d640 1 --2- 172.21.15.45:0/2796003994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a90101790 0x7f2a90103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:22.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.341+0000 7f2a9561d640 1 -- 172.21.15.45:0/2796003994 >> 172.21.15.45:0/2796003994 conn(0x7f2a900fb180 msgr2=0x7f2a900fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:22.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.341+0000 7f2a9561d640 1 -- 172.21.15.45:0/2796003994 shutdown_connections 2024-01-31T05:20:22.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.341+0000 7f2a9561d640 1 -- 172.21.15.45:0/2796003994 wait complete. 2024-01-31T05:20:22.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.341+0000 7f2a9561d640 1 Processor -- start 2024-01-31T05:20:22.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a9561d640 1 -- start start 2024-01-31T05:20:22.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a9561d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a90101790 0x7f2a90101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a9561d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a901040c0 0x7f2a900ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a9561d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a901069f0 0x7f2a900ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a9561d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a901003e0 con 0x7f2a901040c0 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a9561d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90100550 con 0x7f2a901069f0 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a9561d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a901a0ce0 con 0x7f2a90101790 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a8e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a901040c0 0x7f2a900ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a8e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a901040c0 0x7f2a900ff740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33074/0 (socket says 172.21.15.45:33074) 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a8e7fc640 1 -- 172.21.15.45:0/1026827434 learned_addr learned my addr 172.21.15.45:0/1026827434 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a8effd640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a90101790 0x7f2a90101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a8f7fe640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a901069f0 0x7f2a900ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a8e7fc640 1 -- 172.21.15.45:0/1026827434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a90101790 msgr2=0x7f2a90101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a8e7fc640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a90101790 0x7f2a90101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:22.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a8e7fc640 1 -- 172.21.15.45:0/1026827434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a901069f0 msgr2=0x7f2a900ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:22.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a8e7fc640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a901069f0 0x7f2a900ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:22.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a8e7fc640 1 -- 172.21.15.45:0/1026827434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a80009900 con 0x7f2a901040c0 2024-01-31T05:20:22.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a8e7fc640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a901040c0 0x7f2a900ff740 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800d7a0 tx=0x7f2a7800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:22.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.342+0000 7f2a87fff640 1 -- 172.21.15.45:0/1026827434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78009c30 con 0x7f2a901040c0 2024-01-31T05:20:22.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.343+0000 7f2a87fff640 1 -- 172.21.15.45:0/1026827434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a7800deb0 con 0x7f2a901040c0 2024-01-31T05:20:22.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.343+0000 7f2a9561d640 1 -- 172.21.15.45:0/1026827434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a901a0f90 con 0x7f2a901040c0 2024-01-31T05:20:22.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.343+0000 7f2a87fff640 1 -- 172.21.15.45:0/1026827434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800f620 con 0x7f2a901040c0 2024-01-31T05:20:22.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.343+0000 7f2a9561d640 1 -- 172.21.15.45:0/1026827434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a901a1410 con 0x7f2a901040c0 2024-01-31T05:20:22.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.344+0000 7f2a9561d640 1 -- 172.21.15.45:0/1026827434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a900fcc70 con 0x7f2a901040c0 2024-01-31T05:20:22.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.345+0000 7f2a87fff640 1 -- 172.21.15.45:0/1026827434 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f2a78010460 con 0x7f2a901040c0 2024-01-31T05:20:22.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.345+0000 7f2a87fff640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2a7c0a12e0 0x7f2a7c0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:22.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.345+0000 7f2a87fff640 1 -- 172.21.15.45:0/1026827434 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2a780cc290 con 0x7f2a901040c0 2024-01-31T05:20:22.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.345+0000 7f2a8effd640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2a7c0a12e0 0x7f2a7c0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:22.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.346+0000 7f2a8effd640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2a7c0a12e0 0x7f2a7c0a37a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a88005d00 tx=0x7f2a8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:22.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.350+0000 7f2a87fff640 1 -- 172.21.15.45:0/1026827434 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f2a78095c60 con 0x7f2a901040c0 2024-01-31T05:20:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.570+0000 7f2a9561d640 1 -- 172.21.15.45:0/1026827434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a9010c7d0 con 0x7f2a901040c0 2024-01-31T05:20:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.570+0000 7f2a87fff640 1 -- 172.21.15.45:0/1026827434 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6077 (secure 0 0 0) 0x7f2a78099280 con 0x7f2a901040c0 2024-01-31T05:20:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:22.570 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-01-31T05:20:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-01-31T05:20:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.572+0000 7f2a9561d640 1 -- 172.21.15.45:0/1026827434 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2a7c0a12e0 msgr2=0x7f2a7c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.572+0000 7f2a9561d640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2a7c0a12e0 0x7f2a7c0a37a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a88005d00 tx=0x7f2a8800b410 comp rx=0 tx=0).stop 2024-01-31T05:20:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.572+0000 7f2a9561d640 1 -- 172.21.15.45:0/1026827434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a901040c0 msgr2=0x7f2a900ff740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.572+0000 7f2a9561d640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a901040c0 0x7f2a900ff740 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800d7a0 tx=0x7f2a7800dc60 comp rx=0 tx=0).stop 2024-01-31T05:20:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.572+0000 7f2a9561d640 1 -- 172.21.15.45:0/1026827434 shutdown_connections 2024-01-31T05:20:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.572+0000 7f2a9561d640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2a7c0a12e0 0x7f2a7c0a37a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.572+0000 7f2a9561d640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a901040c0 0x7f2a900ff740 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.572+0000 7f2a9561d640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a901069f0 0x7f2a900ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.572+0000 7f2a9561d640 1 --2- 172.21.15.45:0/1026827434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a90101790 0x7f2a90101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.572+0000 7f2a9561d640 1 -- 172.21.15.45:0/1026827434 >> 172.21.15.45:0/1026827434 conn(0x7f2a900fb180 msgr2=0x7f2a901020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.573+0000 7f2a9561d640 1 -- 172.21.15.45:0/1026827434 shutdown_connections 2024-01-31T05:20:22.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:22.573+0000 7f2a9561d640 1 -- 172.21.15.45:0/1026827434 wait complete. 2024-01-31T05:20:22.581 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:22 smithi045 ceph-mon[31585]: Deploying daemon mds.h on smithi156 2024-01-31T05:20:22.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:22 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] up:boot 2024-01-31T05:20:22.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:22 smithi045 ceph-mon[31585]: fsmap 7 up:standby 2024-01-31T05:20:22.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:22 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-31T05:20:22.600 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:22 smithi156 ceph-mon[35720]: Deploying daemon mds.h on smithi156 2024-01-31T05:20:22.600 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:22 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] up:boot 2024-01-31T05:20:22.600 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:22 smithi156 ceph-mon[35720]: fsmap 7 up:standby 2024-01-31T05:20:22.600 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:22 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-31T05:20:22.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:22 smithi186 ceph-mon[35659]: Deploying daemon mds.h on smithi156 2024-01-31T05:20:22.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:22 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] up:boot 2024-01-31T05:20:22.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:22 smithi186 ceph-mon[35659]: fsmap 7 up:standby 2024-01-31T05:20:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:22 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-31T05:20:23.674 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:23 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1026827434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:23.674 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:23 smithi156 ceph-mon[35720]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:23.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:23 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1026827434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:23.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:23 smithi186 ceph-mon[35659]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:23.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:23 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1026827434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:23.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:23 smithi045 ceph-mon[31585]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:24.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:20:24 smithi156 systemd[1]: Started Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:20:24.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.699+0000 7fdecf278640 1 -- 172.21.15.45:0/2477711280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec8100e80 msgr2=0x7fdec8101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.699+0000 7fdecf278640 1 --2- 172.21.15.45:0/2477711280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec8100e80 0x7fdec8101300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009900 tx=0x7fdebc02bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.699+0000 7fdecf278640 1 -- 172.21.15.45:0/2477711280 shutdown_connections 2024-01-31T05:20:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.699+0000 7fdecf278640 1 --2- 172.21.15.45:0/2477711280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec8100e80 0x7fdec8101300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.699+0000 7fdecf278640 1 --2- 172.21.15.45:0/2477711280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdec8101840 0x7fdec81080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.699+0000 7fdecf278640 1 --2- 172.21.15.45:0/2477711280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdec80ffc90 0x7fdec81000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.699+0000 7fdecf278640 1 -- 172.21.15.45:0/2477711280 >> 172.21.15.45:0/2477711280 conn(0x7fdec80fb200 msgr2=0x7fdec80fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.699+0000 7fdecf278640 1 -- 172.21.15.45:0/2477711280 shutdown_connections 2024-01-31T05:20:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.699+0000 7fdecf278640 1 -- 172.21.15.45:0/2477711280 wait complete. 2024-01-31T05:20:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecf278640 1 Processor -- start 2024-01-31T05:20:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecf278640 1 -- start start 2024-01-31T05:20:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecf278640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdec80ffc90 0x7fdec8196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecf278640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec8100e80 0x7fdec8197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecf278640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdec8101840 0x7fdec819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecf278640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec819e7d0 con 0x7fdec8100e80 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecf278640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec819e940 con 0x7fdec8101840 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecf278640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec819eb00 con 0x7fdec80ffc90 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecc813640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec8100e80 0x7fdec8197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecc813640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec8100e80 0x7fdec8197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33084/0 (socket says 172.21.15.45:33084) 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecc813640 1 -- 172.21.15.45:0/1115526589 learned_addr learned my addr 172.21.15.45:0/1115526589 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecd014640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdec80ffc90 0x7fdec8196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecd815640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdec8101840 0x7fdec819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecc813640 1 -- 172.21.15.45:0/1115526589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdec80ffc90 msgr2=0x7fdec8196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecc813640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdec80ffc90 0x7fdec8196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecc813640 1 -- 172.21.15.45:0/1115526589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdec8101840 msgr2=0x7fdec819e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecc813640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdec8101840 0x7fdec819e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecc813640 1 -- 172.21.15.45:0/1115526589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc009560 con 0x7fdec8100e80 2024-01-31T05:20:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.700+0000 7fdecd815640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdec8101840 0x7fdec819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:20:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.701+0000 7fdecc813640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec8100e80 0x7fdec8197230 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009230 tx=0x7fdebc004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.701+0000 7fdec67fc640 1 -- 172.21.15.45:0/1115526589 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc004350 con 0x7fdec8100e80 2024-01-31T05:20:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.701+0000 7fdec67fc640 1 -- 172.21.15.45:0/1115526589 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdebc0044b0 con 0x7fdec8100e80 2024-01-31T05:20:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.701+0000 7fdecf278640 1 -- 172.21.15.45:0/1115526589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec819ed80 con 0x7fdec8100e80 2024-01-31T05:20:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.701+0000 7fdec67fc640 1 -- 172.21.15.45:0/1115526589 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc003e10 con 0x7fdec8100e80 2024-01-31T05:20:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.701+0000 7fdecf278640 1 -- 172.21.15.45:0/1115526589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec819f1d0 con 0x7fdec8100e80 2024-01-31T05:20:24.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.702+0000 7fdecf278640 1 -- 172.21.15.45:0/1115526589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec810b4e0 con 0x7fdec8100e80 2024-01-31T05:20:24.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.703+0000 7fdec67fc640 1 -- 172.21.15.45:0/1115526589 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fdebc005000 con 0x7fdec8100e80 2024-01-31T05:20:24.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.704+0000 7fdec67fc640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdeb00a12e0 0x7fdeb00a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:24.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.704+0000 7fdecd014640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdeb00a12e0 0x7fdeb00a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:24.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.704+0000 7fdec67fc640 1 -- 172.21.15.45:0/1115526589 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdebc0ef5c0 con 0x7fdec8100e80 2024-01-31T05:20:24.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.704+0000 7fdecd014640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdeb00a12e0 0x7fdeb00a37a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdec81981c0 tx=0x7fdeb8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:24.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.707+0000 7fdec67fc640 1 -- 172.21.15.45:0/1115526589 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fdebc0b8db0 con 0x7fdec8100e80 2024-01-31T05:20:24.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.925+0000 7fdecf278640 1 -- 172.21.15.45:0/1115526589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdec819f480 con 0x7fdec8100e80 2024-01-31T05:20:24.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.925+0000 7fdec67fc640 1 -- 172.21.15.45:0/1115526589 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7fdebc0dae60 con 0x7fdec8100e80 2024-01-31T05:20:24.925 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:24.925 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3782584640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3782584640},{"type":"v1","addr":"172.21.15.156:6837","nonce":3782584640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-01-31T05:20:24.925 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2024-01-31T05:20:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.927+0000 7fdecf278640 1 -- 172.21.15.45:0/1115526589 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdeb00a12e0 msgr2=0x7fdeb00a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.927+0000 7fdecf278640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdeb00a12e0 0x7fdeb00a37a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdec81981c0 tx=0x7fdeb8009500 comp rx=0 tx=0).stop 2024-01-31T05:20:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.927+0000 7fdecf278640 1 -- 172.21.15.45:0/1115526589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec8100e80 msgr2=0x7fdec8197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.927+0000 7fdecf278640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec8100e80 0x7fdec8197230 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009230 tx=0x7fdebc004ec0 comp rx=0 tx=0).stop 2024-01-31T05:20:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.927+0000 7fdecf278640 1 -- 172.21.15.45:0/1115526589 shutdown_connections 2024-01-31T05:20:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.927+0000 7fdecf278640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdeb00a12e0 0x7fdeb00a37a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.927+0000 7fdecf278640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec8100e80 0x7fdec8197230 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:24.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.927+0000 7fdecf278640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdec8101840 0x7fdec819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:24.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.927+0000 7fdecf278640 1 --2- 172.21.15.45:0/1115526589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdec80ffc90 0x7fdec8196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:24.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.927+0000 7fdecf278640 1 -- 172.21.15.45:0/1115526589 >> 172.21.15.45:0/1115526589 conn(0x7fdec80fb200 msgr2=0x7fdec80fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:24.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.928+0000 7fdecf278640 1 -- 172.21.15.45:0/1115526589 shutdown_connections 2024-01-31T05:20:24.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:24.928+0000 7fdecf278640 1 -- 172.21.15.45:0/1115526589 wait complete. 2024-01-31T05:20:25.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:25.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:25.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:25.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:25.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:25.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:25.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:25 smithi186 ceph-mon[35659]: Deploying daemon mds.i on smithi186 2024-01-31T05:20:25.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:25 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.156:6836/3782584640,v1:172.21.15.156:6837/3782584640] up:boot 2024-01-31T05:20:25.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:25 smithi186 ceph-mon[35659]: fsmap 8 up:standby 2024-01-31T05:20:25.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:25 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:20:25.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:25 smithi186 ceph-mon[35659]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:25.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:25 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1115526589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:25.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:25.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:25.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:25.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:25.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:25.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:25.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:25 smithi045 ceph-mon[31585]: Deploying daemon mds.i on smithi186 2024-01-31T05:20:25.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:25 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.156:6836/3782584640,v1:172.21.15.156:6837/3782584640] up:boot 2024-01-31T05:20:25.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:25 smithi045 ceph-mon[31585]: fsmap 8 up:standby 2024-01-31T05:20:25.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:25 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:20:25.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:25 smithi045 ceph-mon[31585]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:25.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:25 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1115526589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:25.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:25.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:25.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:25.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:25.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:25.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:25.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:25 smithi156 ceph-mon[35720]: Deploying daemon mds.i on smithi186 2024-01-31T05:20:25.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:25 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.156:6836/3782584640,v1:172.21.15.156:6837/3782584640] up:boot 2024-01-31T05:20:25.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:25 smithi156 ceph-mon[35720]: fsmap 8 up:standby 2024-01-31T05:20:25.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:25 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:20:25.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:25 smithi156 ceph-mon[35720]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:25.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:25 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1115526589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:26.341 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:20:26 smithi186 systemd[1]: Starting Ceph mds.i for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:20:26.678 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:20:26 smithi186 podman[66965]: 2024-01-31 05:20:26.322480409 +0000 UTC m=+0.017721011 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:20:26.678 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:20:26 smithi186 podman[66965]: 2024-01-31 05:20:26.551270435 +0000 UTC m=+0.246511019 container create d1b57c01fe3a4fcf76412ccba4e0b30ae19741e237dde3b8452697288d188d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-i, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-01-31T05:20:26.938 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:27.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.056+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1850571038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a781017c0 msgr2=0x7f9a78108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:27.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.056+0000 7f9a7f7bb640 1 --2- 172.21.15.45:0/1850571038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a781017c0 0x7f9a78108050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009ca0 tx=0x7f9a7002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:27.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.056+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1850571038 shutdown_connections 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.056+0000 7f9a7f7bb640 1 --2- 172.21.15.45:0/1850571038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a781017c0 0x7f9a78108050 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.056+0000 7f9a7f7bb640 1 --2- 172.21.15.45:0/1850571038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a78100e00 0x7f9a78101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.056+0000 7f9a7f7bb640 1 --2- 172.21.15.45:0/1850571038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a780ffc10 0x7f9a78100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.056+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1850571038 >> 172.21.15.45:0/1850571038 conn(0x7f9a780fb180 msgr2=0x7f9a780fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.056+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1850571038 shutdown_connections 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.056+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1850571038 wait complete. 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7f7bb640 1 Processor -- start 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7f7bb640 1 -- start start 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7f7bb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a780ffc10 0x7f9a78196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7f7bb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a78100e00 0x7f9a781971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7f7bb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a781017c0 0x7f9a7819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7f7bb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7819e7a0 con 0x7f9a780ffc10 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7f7bb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7819e910 con 0x7f9a78100e00 2024-01-31T05:20:27.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7f7bb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7819ead0 con 0x7f9a781017c0 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7d557640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a780ffc10 0x7f9a78196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7d557640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a780ffc10 0x7f9a78196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33104/0 (socket says 172.21.15.45:33104) 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7d557640 1 -- 172.21.15.45:0/1887971311 learned_addr learned my addr 172.21.15.45:0/1887971311 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7cd56640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a78100e00 0x7f9a781971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7dd58640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a781017c0 0x7f9a7819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7d557640 1 -- 172.21.15.45:0/1887971311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a781017c0 msgr2=0x7f9a7819e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7d557640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a781017c0 0x7f9a7819e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7d557640 1 -- 172.21.15.45:0/1887971311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a78100e00 msgr2=0x7f9a781971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7d557640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a78100e00 0x7f9a781971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.057+0000 7f9a7d557640 1 -- 172.21.15.45:0/1887971311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a70009900 con 0x7f9a780ffc10 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.058+0000 7f9a7d557640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a780ffc10 0x7f9a78196c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400e9c0 tx=0x7f9a7400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.058+0000 7f9a6e7fc640 1 -- 172.21.15.45:0/1887971311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a740096e0 con 0x7f9a780ffc10 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.058+0000 7f9a6e7fc640 1 -- 172.21.15.45:0/1887971311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a740044f0 con 0x7f9a780ffc10 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.058+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1887971311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7819edb0 con 0x7f9a780ffc10 2024-01-31T05:20:27.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.058+0000 7f9a6e7fc640 1 -- 172.21.15.45:0/1887971311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a74010430 con 0x7f9a780ffc10 2024-01-31T05:20:27.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.058+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1887971311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7819f1b0 con 0x7f9a780ffc10 2024-01-31T05:20:27.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.059+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1887971311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7810b4d0 con 0x7f9a780ffc10 2024-01-31T05:20:27.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.061+0000 7f9a6e7fc640 1 -- 172.21.15.45:0/1887971311 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f9a740035f0 con 0x7f9a780ffc10 2024-01-31T05:20:27.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.061+0000 7f9a6e7fc640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f9a640a1360 0x7f9a640a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:27.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.062+0000 7f9a7cd56640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f9a640a1360 0x7f9a640a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:27.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.062+0000 7f9a6e7fc640 1 -- 172.21.15.45:0/1887971311 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9a74014070 con 0x7f9a780ffc10 2024-01-31T05:20:27.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.062+0000 7f9a7cd56640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f9a640a1360 0x7f9a640a3820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9a78198190 tx=0x7f9a68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:27.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.065+0000 7f9a6e7fc640 1 -- 172.21.15.45:0/1887971311 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f9a740963a0 con 0x7f9a780ffc10 2024-01-31T05:20:27.091 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:20:26 smithi186 podman[66965]: 2024-01-31 05:20:26.787876406 +0000 UTC m=+0.483117002 container init d1b57c01fe3a4fcf76412ccba4e0b30ae19741e237dde3b8452697288d188d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-i, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-01-31T05:20:27.091 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:20:26 smithi186 podman[66965]: 2024-01-31 05:20:26.790555477 +0000 UTC m=+0.485796067 container start d1b57c01fe3a4fcf76412ccba4e0b30ae19741e237dde3b8452697288d188d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-i, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-01-31T05:20:27.091 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:20:26 smithi186 ceph-mds[66983]: -- 172.21.15.186:0/987999247 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bc6ab03520 con 0x55bc6aad8c00 2024-01-31T05:20:27.091 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:20:26 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-i[66979]: starting mds.i at 2024-01-31T05:20:27.092 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:20:26 smithi186 bash[66965]: d1b57c01fe3a4fcf76412ccba4e0b30ae19741e237dde3b8452697288d188d86 2024-01-31T05:20:27.092 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:20:26 smithi186 systemd[1]: Started Ceph mds.i for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:20:27.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.284+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1887971311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a7810b690 con 0x7f9a780ffc10 2024-01-31T05:20:27.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.284+0000 7f9a6e7fc640 1 -- 172.21.15.45:0/1887971311 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7f9a740999c0 con 0x7f9a780ffc10 2024-01-31T05:20:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:27.284 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3782584640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3782584640},{"type":"v1","addr":"172.21.15.156:6837","nonce":3782584640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-01-31T05:20:27.284 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2024-01-31T05:20:27.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.286+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1887971311 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f9a640a1360 msgr2=0x7f9a640a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:27.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.286+0000 7f9a7f7bb640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f9a640a1360 0x7f9a640a3820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9a78198190 tx=0x7f9a68005f90 comp rx=0 tx=0).stop 2024-01-31T05:20:27.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.286+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1887971311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a780ffc10 msgr2=0x7f9a78196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:27.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.286+0000 7f9a7f7bb640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a780ffc10 0x7f9a78196c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400e9c0 tx=0x7f9a7400ee80 comp rx=0 tx=0).stop 2024-01-31T05:20:27.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.287+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1887971311 shutdown_connections 2024-01-31T05:20:27.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.287+0000 7f9a7f7bb640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f9a640a1360 0x7f9a640a3820 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:27.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.287+0000 7f9a7f7bb640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a781017c0 0x7f9a7819e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.287+0000 7f9a7f7bb640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a78100e00 0x7f9a781971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.287+0000 7f9a7f7bb640 1 --2- 172.21.15.45:0/1887971311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a780ffc10 0x7f9a78196c90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.287+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1887971311 >> 172.21.15.45:0/1887971311 conn(0x7f9a780fb180 msgr2=0x7f9a780fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.287+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1887971311 shutdown_connections 2024-01-31T05:20:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:27.287+0000 7f9a7f7bb640 1 -- 172.21.15.45:0/1887971311 wait complete. 2024-01-31T05:20:27.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:27 smithi045 ceph-mon[31585]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:27.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:27 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:27.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:27 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:27.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:27 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:27.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:27 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:27.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:27 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:27.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:27 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:27.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:27 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1887971311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:27.920 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:27 smithi186 ceph-mon[35659]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:27.920 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:27 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:27.920 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:27 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:27.920 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:27 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:27.920 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:27 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:27.920 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:27 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:27.921 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:27 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:27.921 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:27 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1887971311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:27 smithi156 ceph-mon[35720]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:27 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:27 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:27 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:27 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:27 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:27 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:27.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:27 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1887971311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:28.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:28 smithi045 ceph-mon[31585]: Deploying daemon mds.g on smithi045 2024-01-31T05:20:28.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:28 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] up:boot 2024-01-31T05:20:28.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:28 smithi045 ceph-mon[31585]: fsmap 9 up:standby 2024-01-31T05:20:28.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:28 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-31T05:20:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:28 smithi156 ceph-mon[35720]: Deploying daemon mds.g on smithi045 2024-01-31T05:20:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:28 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] up:boot 2024-01-31T05:20:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:28 smithi156 ceph-mon[35720]: fsmap 9 up:standby 2024-01-31T05:20:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:28 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-31T05:20:29.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:28 smithi186 ceph-mon[35659]: Deploying daemon mds.g on smithi045 2024-01-31T05:20:29.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:28 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] up:boot 2024-01-31T05:20:29.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:28 smithi186 ceph-mon[35659]: fsmap 9 up:standby 2024-01-31T05:20:29.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:28 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-31T05:20:29.298 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:29.314 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:20:29 smithi045 systemd[1]: Starting Ceph mds.g for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:20:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.406+0000 7f94167ee640 1 -- 172.21.15.45:0/852798231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9410100e20 msgr2=0x7f94101012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.406+0000 7f94167ee640 1 --2- 172.21.15.45:0/852798231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9410100e20 0x7f94101012a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9404009900 tx=0x7f940402bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.406+0000 7f94167ee640 1 -- 172.21.15.45:0/852798231 shutdown_connections 2024-01-31T05:20:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.406+0000 7f94167ee640 1 --2- 172.21.15.45:0/852798231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9410100e20 0x7f94101012a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.406+0000 7f94167ee640 1 --2- 172.21.15.45:0/852798231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94101017e0 0x7f9410108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.406+0000 7f94167ee640 1 --2- 172.21.15.45:0/852798231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94100ffc30 0x7f9410100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.406+0000 7f94167ee640 1 -- 172.21.15.45:0/852798231 >> 172.21.15.45:0/852798231 conn(0x7f94100fb1a0 msgr2=0x7f94100fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.406+0000 7f94167ee640 1 -- 172.21.15.45:0/852798231 shutdown_connections 2024-01-31T05:20:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.406+0000 7f94167ee640 1 -- 172.21.15.45:0/852798231 wait complete. 2024-01-31T05:20:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.406+0000 7f94167ee640 1 Processor -- start 2024-01-31T05:20:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.406+0000 7f94167ee640 1 -- start start 2024-01-31T05:20:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f94167ee640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94100ffc30 0x7f9410103bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f94167ee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9410100e20 0x7f9410102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f94167ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94101017e0 0x7f9410102740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f94167ee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9410102ea0 con 0x7f94101017e0 2024-01-31T05:20:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f94167ee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9410103010 con 0x7f9410100e20 2024-01-31T05:20:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f94167ee640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94101a0db0 con 0x7f94100ffc30 2024-01-31T05:20:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f9414d8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94101017e0 0x7f9410102740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f9414d8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94101017e0 0x7f9410102740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35550/0 (socket says 172.21.15.45:35550) 2024-01-31T05:20:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f9414d8b640 1 -- 172.21.15.45:0/3368279656 learned_addr learned my addr 172.21.15.45:0/3368279656 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f940ffff640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94100ffc30 0x7f9410103bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f940f7fe640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9410100e20 0x7f9410102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f9414d8b640 1 -- 172.21.15.45:0/3368279656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94100ffc30 msgr2=0x7f9410103bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f9414d8b640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94100ffc30 0x7f9410103bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f9414d8b640 1 -- 172.21.15.45:0/3368279656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9410100e20 msgr2=0x7f9410102200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f9414d8b640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9410100e20 0x7f9410102200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f9414d8b640 1 -- 172.21.15.45:0/3368279656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9404009560 con 0x7f94101017e0 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f940f7fe640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9410100e20 0x7f9410102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f940ffff640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94100ffc30 0x7f9410103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f9414d8b640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94101017e0 0x7f9410102740 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f940800db20 tx=0x7f940800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.407+0000 7f940d7fa640 1 -- 172.21.15.45:0/3368279656 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940800bea0 con 0x7f94101017e0 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.408+0000 7f940d7fa640 1 -- 172.21.15.45:0/3368279656 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9408009940 con 0x7f94101017e0 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.408+0000 7f94167ee640 1 -- 172.21.15.45:0/3368279656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94101a1060 con 0x7f94101017e0 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.408+0000 7f940d7fa640 1 -- 172.21.15.45:0/3368279656 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94080106c0 con 0x7f94101017e0 2024-01-31T05:20:29.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.408+0000 7f94167ee640 1 -- 172.21.15.45:0/3368279656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94101a1520 con 0x7f94101017e0 2024-01-31T05:20:29.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.409+0000 7f94167ee640 1 -- 172.21.15.45:0/3368279656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941010b4d0 con 0x7f94101017e0 2024-01-31T05:20:29.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.410+0000 7f940d7fa640 1 -- 172.21.15.45:0/3368279656 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f94080041b0 con 0x7f94101017e0 2024-01-31T05:20:29.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.410+0000 7f940d7fa640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f93f80a12e0 0x7f93f80a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:29.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.410+0000 7f940ffff640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f93f80a12e0 0x7f93f80a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.410+0000 7f940d7fa640 1 -- 172.21.15.45:0/3368279656 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f94080cc4e0 con 0x7f94101017e0 2024-01-31T05:20:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.411+0000 7f940ffff640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f93f80a12e0 0x7f93f80a37a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f94000058c0 tx=0x7f9400005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:29.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.414+0000 7f940d7fa640 1 -- 172.21.15.45:0/3368279656 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f9408095d60 con 0x7f94101017e0 2024-01-31T05:20:29.600 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:20:29 smithi045 podman[73045]: 2024-01-31 05:20:29.219275298 +0000 UTC m=+0.017788589 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:20:29.600 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:20:29 smithi045 podman[73045]: 2024-01-31 05:20:29.340809205 +0000 UTC m=+0.139322489 container create 4dc91940072917083e37d55e6a3b7dae9c582bc2850d4144915eb12d3605c2af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-g, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-01-31T05:20:29.600 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:20:29 smithi045 podman[73045]: 2024-01-31 05:20:29.602045821 +0000 UTC m=+0.400559118 container init 4dc91940072917083e37d55e6a3b7dae9c582bc2850d4144915eb12d3605c2af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-g, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True) 2024-01-31T05:20:29.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.637+0000 7f94167ee640 1 -- 172.21.15.45:0/3368279656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f941010b690 con 0x7f94101017e0 2024-01-31T05:20:29.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.638+0000 7f940d7fa640 1 -- 172.21.15.45:0/3368279656 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7f9408099380 con 0x7f94101017e0 2024-01-31T05:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3782584640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3782584640},{"type":"v1","addr":"172.21.15.156:6837","nonce":3782584640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/3821255404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3821255404},{"type":"v1","addr":"172.21.15.186:6839","nonce":3821255404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-01-31T05:20:29.637 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 10 2024-01-31T05:20:29.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.640+0000 7f94167ee640 1 -- 172.21.15.45:0/3368279656 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f93f80a12e0 msgr2=0x7f93f80a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:29.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.640+0000 7f94167ee640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f93f80a12e0 0x7f93f80a37a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f94000058c0 tx=0x7f9400005850 comp rx=0 tx=0).stop 2024-01-31T05:20:29.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.640+0000 7f94167ee640 1 -- 172.21.15.45:0/3368279656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94101017e0 msgr2=0x7f9410102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:29.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.640+0000 7f94167ee640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94101017e0 0x7f9410102740 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f940800db20 tx=0x7f940800b260 comp rx=0 tx=0).stop 2024-01-31T05:20:29.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.640+0000 7f94167ee640 1 -- 172.21.15.45:0/3368279656 shutdown_connections 2024-01-31T05:20:29.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.640+0000 7f94167ee640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f93f80a12e0 0x7f93f80a37a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:29.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.640+0000 7f94167ee640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94101017e0 0x7f9410102740 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:29.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.640+0000 7f94167ee640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9410100e20 0x7f9410102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:29.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.640+0000 7f94167ee640 1 --2- 172.21.15.45:0/3368279656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94100ffc30 0x7f9410103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:29.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.640+0000 7f94167ee640 1 -- 172.21.15.45:0/3368279656 >> 172.21.15.45:0/3368279656 conn(0x7f94100fb1a0 msgr2=0x7f94100693d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:29.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.640+0000 7f94167ee640 1 -- 172.21.15.45:0/3368279656 shutdown_connections 2024-01-31T05:20:29.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:29.640+0000 7f94167ee640 1 -- 172.21.15.45:0/3368279656 wait complete. 2024-01-31T05:20:29.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:29 smithi045 ceph-mon[31585]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:29.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:29 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3368279656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:29.893 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:20:29 smithi045 podman[73045]: 2024-01-31 05:20:29.605573638 +0000 UTC m=+0.404086931 container start 4dc91940072917083e37d55e6a3b7dae9c582bc2850d4144915eb12d3605c2af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-g, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.schema-version=1.0) 2024-01-31T05:20:29.893 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:20:29 smithi045 ceph-mds[73103]: -- 172.21.15.45:0/3778858404 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56488cf29520 con 0x56488ceff000 2024-01-31T05:20:29.893 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:20:29 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-g[73099]: starting mds.g at 2024-01-31T05:20:29.893 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:20:29 smithi045 bash[73045]: 4dc91940072917083e37d55e6a3b7dae9c582bc2850d4144915eb12d3605c2af 2024-01-31T05:20:29.893 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:20:29 smithi045 systemd[1]: Started Ceph mds.g for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:20:29.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:29 smithi156 ceph-mon[35720]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:29.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:29 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3368279656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:30.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:29 smithi186 ceph-mon[35659]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:30.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:29 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3368279656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:30 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.45:6838/1801709710,v1:172.21.15.45:6839/1801709710] up:boot 2024-01-31T05:20:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:30 smithi156 ceph-mon[35720]: fsmap 10 up:standby 2024-01-31T05:20:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:30 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-31T05:20:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:30 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:30 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:30 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:30.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:30 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:30.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:30 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:30.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:30 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:31.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:30 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.45:6838/1801709710,v1:172.21.15.45:6839/1801709710] up:boot 2024-01-31T05:20:31.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:30 smithi186 ceph-mon[35659]: fsmap 10 up:standby 2024-01-31T05:20:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:30 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-31T05:20:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:30 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:30 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:30 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:30 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:30 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:30 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:31.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:30 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.45:6838/1801709710,v1:172.21.15.45:6839/1801709710] up:boot 2024-01-31T05:20:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:30 smithi045 ceph-mon[31585]: fsmap 10 up:standby 2024-01-31T05:20:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:30 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-31T05:20:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:30 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:30 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:30 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:30 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:30 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:30 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:31.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:31.820 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:31 smithi045 ceph-mon[31585]: Deploying daemon mds.a on smithi045 2024-01-31T05:20:31.820 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:31 smithi045 ceph-mon[31585]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:31.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.837+0000 7f2680899640 1 -- 172.21.15.45:0/1364898899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c101790 msgr2=0x7f267c103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:31.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.837+0000 7f2680899640 1 --2- 172.21.15.45:0/1364898899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c101790 0x7f267c103b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f266c009900 tx=0x7f266c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:31.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.837+0000 7f2680899640 1 -- 172.21.15.45:0/1364898899 shutdown_connections 2024-01-31T05:20:31.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.837+0000 7f2680899640 1 --2- 172.21.15.45:0/1364898899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c1069f0 0x7f267c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:31.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.837+0000 7f2680899640 1 --2- 172.21.15.45:0/1364898899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f267c1040c0 0x7f267c1064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:31.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.837+0000 7f2680899640 1 --2- 172.21.15.45:0/1364898899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c101790 0x7f267c103b80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:31.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.837+0000 7f2680899640 1 -- 172.21.15.45:0/1364898899 >> 172.21.15.45:0/1364898899 conn(0x7f267c0fb180 msgr2=0x7f267c0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:31.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.838+0000 7f2680899640 1 -- 172.21.15.45:0/1364898899 shutdown_connections 2024-01-31T05:20:31.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.838+0000 7f2680899640 1 -- 172.21.15.45:0/1364898899 wait complete. 2024-01-31T05:20:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.838+0000 7f2680899640 1 Processor -- start 2024-01-31T05:20:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.838+0000 7f2680899640 1 -- start start 2024-01-31T05:20:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f2680899640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c101790 0x7f267c196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f2680899640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c1040c0 0x7f267c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f2680899640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f267c1069f0 0x7f267c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f2680899640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c19e830 con 0x7f267c101790 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f2680899640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c19e9a0 con 0x7f267c1069f0 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f2680899640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c19eb60 con 0x7f267c1040c0 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f267a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c101790 0x7f267c196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f267a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c101790 0x7f267c196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35590/0 (socket says 172.21.15.45:35590) 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f267a59c640 1 -- 172.21.15.45:0/1115986979 learned_addr learned my addr 172.21.15.45:0/1115986979 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f267ad9d640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f267c1069f0 0x7f267c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f2679d9b640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c1040c0 0x7f267c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f267a59c640 1 -- 172.21.15.45:0/1115986979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c1040c0 msgr2=0x7f267c197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f267a59c640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c1040c0 0x7f267c197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f267a59c640 1 -- 172.21.15.45:0/1115986979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f267c1069f0 msgr2=0x7f267c19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f267a59c640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f267c1069f0 0x7f267c19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f267a59c640 1 -- 172.21.15.45:0/1115986979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c009560 con 0x7f267c101790 2024-01-31T05:20:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f267a59c640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c101790 0x7f267c196d50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f266c009230 tx=0x7f266c004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f266b7fe640 1 -- 172.21.15.45:0/1115986979 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c004350 con 0x7f267c101790 2024-01-31T05:20:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f266b7fe640 1 -- 172.21.15.45:0/1115986979 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f266c0044b0 con 0x7f267c101790 2024-01-31T05:20:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f2680899640 1 -- 172.21.15.45:0/1115986979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c19ede0 con 0x7f267c101790 2024-01-31T05:20:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.839+0000 7f266b7fe640 1 -- 172.21.15.45:0/1115986979 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c003e10 con 0x7f267c101790 2024-01-31T05:20:31.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.840+0000 7f2680899640 1 -- 172.21.15.45:0/1115986979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c19f200 con 0x7f267c101790 2024-01-31T05:20:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.841+0000 7f2680899640 1 -- 172.21.15.45:0/1115986979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267c0fcc70 con 0x7f267c101790 2024-01-31T05:20:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.842+0000 7f266b7fe640 1 -- 172.21.15.45:0/1115986979 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f266c005000 con 0x7f267c101790 2024-01-31T05:20:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.842+0000 7f266b7fe640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f26640a12e0 0x7f26640a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.843+0000 7f266b7fe640 1 -- 172.21.15.45:0/1115986979 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f266c0eef90 con 0x7f267c101790 2024-01-31T05:20:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.843+0000 7f2679d9b640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f26640a12e0 0x7f26640a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.843+0000 7f2679d9b640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f26640a12e0 0x7f26640a37a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f267c0689b0 tx=0x7f2670005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:31.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:31.849+0000 7f266b7fe640 1 -- 172.21.15.45:0/1115986979 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f266c0b8820 con 0x7f267c101790 2024-01-31T05:20:31.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:31 smithi156 ceph-mon[35720]: Deploying daemon mds.a on smithi045 2024-01-31T05:20:31.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:31 smithi156 ceph-mon[35720]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:32.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.075+0000 7f2680899640 1 -- 172.21.15.45:0/1115986979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f267c10c7d0 con 0x7f267c101790 2024-01-31T05:20:32.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.076+0000 7f266b7fe640 1 -- 172.21.15.45:0/1115986979 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8459 (secure 0 0 0) 0x7f266c043090 con 0x7f267c101790 2024-01-31T05:20:32.075 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:32.076 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1801709710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1801709710},{"type":"v1","addr":"172.21.15.45:6839","nonce":1801709710}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3782584640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3782584640},{"type":"v1","addr":"172.21.15.156:6837","nonce":3782584640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/3821255404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3821255404},{"type":"v1","addr":"172.21.15.186:6839","nonce":3821255404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-01-31T05:20:32.076 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 11 2024-01-31T05:20:32.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.078+0000 7f2680899640 1 -- 172.21.15.45:0/1115986979 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f26640a12e0 msgr2=0x7f26640a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.078+0000 7f2680899640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f26640a12e0 0x7f26640a37a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f267c0689b0 tx=0x7f2670005f90 comp rx=0 tx=0).stop 2024-01-31T05:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.078+0000 7f2680899640 1 -- 172.21.15.45:0/1115986979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c101790 msgr2=0x7f267c196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.078+0000 7f2680899640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c101790 0x7f267c196d50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f266c009230 tx=0x7f266c004ec0 comp rx=0 tx=0).stop 2024-01-31T05:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.078+0000 7f2680899640 1 -- 172.21.15.45:0/1115986979 shutdown_connections 2024-01-31T05:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.078+0000 7f2680899640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f26640a12e0 0x7f26640a37a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.078+0000 7f2680899640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f267c1069f0 0x7f267c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.078+0000 7f2680899640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c1040c0 0x7f267c197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.078+0000 7f2680899640 1 --2- 172.21.15.45:0/1115986979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c101790 0x7f267c196d50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.078+0000 7f2680899640 1 -- 172.21.15.45:0/1115986979 >> 172.21.15.45:0/1115986979 conn(0x7f267c0fb180 msgr2=0x7f267c102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.078+0000 7f2680899640 1 -- 172.21.15.45:0/1115986979 shutdown_connections 2024-01-31T05:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:32.078+0000 7f2680899640 1 -- 172.21.15.45:0/1115986979 wait complete. 2024-01-31T05:20:32.088 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:20:32 smithi045 podman[73427]: 2024-01-31 05:20:32.055233146 +0000 UTC m=+0.171313871 container create 63269242125a7a34628ea2701d23dbfb2bb73600cf4f743ba63b164af3324c6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.31.3) 2024-01-31T05:20:32.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:31 smithi186 ceph-mon[35659]: Deploying daemon mds.a on smithi045 2024-01-31T05:20:32.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:31 smithi186 ceph-mon[35659]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:32.390 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:20:32 smithi045 podman[73427]: 2024-01-31 05:20:32.291284541 +0000 UTC m=+0.407365281 container init 63269242125a7a34628ea2701d23dbfb2bb73600cf4f743ba63b164af3324c6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-a, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:20:32.390 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:20:32 smithi045 podman[73427]: 2024-01-31 05:20:32.294193398 +0000 UTC m=+0.410274122 container start 63269242125a7a34628ea2701d23dbfb2bb73600cf4f743ba63b164af3324c6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-a, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0) 2024-01-31T05:20:32.390 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:20:32 smithi045 ceph-mds[73458]: -- 172.21.15.45:0/1893252248 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563d5b907520 con 0x563d5b8dd000 2024-01-31T05:20:32.390 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:20:32 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-a[73452]: starting mds.a at 2024-01-31T05:20:32.390 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:20:32 smithi045 bash[73427]: 63269242125a7a34628ea2701d23dbfb2bb73600cf4f743ba63b164af3324c6b 2024-01-31T05:20:32.642 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:20:32 smithi045 systemd[1]: Started Ceph mds.a for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:20:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:32 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1115986979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:32 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:32 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:32 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:32 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:32.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:32 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:32.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:32 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:33.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:32 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1115986979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:33.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:32 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:33.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:32 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:33.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:32 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:33.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:32 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:33.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:32 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:33.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:32 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:33.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:32 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1115986979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:33.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:32 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:33.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:32 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:33.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:32 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:33.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:32 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-31T05:20:33.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:32 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-31T05:20:33.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:32 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:33 smithi156 ceph-mon[35720]: Deploying daemon mds.b on smithi156 2024-01-31T05:20:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:33 smithi156 ceph-mon[35720]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:33 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.45:6840/1148586810,v1:172.21.15.45:6841/1148586810] up:boot 2024-01-31T05:20:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:33 smithi156 ceph-mon[35720]: fsmap 11 up:standby 2024-01-31T05:20:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:33 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-31T05:20:34.090 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:34.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:33 smithi186 ceph-mon[35659]: Deploying daemon mds.b on smithi156 2024-01-31T05:20:34.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:33 smithi186 ceph-mon[35659]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:34.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:33 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.45:6840/1148586810,v1:172.21.15.45:6841/1148586810] up:boot 2024-01-31T05:20:34.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:33 smithi186 ceph-mon[35659]: fsmap 11 up:standby 2024-01-31T05:20:34.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:33 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-31T05:20:34.120 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:33 smithi045 ceph-mon[31585]: Deploying daemon mds.b on smithi156 2024-01-31T05:20:34.120 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:33 smithi045 ceph-mon[31585]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:34.120 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:33 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.45:6840/1148586810,v1:172.21.15.45:6841/1148586810] up:boot 2024-01-31T05:20:34.120 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:33 smithi045 ceph-mon[31585]: fsmap 11 up:standby 2024-01-31T05:20:34.121 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:33 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-31T05:20:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.209+0000 7fc2fbcae640 1 -- 172.21.15.45:0/2496484290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2f41017c0 msgr2=0x7fc2f4108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.209+0000 7fc2fbcae640 1 --2- 172.21.15.45:0/2496484290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2f41017c0 0x7fc2f4108050 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009ca0 tx=0x7fc2ec02c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.209+0000 7fc2fbcae640 1 -- 172.21.15.45:0/2496484290 shutdown_connections 2024-01-31T05:20:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.209+0000 7fc2fbcae640 1 --2- 172.21.15.45:0/2496484290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2f41017c0 0x7fc2f4108050 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.209+0000 7fc2fbcae640 1 --2- 172.21.15.45:0/2496484290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2f4100e00 0x7fc2f4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.209+0000 7fc2fbcae640 1 --2- 172.21.15.45:0/2496484290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2f40ffc10 0x7fc2f4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.209+0000 7fc2fbcae640 1 -- 172.21.15.45:0/2496484290 >> 172.21.15.45:0/2496484290 conn(0x7fc2f40fb180 msgr2=0x7fc2f40fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.209+0000 7fc2fbcae640 1 -- 172.21.15.45:0/2496484290 shutdown_connections 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.209+0000 7fc2fbcae640 1 -- 172.21.15.45:0/2496484290 wait complete. 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.209+0000 7fc2fbcae640 1 Processor -- start 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.209+0000 7fc2fbcae640 1 -- start start 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2fbcae640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2f40ffc10 0x7fc2f4196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2fbcae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2f4100e00 0x7fc2f4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2fbcae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2f41017c0 0x7fc2f419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2fbcae640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f419e7d0 con 0x7fc2f4100e00 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2fbcae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f419e940 con 0x7fc2f41017c0 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2fbcae640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f419eb00 con 0x7fc2f40ffc10 2024-01-31T05:20:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2f9249640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2f4100e00 0x7fc2f4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2f9249640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2f4100e00 0x7fc2f4197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35614/0 (socket says 172.21.15.45:35614) 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2f9249640 1 -- 172.21.15.45:0/4222401541 learned_addr learned my addr 172.21.15.45:0/4222401541 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2f9a4a640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2f40ffc10 0x7fc2f4196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2f9249640 1 -- 172.21.15.45:0/4222401541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2f40ffc10 msgr2=0x7fc2f4196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2f9249640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2f40ffc10 0x7fc2f4196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2f9249640 1 -- 172.21.15.45:0/4222401541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2f41017c0 msgr2=0x7fc2f419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2f9249640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2f41017c0 0x7fc2f419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2f9249640 1 -- 172.21.15.45:0/4222401541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec009900 con 0x7fc2f4100e00 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2f9249640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2f4100e00 0x7fc2f4197200 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400d770 tx=0x7fc2e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2eaffd640 1 -- 172.21.15.45:0/4222401541 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4009c30 con 0x7fc2f4100e00 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2eaffd640 1 -- 172.21.15.45:0/4222401541 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2e400de80 con 0x7fc2f4100e00 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2fbcae640 1 -- 172.21.15.45:0/4222401541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f419ede0 con 0x7fc2f4100e00 2024-01-31T05:20:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2eaffd640 1 -- 172.21.15.45:0/4222401541 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4004de0 con 0x7fc2f4100e00 2024-01-31T05:20:34.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.210+0000 7fc2fbcae640 1 -- 172.21.15.45:0/4222401541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f419f260 con 0x7fc2f4100e00 2024-01-31T05:20:34.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.211+0000 7fc2fbcae640 1 -- 172.21.15.45:0/4222401541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f410b4d0 con 0x7fc2f4100e00 2024-01-31T05:20:34.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.212+0000 7fc2eaffd640 1 -- 172.21.15.45:0/4222401541 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fc2e4004f40 con 0x7fc2f4100e00 2024-01-31T05:20:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.213+0000 7fc2eaffd640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc2e00a1360 0x7fc2e00a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.213+0000 7fc2f9a4a640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc2e00a1360 0x7fc2e00a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.213+0000 7fc2eaffd640 1 -- 172.21.15.45:0/4222401541 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc2e40cc210 con 0x7fc2f4100e00 2024-01-31T05:20:34.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.213+0000 7fc2f9a4a640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc2e00a1360 0x7fc2e00a3820 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0005fd0 tx=0x7fc2f0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.217+0000 7fc2eaffd640 1 -- 172.21.15.45:0/4222401541 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fc2e4095a90 con 0x7fc2f4100e00 2024-01-31T05:20:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.434+0000 7fc2fbcae640 1 -- 172.21.15.45:0/4222401541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2f4102250 con 0x7fc2f4100e00 2024-01-31T05:20:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.434+0000 7fc2eaffd640 1 -- 172.21.15.45:0/4222401541 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9251 (secure 0 0 0) 0x7fc2e40990b0 con 0x7fc2f4100e00 2024-01-31T05:20:34.434 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:34.434 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1801709710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1801709710},{"type":"v1","addr":"172.21.15.45:6839","nonce":1801709710}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/1148586810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1148586810},{"type":"v1","addr":"172.21.15.45:6841","nonce":1148586810}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3782584640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3782584640},{"type":"v1","addr":"172.21.15.156:6837","nonce":3782584640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/3821255404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3821255404},{"type":"v1","addr":"172.21.15.186:6839","nonce":3821255404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-01-31T05:20:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 12 2024-01-31T05:20:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.436+0000 7fc2fbcae640 1 -- 172.21.15.45:0/4222401541 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc2e00a1360 msgr2=0x7fc2e00a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.436+0000 7fc2fbcae640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc2e00a1360 0x7fc2e00a3820 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0005fd0 tx=0x7fc2f0005850 comp rx=0 tx=0).stop 2024-01-31T05:20:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.436+0000 7fc2fbcae640 1 -- 172.21.15.45:0/4222401541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2f4100e00 msgr2=0x7fc2f4197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.436+0000 7fc2fbcae640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2f4100e00 0x7fc2f4197200 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400d770 tx=0x7fc2e400dc30 comp rx=0 tx=0).stop 2024-01-31T05:20:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.436+0000 7fc2fbcae640 1 -- 172.21.15.45:0/4222401541 shutdown_connections 2024-01-31T05:20:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.436+0000 7fc2fbcae640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc2e00a1360 0x7fc2e00a3820 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.436+0000 7fc2fbcae640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2f4100e00 0x7fc2f4197200 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.436+0000 7fc2fbcae640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2f41017c0 0x7fc2f419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.436+0000 7fc2fbcae640 1 --2- 172.21.15.45:0/4222401541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2f40ffc10 0x7fc2f4196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.436+0000 7fc2fbcae640 1 -- 172.21.15.45:0/4222401541 >> 172.21.15.45:0/4222401541 conn(0x7fc2f40fb180 msgr2=0x7fc2f40fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.436+0000 7fc2fbcae640 1 -- 172.21.15.45:0/4222401541 shutdown_connections 2024-01-31T05:20:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:34.436+0000 7fc2fbcae640 1 -- 172.21.15.45:0/4222401541 wait complete. 2024-01-31T05:20:34.667 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:20:34 smithi156 systemd[1]: Starting Ceph mds.b for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:20:34.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:34 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/4222401541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:34.967 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:34 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/4222401541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:34.968 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:20:34 smithi156 podman[66585]: 2024-01-31 05:20:34.642452864 +0000 UTC m=+0.017418222 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:20:34.968 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:20:34 smithi156 podman[66585]: 2024-01-31 05:20:34.75894995 +0000 UTC m=+0.133915297 container create d817dbe780f46cafc31cd0710c2390c59de6d0a1560ed518253e9fe27fbb43f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-01-31T05:20:35.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:34 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/4222401541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:35.226 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:20:34 smithi156 podman[66585]: 2024-01-31 05:20:34.969509313 +0000 UTC m=+0.344474666 container init d817dbe780f46cafc31cd0710c2390c59de6d0a1560ed518253e9fe27fbb43f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-01-31T05:20:35.226 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:20:34 smithi156 podman[66585]: 2024-01-31 05:20:34.972180244 +0000 UTC m=+0.347145592 container start d817dbe780f46cafc31cd0710c2390c59de6d0a1560ed518253e9fe27fbb43f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-31T05:20:35.226 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:20:34 smithi156 ceph-mds[66604]: -- 172.21.15.156:0/2603185069 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558844bbb520 con 0x558844b90800 2024-01-31T05:20:35.226 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:20:34 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b[66600]: starting mds.b at 2024-01-31T05:20:35.226 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:20:35 smithi156 bash[66585]: d817dbe780f46cafc31cd0710c2390c59de6d0a1560ed518253e9fe27fbb43f5 2024-01-31T05:20:35.226 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:20:35 smithi156 systemd[1]: Started Ceph mds.b for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:20:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:35 smithi156 ceph-mon[35720]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:35.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:35 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:20:36.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:35 smithi186 ceph-mon[35659]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:36.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:36.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:36.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:36.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:36.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:36.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:35 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:20:36.135 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:35 smithi045 ceph-mon[31585]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:36.135 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:36.135 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:36.136 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:36.136 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:36.136 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:36.136 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:35 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:20:36.447 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.554+0000 7f2e433cd640 1 -- 172.21.15.45:0/1945501337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c101f00 msgr2=0x7f2e3c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.554+0000 7f2e433cd640 1 --2- 172.21.15.45:0/1945501337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c101f00 0x7f2e3c102380 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009900 tx=0x7f2e3002bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.554+0000 7f2e433cd640 1 -- 172.21.15.45:0/1945501337 shutdown_connections 2024-01-31T05:20:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.554+0000 7f2e433cd640 1 --2- 172.21.15.45:0/1945501337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c101f00 0x7f2e3c102380 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.554+0000 7f2e433cd640 1 --2- 172.21.15.45:0/1945501337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e3c1028c0 0x7f2e3c109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.554+0000 7f2e433cd640 1 --2- 172.21.15.45:0/1945501337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e3c100d10 0x7f2e3c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.554+0000 7f2e433cd640 1 -- 172.21.15.45:0/1945501337 >> 172.21.15.45:0/1945501337 conn(0x7f2e3c0fc2a0 msgr2=0x7f2e3c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.554+0000 7f2e433cd640 1 -- 172.21.15.45:0/1945501337 shutdown_connections 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.554+0000 7f2e433cd640 1 -- 172.21.15.45:0/1945501337 wait complete. 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.554+0000 7f2e433cd640 1 Processor -- start 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.554+0000 7f2e433cd640 1 -- start start 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e433cd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e3c100d10 0x7f2e3c196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e433cd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e3c101f00 0x7f2e3c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e433cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c1028c0 0x7f2e3c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e433cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c19e7d0 con 0x7f2e3c1028c0 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e433cd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c19e940 con 0x7f2e3c101f00 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e433cd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c19eb00 con 0x7f2e3c100d10 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e4196a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c1028c0 0x7f2e3c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e4196a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c1028c0 0x7f2e3c19e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35646/0 (socket says 172.21.15.45:35646) 2024-01-31T05:20:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e4196a640 1 -- 172.21.15.45:0/2235790297 learned_addr learned my addr 172.21.15.45:0/2235790297 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e41169640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e3c100d10 0x7f2e3c196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e40968640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e3c101f00 0x7f2e3c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e4196a640 1 -- 172.21.15.45:0/2235790297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e3c100d10 msgr2=0x7f2e3c196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e4196a640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e3c100d10 0x7f2e3c196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e4196a640 1 -- 172.21.15.45:0/2235790297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e3c101f00 msgr2=0x7f2e3c197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e4196a640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e3c101f00 0x7f2e3c197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e4196a640 1 -- 172.21.15.45:0/2235790297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e30009560 con 0x7f2e3c1028c0 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e4196a640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c1028c0 0x7f2e3c19e260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00cad0 tx=0x7f2e2c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e3a7fc640 1 -- 172.21.15.45:0/2235790297 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c00ae20 con 0x7f2e3c1028c0 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e3a7fc640 1 -- 172.21.15.45:0/2235790297 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e2c005020 con 0x7f2e3c1028c0 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e433cd640 1 -- 172.21.15.45:0/2235790297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c19ede0 con 0x7f2e3c1028c0 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e3a7fc640 1 -- 172.21.15.45:0/2235790297 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c00f6b0 con 0x7f2e3c1028c0 2024-01-31T05:20:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.555+0000 7f2e433cd640 1 -- 172.21.15.45:0/2235790297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c19f1b0 con 0x7f2e3c1028c0 2024-01-31T05:20:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.556+0000 7f2e433cd640 1 -- 172.21.15.45:0/2235790297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3c1080b0 con 0x7f2e3c1028c0 2024-01-31T05:20:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.557+0000 7f2e3a7fc640 1 -- 172.21.15.45:0/2235790297 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f2e2c005190 con 0x7f2e3c1028c0 2024-01-31T05:20:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.558+0000 7f2e3a7fc640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2e240a12e0 0x7f2e240a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.558+0000 7f2e41169640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2e240a12e0 0x7f2e240a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.558+0000 7f2e3a7fc640 1 -- 172.21.15.45:0/2235790297 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2e2c0cc780 con 0x7f2e3c1028c0 2024-01-31T05:20:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.558+0000 7f2e41169640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2e240a12e0 0x7f2e240a37a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c1981c0 tx=0x7f2e3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:36.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.562+0000 7f2e3a7fc640 1 -- 172.21.15.45:0/2235790297 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f2e2c096020 con 0x7f2e3c1028c0 2024-01-31T05:20:36.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.797+0000 7f2e433cd640 1 -- 172.21.15.45:0/2235790297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e3c10c5b0 con 0x7f2e3c1028c0 2024-01-31T05:20:36.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.797+0000 7f2e3a7fc640 1 -- 172.21.15.45:0/2235790297 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10046 (secure 0 0 0) 0x7f2e2c097f50 con 0x7f2e3c1028c0 2024-01-31T05:20:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:36.797 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1801709710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1801709710},{"type":"v1","addr":"172.21.15.45:6839","nonce":1801709710}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/1148586810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1148586810},{"type":"v1","addr":"172.21.15.45:6841","nonce":1148586810}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3782584640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3782584640},{"type":"v1","addr":"172.21.15.156:6837","nonce":3782584640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/3821255404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3821255404},{"type":"v1","addr":"172.21.15.186:6839","nonce":3821255404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24503,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/3137398384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3137398384},{"type":"v1","addr":"172.21.15.156:6839","nonce":3137398384}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-01-31T05:20:36.797 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 13 2024-01-31T05:20:36.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.800+0000 7f2e433cd640 1 -- 172.21.15.45:0/2235790297 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2e240a12e0 msgr2=0x7f2e240a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:36.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.800+0000 7f2e433cd640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2e240a12e0 0x7f2e240a37a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c1981c0 tx=0x7f2e3400b410 comp rx=0 tx=0).stop 2024-01-31T05:20:36.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.800+0000 7f2e433cd640 1 -- 172.21.15.45:0/2235790297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c1028c0 msgr2=0x7f2e3c19e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:36.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.800+0000 7f2e433cd640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c1028c0 0x7f2e3c19e260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00cad0 tx=0x7f2e2c00cf90 comp rx=0 tx=0).stop 2024-01-31T05:20:36.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.800+0000 7f2e433cd640 1 -- 172.21.15.45:0/2235790297 shutdown_connections 2024-01-31T05:20:36.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.800+0000 7f2e433cd640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2e240a12e0 0x7f2e240a37a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.800+0000 7f2e433cd640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c1028c0 0x7f2e3c19e260 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.800+0000 7f2e433cd640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e3c101f00 0x7f2e3c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.800+0000 7f2e433cd640 1 --2- 172.21.15.45:0/2235790297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e3c100d10 0x7f2e3c196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.800+0000 7f2e433cd640 1 -- 172.21.15.45:0/2235790297 >> 172.21.15.45:0/2235790297 conn(0x7f2e3c0fc2a0 msgr2=0x7f2e3c105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:36.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.800+0000 7f2e433cd640 1 -- 172.21.15.45:0/2235790297 shutdown_connections 2024-01-31T05:20:36.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.800+0000 7f2e433cd640 1 -- 172.21.15.45:0/2235790297 wait complete. 2024-01-31T05:20:36.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:36 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] up:boot 2024-01-31T05:20:36.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:36 smithi045 ceph-mon[31585]: fsmap 12 up:standby 2024-01-31T05:20:36.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:36 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-31T05:20:36.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:36 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:36.811 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:20:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.932+0000 7fb82bec9640 1 -- 172.21.15.45:0/4275327704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8241069f0 msgr2=0x7fb824108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.932+0000 7fb82bec9640 1 --2- 172.21.15.45:0/4275327704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8241069f0 0x7fb824108e40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009ca0 tx=0x7fb81c02c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.932+0000 7fb82bec9640 1 -- 172.21.15.45:0/4275327704 shutdown_connections 2024-01-31T05:20:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.932+0000 7fb82bec9640 1 --2- 172.21.15.45:0/4275327704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8241069f0 0x7fb824108e40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.932+0000 7fb82bec9640 1 --2- 172.21.15.45:0/4275327704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8241040c0 0x7fb8241064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.932+0000 7fb82bec9640 1 --2- 172.21.15.45:0/4275327704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb824101790 0x7fb824103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.932+0000 7fb82bec9640 1 -- 172.21.15.45:0/4275327704 >> 172.21.15.45:0/4275327704 conn(0x7fb8240fb180 msgr2=0x7fb8240fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb82bec9640 1 -- 172.21.15.45:0/4275327704 shutdown_connections 2024-01-31T05:20:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb82bec9640 1 -- 172.21.15.45:0/4275327704 wait complete. 2024-01-31T05:20:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb82bec9640 1 Processor -- start 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb82bec9640 1 -- start start 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb82bec9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb824101790 0x7fb824196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb82bec9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8241040c0 0x7fb8241971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb82bec9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8241069f0 0x7fb82419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb82bec9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82419e7a0 con 0x7fb824101790 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb82bec9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82419e910 con 0x7fb8241040c0 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb82bec9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82419ead0 con 0x7fb8241069f0 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb829c65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb824101790 0x7fb824196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb829c65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb824101790 0x7fb824196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35662/0 (socket says 172.21.15.45:35662) 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb829c65640 1 -- 172.21.15.45:0/2679738249 learned_addr learned my addr 172.21.15.45:0/2679738249 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.933+0000 7fb82a466640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8241069f0 0x7fb82419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb829464640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8241040c0 0x7fb8241971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb829c65640 1 -- 172.21.15.45:0/2679738249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8241069f0 msgr2=0x7fb82419e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb829c65640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8241069f0 0x7fb82419e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb829c65640 1 -- 172.21.15.45:0/2679738249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8241040c0 msgr2=0x7fb8241971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:36.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb829c65640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8241040c0 0x7fb8241971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:36.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb829c65640 1 -- 172.21.15.45:0/2679738249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c009900 con 0x7fb824101790 2024-01-31T05:20:36.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb829464640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8241040c0 0x7fb8241971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:20:36.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb829c65640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb824101790 0x7fb824196c90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb82000e9f0 tx=0x7fb82000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:36.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb81affd640 1 -- 172.21.15.45:0/2679738249 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820009700 con 0x7fb824101790 2024-01-31T05:20:36.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb82bec9640 1 -- 172.21.15.45:0/2679738249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82419edb0 con 0x7fb824101790 2024-01-31T05:20:36.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb81affd640 1 -- 172.21.15.45:0/2679738249 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb82000cd80 con 0x7fb824101790 2024-01-31T05:20:36.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb81affd640 1 -- 172.21.15.45:0/2679738249 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820018860 con 0x7fb824101790 2024-01-31T05:20:36.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.934+0000 7fb82bec9640 1 -- 172.21.15.45:0/2679738249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82419f200 con 0x7fb824101790 2024-01-31T05:20:36.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.935+0000 7fb82bec9640 1 -- 172.21.15.45:0/2679738249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8240fcc70 con 0x7fb824101790 2024-01-31T05:20:36.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.936+0000 7fb81affd640 1 -- 172.21.15.45:0/2679738249 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fb8200189c0 con 0x7fb824101790 2024-01-31T05:20:36.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.937+0000 7fb81affd640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8100a12e0 0x7fb8100a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:36.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.937+0000 7fb81affd640 1 -- 172.21.15.45:0/2679738249 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb820014070 con 0x7fb824101790 2024-01-31T05:20:36.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.937+0000 7fb829464640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8100a12e0 0x7fb8100a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:36.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.937+0000 7fb829464640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8100a12e0 0x7fb8100a37a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb824198190 tx=0x7fb814005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:36.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:36.941+0000 7fb81affd640 1 -- 172.21.15.45:0/2679738249 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb8200967b0 con 0x7fb824101790 2024-01-31T05:20:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:36 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] up:boot 2024-01-31T05:20:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:36 smithi156 ceph-mon[35720]: fsmap 12 up:standby 2024-01-31T05:20:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:36 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-31T05:20:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:36 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:36 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] up:boot 2024-01-31T05:20:37.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:36 smithi186 ceph-mon[35659]: fsmap 12 up:standby 2024-01-31T05:20:37.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:36 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-31T05:20:37.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:36 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.130+0000 7fb82bec9640 1 -- 172.21.15.45:0/2679738249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb82410c5f0 con 0x7fb824101790 2024-01-31T05:20:37.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.131+0000 7fb81affd640 1 -- 172.21.15.45:0/2679738249 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18436 (secure 0 0 0) 0x7fb820099dd0 con 0x7fb824101790 2024-01-31T05:20:37.130 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:37.130 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:19:50.733440+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:20:37.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.133+0000 7fb82bec9640 1 -- 172.21.15.45:0/2679738249 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8100a12e0 msgr2=0x7fb8100a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.133+0000 7fb82bec9640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8100a12e0 0x7fb8100a37a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb824198190 tx=0x7fb814005f90 comp rx=0 tx=0).stop 2024-01-31T05:20:37.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.133+0000 7fb82bec9640 1 -- 172.21.15.45:0/2679738249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb824101790 msgr2=0x7fb824196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.133+0000 7fb82bec9640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb824101790 0x7fb824196c90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb82000e9f0 tx=0x7fb82000eeb0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.133+0000 7fb82bec9640 1 -- 172.21.15.45:0/2679738249 shutdown_connections 2024-01-31T05:20:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.133+0000 7fb82bec9640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8100a12e0 0x7fb8100a37a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.133+0000 7fb82bec9640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8241069f0 0x7fb82419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.133+0000 7fb82bec9640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8241040c0 0x7fb8241971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.133+0000 7fb82bec9640 1 --2- 172.21.15.45:0/2679738249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb824101790 0x7fb824196c90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.133+0000 7fb82bec9640 1 -- 172.21.15.45:0/2679738249 >> 172.21.15.45:0/2679738249 conn(0x7fb8240fb180 msgr2=0x7fb8241020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.133+0000 7fb82bec9640 1 -- 172.21.15.45:0/2679738249 shutdown_connections 2024-01-31T05:20:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.133+0000 7fb82bec9640 1 -- 172.21.15.45:0/2679738249 wait complete. 2024-01-31T05:20:37.143 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:16:10.856581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:20:37.143 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:20:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.257+0000 7fee281bc640 1 -- 172.21.15.45:0/2944506259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee201017e0 msgr2=0x7fee20108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.257+0000 7fee281bc640 1 --2- 172.21.15.45:0/2944506259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee201017e0 0x7fee20108070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009900 tx=0x7fee1c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.257+0000 7fee281bc640 1 -- 172.21.15.45:0/2944506259 shutdown_connections 2024-01-31T05:20:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.257+0000 7fee281bc640 1 --2- 172.21.15.45:0/2944506259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee201017e0 0x7fee20108070 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.257+0000 7fee281bc640 1 --2- 172.21.15.45:0/2944506259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee20100e20 0x7fee201012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.257+0000 7fee281bc640 1 --2- 172.21.15.45:0/2944506259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee200ffc30 0x7fee20100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.257+0000 7fee281bc640 1 -- 172.21.15.45:0/2944506259 >> 172.21.15.45:0/2944506259 conn(0x7fee200fb1a0 msgr2=0x7fee200fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.257+0000 7fee281bc640 1 -- 172.21.15.45:0/2944506259 shutdown_connections 2024-01-31T05:20:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.257+0000 7fee281bc640 1 -- 172.21.15.45:0/2944506259 wait complete. 2024-01-31T05:20:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee281bc640 1 Processor -- start 2024-01-31T05:20:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee281bc640 1 -- start start 2024-01-31T05:20:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee281bc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee200ffc30 0x7fee2010a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee281bc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee20100e20 0x7fee20108b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee281bc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee201017e0 0x7fee20109070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee281bc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee201097d0 con 0x7fee201017e0 2024-01-31T05:20:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee281bc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee20109940 con 0x7fee200ffc30 2024-01-31T05:20:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee281bc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee201a0d30 con 0x7fee20100e20 2024-01-31T05:20:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee26759640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee201017e0 0x7fee20109070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee26759640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee201017e0 0x7fee20109070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35682/0 (socket says 172.21.15.45:35682) 2024-01-31T05:20:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee26759640 1 -- 172.21.15.45:0/2679522081 learned_addr learned my addr 172.21.15.45:0/2679522081 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee25757640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee20100e20 0x7fee20108b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee25f58640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee200ffc30 0x7fee2010a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee26759640 1 -- 172.21.15.45:0/2679522081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee20100e20 msgr2=0x7fee20108b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee26759640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee20100e20 0x7fee20108b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee26759640 1 -- 172.21.15.45:0/2679522081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee200ffc30 msgr2=0x7fee2010a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee26759640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee200ffc30 0x7fee2010a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.258+0000 7fee26759640 1 -- 172.21.15.45:0/2679522081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c009560 con 0x7fee201017e0 2024-01-31T05:20:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.259+0000 7fee26759640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee201017e0 0x7fee20109070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009430 tx=0x7fee1c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.259+0000 7fee16ffd640 1 -- 172.21.15.45:0/2679522081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c004310 con 0x7fee201017e0 2024-01-31T05:20:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.259+0000 7fee281bc640 1 -- 172.21.15.45:0/2679522081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee201a0f80 con 0x7fee201017e0 2024-01-31T05:20:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.259+0000 7fee16ffd640 1 -- 172.21.15.45:0/2679522081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee1c004470 con 0x7fee201017e0 2024-01-31T05:20:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.259+0000 7fee16ffd640 1 -- 172.21.15.45:0/2679522081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c003c60 con 0x7fee201017e0 2024-01-31T05:20:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.259+0000 7fee281bc640 1 -- 172.21.15.45:0/2679522081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee201a1450 con 0x7fee201017e0 2024-01-31T05:20:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.260+0000 7fee281bc640 1 -- 172.21.15.45:0/2679522081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee20107020 con 0x7fee201017e0 2024-01-31T05:20:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.261+0000 7fee16ffd640 1 -- 172.21.15.45:0/2679522081 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fee1c005000 con 0x7fee201017e0 2024-01-31T05:20:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.262+0000 7fee16ffd640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fee0c0a12e0 0x7fee0c0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.262+0000 7fee25f58640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fee0c0a12e0 0x7fee0c0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.262+0000 7fee16ffd640 1 -- 172.21.15.45:0/2679522081 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fee1c0ee8c0 con 0x7fee201017e0 2024-01-31T05:20:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.262+0000 7fee25f58640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fee0c0a12e0 0x7fee0c0a37a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fee20100c80 tx=0x7fee18009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.266+0000 7fee16ffd640 1 -- 172.21.15.45:0/2679522081 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fee1c0b8140 con 0x7fee201017e0 2024-01-31T05:20:37.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.465+0000 7fee281bc640 1 -- 172.21.15.45:0/2679522081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee2002caa0 con 0x7fee201017e0 2024-01-31T05:20:37.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.466+0000 7fee16ffd640 1 -- 172.21.15.45:0/2679522081 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee1c0052b0 con 0x7fee201017e0 2024-01-31T05:20:37.465 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-01-31T05:20:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.468+0000 7fee281bc640 1 -- 172.21.15.45:0/2679522081 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fee0c0a12e0 msgr2=0x7fee0c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.468+0000 7fee281bc640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fee0c0a12e0 0x7fee0c0a37a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fee20100c80 tx=0x7fee18009500 comp rx=0 tx=0).stop 2024-01-31T05:20:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.468+0000 7fee281bc640 1 -- 172.21.15.45:0/2679522081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee201017e0 msgr2=0x7fee20109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.468+0000 7fee281bc640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee201017e0 0x7fee20109070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009430 tx=0x7fee1c005e40 comp rx=0 tx=0).stop 2024-01-31T05:20:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.468+0000 7fee281bc640 1 -- 172.21.15.45:0/2679522081 shutdown_connections 2024-01-31T05:20:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.468+0000 7fee281bc640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fee0c0a12e0 0x7fee0c0a37a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.468+0000 7fee281bc640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee201017e0 0x7fee20109070 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.468+0000 7fee281bc640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee20100e20 0x7fee20108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.468+0000 7fee281bc640 1 --2- 172.21.15.45:0/2679522081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee200ffc30 0x7fee2010a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.468+0000 7fee281bc640 1 -- 172.21.15.45:0/2679522081 >> 172.21.15.45:0/2679522081 conn(0x7fee200fb1a0 msgr2=0x7fee201041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.468+0000 7fee281bc640 1 -- 172.21.15.45:0/2679522081 shutdown_connections 2024-01-31T05:20:37.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.468+0000 7fee281bc640 1 -- 172.21.15.45:0/2679522081 wait complete. 2024-01-31T05:20:37.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-31T05:20:37.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.583+0000 7fea01821640 1 -- 172.21.15.45:0/576483654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9fc100e20 msgr2=0x7fe9fc1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.583+0000 7fea01821640 1 --2- 172.21.15.45:0/576483654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9fc100e20 0x7fe9fc1012a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009900 tx=0x7fe9f002bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:37.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.583+0000 7fea01821640 1 -- 172.21.15.45:0/576483654 shutdown_connections 2024-01-31T05:20:37.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.583+0000 7fea01821640 1 --2- 172.21.15.45:0/576483654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9fc100e20 0x7fe9fc1012a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.583+0000 7fea01821640 1 --2- 172.21.15.45:0/576483654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9fc1017e0 0x7fe9fc108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.583+0000 7fea01821640 1 --2- 172.21.15.45:0/576483654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc0ffc30 0x7fe9fc100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.583+0000 7fea01821640 1 -- 172.21.15.45:0/576483654 >> 172.21.15.45:0/576483654 conn(0x7fe9fc0fb1a0 msgr2=0x7fe9fc0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.583+0000 7fea01821640 1 -- 172.21.15.45:0/576483654 shutdown_connections 2024-01-31T05:20:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.583+0000 7fea01821640 1 -- 172.21.15.45:0/576483654 wait complete. 2024-01-31T05:20:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fea01821640 1 Processor -- start 2024-01-31T05:20:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fea01821640 1 -- start start 2024-01-31T05:20:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fea01821640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9fc0ffc30 0x7fe9fc103bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fea01821640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9fc100e20 0x7fe9fc102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fea01821640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1017e0 0x7fe9fc102740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fea01821640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc102ea0 con 0x7fe9fc100e20 2024-01-31T05:20:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fea01821640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc103010 con 0x7fe9fc0ffc30 2024-01-31T05:20:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fea01821640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc1a0d80 con 0x7fe9fc1017e0 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fe9fa7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9fc100e20 0x7fe9fc102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fe9fa7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9fc100e20 0x7fe9fc102200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54648/0 (socket says 172.21.15.45:54648) 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fe9fa7fc640 1 -- 172.21.15.45:0/2931819270 learned_addr learned my addr 172.21.15.45:0/2931819270 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fe9faffd640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9fc0ffc30 0x7fe9fc103bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.584+0000 7fe9fb7fe640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1017e0 0x7fe9fc102740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.585+0000 7fe9fa7fc640 1 -- 172.21.15.45:0/2931819270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1017e0 msgr2=0x7fe9fc102740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.585+0000 7fe9fa7fc640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1017e0 0x7fe9fc102740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.585+0000 7fe9fa7fc640 1 -- 172.21.15.45:0/2931819270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9fc0ffc30 msgr2=0x7fe9fc103bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.585+0000 7fe9fa7fc640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9fc0ffc30 0x7fe9fc103bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.585+0000 7fe9fa7fc640 1 -- 172.21.15.45:0/2931819270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0009560 con 0x7fe9fc100e20 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.585+0000 7fe9fa7fc640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9fc100e20 0x7fe9fc102200 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009430 tx=0x7fe9f0004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.585+0000 7fea0081f640 1 -- 172.21.15.45:0/2931819270 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00042d0 con 0x7fe9fc100e20 2024-01-31T05:20:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.585+0000 7fea0081f640 1 -- 172.21.15.45:0/2931819270 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f0004430 con 0x7fe9fc100e20 2024-01-31T05:20:37.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.585+0000 7fea01821640 1 -- 172.21.15.45:0/2931819270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc1a0fd0 con 0x7fe9fc100e20 2024-01-31T05:20:37.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.585+0000 7fea0081f640 1 -- 172.21.15.45:0/2931819270 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0003ca0 con 0x7fe9fc100e20 2024-01-31T05:20:37.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.585+0000 7fea01821640 1 -- 172.21.15.45:0/2931819270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc1a1450 con 0x7fe9fc100e20 2024-01-31T05:20:37.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.586+0000 7fea01821640 1 -- 172.21.15.45:0/2931819270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9fc10b4d0 con 0x7fe9fc100e20 2024-01-31T05:20:37.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.587+0000 7fea0081f640 1 -- 172.21.15.45:0/2931819270 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe9f0005000 con 0x7fe9fc100e20 2024-01-31T05:20:37.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.588+0000 7fea0081f640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9e40a12e0 0x7fe9e40a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:37.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.588+0000 7fe9faffd640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9e40a12e0 0x7fe9e40a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:37.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.588+0000 7fea0081f640 1 -- 172.21.15.45:0/2931819270 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe9f00eef30 con 0x7fe9fc100e20 2024-01-31T05:20:37.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.588+0000 7fe9faffd640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9e40a12e0 0x7fe9e40a37a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc100c80 tx=0x7fe9ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:37.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.592+0000 7fea0081f640 1 -- 172.21.15.45:0/2931819270 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe9f00b8850 con 0x7fe9fc100e20 2024-01-31T05:20:37.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.783+0000 7fea01821640 1 -- 172.21.15.45:0/2931819270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe9fc10b690 con 0x7fe9fc100e20 2024-01-31T05:20:37.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.783+0000 7fea0081f640 1 -- 172.21.15.45:0/2931819270 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe9f00bbe70 con 0x7fe9fc100e20 2024-01-31T05:20:37.782 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:37.783 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-01-31T05:20:37.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:20:37.783 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-01-31T05:20:37.783 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-01-31T05:20:37.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:20:37.783 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-01-31T05:20:37.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.785+0000 7fea01821640 1 -- 172.21.15.45:0/2931819270 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9e40a12e0 msgr2=0x7fe9e40a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.785+0000 7fea01821640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9e40a12e0 0x7fe9e40a37a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc100c80 tx=0x7fe9ec005f90 comp rx=0 tx=0).stop 2024-01-31T05:20:37.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.785+0000 7fea01821640 1 -- 172.21.15.45:0/2931819270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9fc100e20 msgr2=0x7fe9fc102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.785+0000 7fea01821640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9fc100e20 0x7fe9fc102200 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009430 tx=0x7fe9f0004dd0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.785+0000 7fea01821640 1 -- 172.21.15.45:0/2931819270 shutdown_connections 2024-01-31T05:20:37.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.785+0000 7fea01821640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9e40a12e0 0x7fe9e40a37a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.785+0000 7fea01821640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9fc100e20 0x7fe9fc102200 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.785+0000 7fea01821640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1017e0 0x7fe9fc102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.785+0000 7fea01821640 1 --2- 172.21.15.45:0/2931819270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9fc0ffc30 0x7fe9fc103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.785+0000 7fea01821640 1 -- 172.21.15.45:0/2931819270 >> 172.21.15.45:0/2931819270 conn(0x7fe9fc0fb1a0 msgr2=0x7fe9fc0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:37.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.786+0000 7fea01821640 1 -- 172.21.15.45:0/2931819270 shutdown_connections 2024-01-31T05:20:37.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.786+0000 7fea01821640 1 -- 172.21.15.45:0/2931819270 wait complete. 2024-01-31T05:20:37.796 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:37.796 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2235790297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:37.796 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.796 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.796 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.796 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.796 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.796 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.796 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:37.796 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:20:37.797 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.797 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2679738249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:37.797 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:20:37.797 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:37 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2679522081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:20:37.798 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-31T05:20:37.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-31T05:20:37.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.905+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/1559945151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc0ffc10 msgr2=0x7fd9bc100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.905+0000 7fd9c0d3b640 1 --2- 172.21.15.45:0/1559945151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc0ffc10 0x7fd9bc100030 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009900 tx=0x7fd9ac02bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:37.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.905+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/1559945151 shutdown_connections 2024-01-31T05:20:37.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.905+0000 7fd9c0d3b640 1 --2- 172.21.15.45:0/1559945151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9bc1017c0 0x7fd9bc108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.905+0000 7fd9c0d3b640 1 --2- 172.21.15.45:0/1559945151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9bc100e00 0x7fd9bc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.905+0000 7fd9c0d3b640 1 --2- 172.21.15.45:0/1559945151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc0ffc10 0x7fd9bc100030 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.905+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/1559945151 >> 172.21.15.45:0/1559945151 conn(0x7fd9bc0fb1a0 msgr2=0x7fd9bc0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:37.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/1559945151 shutdown_connections 2024-01-31T05:20:37.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/1559945151 wait complete. 2024-01-31T05:20:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9c0d3b640 1 Processor -- start 2024-01-31T05:20:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9c0d3b640 1 -- start start 2024-01-31T05:20:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9c0d3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc0ffc10 0x7fd9bc196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9c0d3b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9bc100e00 0x7fd9bc197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9c0d3b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9bc1017c0 0x7fd9bc19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9c0d3b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc19e6e0 con 0x7fd9bc0ffc10 2024-01-31T05:20:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9c0d3b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc19e850 con 0x7fd9bc100e00 2024-01-31T05:20:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9c0d3b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc19ea10 con 0x7fd9bc1017c0 2024-01-31T05:20:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc0ffc10 0x7fd9bc196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc0ffc10 0x7fd9bc196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54664/0 (socket says 172.21.15.45:54664) 2024-01-31T05:20:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.906+0000 7fd9ba59c640 1 -- 172.21.15.45:0/249701284 learned_addr learned my addr 172.21.15.45:0/249701284 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9b3fff640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9bc100e00 0x7fd9bc197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9bad9d640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9bc1017c0 0x7fd9bc19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9ba59c640 1 -- 172.21.15.45:0/249701284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9bc1017c0 msgr2=0x7fd9bc19e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9ba59c640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9bc1017c0 0x7fd9bc19e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9ba59c640 1 -- 172.21.15.45:0/249701284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9bc100e00 msgr2=0x7fd9bc197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9ba59c640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9bc100e00 0x7fd9bc197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9ba59c640 1 -- 172.21.15.45:0/249701284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac009560 con 0x7fd9bc0ffc10 2024-01-31T05:20:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9bad9d640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9bc1017c0 0x7fd9bc19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9ba59c640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc0ffc10 0x7fd9bc196bf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009230 tx=0x7fd9ac0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9b37fe640 1 -- 172.21.15.45:0/249701284 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0044c0 con 0x7fd9bc0ffc10 2024-01-31T05:20:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/249701284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc19ec90 con 0x7fd9bc0ffc10 2024-01-31T05:20:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9b37fe640 1 -- 172.21.15.45:0/249701284 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9ac0054c0 con 0x7fd9bc0ffc10 2024-01-31T05:20:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9b37fe640 1 -- 172.21.15.45:0/249701284 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac03e450 con 0x7fd9bc0ffc10 2024-01-31T05:20:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.907+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/249701284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc19f1a0 con 0x7fd9bc0ffc10 2024-01-31T05:20:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.908+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/249701284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc106f50 con 0x7fd9bc0ffc10 2024-01-31T05:20:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.910+0000 7fd9b37fe640 1 -- 172.21.15.45:0/249701284 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fd9ac003680 con 0x7fd9bc0ffc10 2024-01-31T05:20:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.910+0000 7fd9b37fe640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd9a80a12e0 0x7fd9a80a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.910+0000 7fd9b3fff640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd9a80a12e0 0x7fd9a80a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.910+0000 7fd9b37fe640 1 -- 172.21.15.45:0/249701284 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd9ac0ef540 con 0x7fd9bc0ffc10 2024-01-31T05:20:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.911+0000 7fd9b3fff640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd9a80a12e0 0x7fd9a80a37a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc198100 tx=0x7fd9a400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:37.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:37.914+0000 7fd9b37fe640 1 -- 172.21.15.45:0/249701284 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd9ac0b8f80 con 0x7fd9bc0ffc10 2024-01-31T05:20:37.958 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:37.958 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2235790297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:37.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:37.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:20:37.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2679738249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:37.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:20:37.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:37 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2679522081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:20:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2235790297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:20:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:37.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2679738249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:37.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:20:37.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:37 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2679522081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:20:38.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.097+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/249701284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd9bc10c7d0 con 0x7fd9bc0ffc10 2024-01-31T05:20:38.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.685+0000 7fd9b37fe640 1 -- 172.21.15.45:0/249701284 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd9ac0430b0 con 0x7fd9bc0ffc10 2024-01-31T05:20:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-01-31T05:20:38.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.687+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/249701284 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd9a80a12e0 msgr2=0x7fd9a80a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:38.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.687+0000 7fd9c0d3b640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd9a80a12e0 0x7fd9a80a37a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc198100 tx=0x7fd9a400a2d0 comp rx=0 tx=0).stop 2024-01-31T05:20:38.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.687+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/249701284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc0ffc10 msgr2=0x7fd9bc196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:38.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.687+0000 7fd9c0d3b640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc0ffc10 0x7fd9bc196bf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009230 tx=0x7fd9ac0040c0 comp rx=0 tx=0).stop 2024-01-31T05:20:38.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.688+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/249701284 shutdown_connections 2024-01-31T05:20:38.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.688+0000 7fd9c0d3b640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd9a80a12e0 0x7fd9a80a37a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:38.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.688+0000 7fd9c0d3b640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9bc1017c0 0x7fd9bc19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:38.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.688+0000 7fd9c0d3b640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9bc100e00 0x7fd9bc197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:38.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.688+0000 7fd9c0d3b640 1 --2- 172.21.15.45:0/249701284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9bc0ffc10 0x7fd9bc196bf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:38.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.688+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/249701284 >> 172.21.15.45:0/249701284 conn(0x7fd9bc0fb1a0 msgr2=0x7fd9bc104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:38.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.688+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/249701284 shutdown_connections 2024-01-31T05:20:38.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.688+0000 7fd9c0d3b640 1 -- 172.21.15.45:0/249701284 wait complete. 2024-01-31T05:20:38.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-31T05:20:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.827+0000 7f0bff953640 1 -- 172.21.15.45:0/4197314178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf81017a0 msgr2=0x7f0bf8108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.827+0000 7f0bff953640 1 --2- 172.21.15.45:0/4197314178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf81017a0 0x7f0bf8108030 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009ca0 tx=0x7f0bf002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.827+0000 7f0bff953640 1 -- 172.21.15.45:0/4197314178 shutdown_connections 2024-01-31T05:20:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.827+0000 7f0bff953640 1 --2- 172.21.15.45:0/4197314178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf81017a0 0x7f0bf8108030 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.827+0000 7f0bff953640 1 --2- 172.21.15.45:0/4197314178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bf8100de0 0x7f0bf8101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.827+0000 7f0bff953640 1 --2- 172.21.15.45:0/4197314178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bf80ffbf0 0x7f0bf8100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.827+0000 7f0bff953640 1 -- 172.21.15.45:0/4197314178 >> 172.21.15.45:0/4197314178 conn(0x7f0bf80fb180 msgr2=0x7f0bf80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.827+0000 7f0bff953640 1 -- 172.21.15.45:0/4197314178 shutdown_connections 2024-01-31T05:20:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.827+0000 7f0bff953640 1 -- 172.21.15.45:0/4197314178 wait complete. 2024-01-31T05:20:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bff953640 1 Processor -- start 2024-01-31T05:20:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bff953640 1 -- start start 2024-01-31T05:20:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bff953640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bf80ffbf0 0x7f0bf8198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bff953640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bf8100de0 0x7f0bf81993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bff953640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf81017a0 0x7f0bf81a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bff953640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf81a0960 con 0x7f0bf81017a0 2024-01-31T05:20:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bff953640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf81a0ad0 con 0x7f0bf8100de0 2024-01-31T05:20:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bff953640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf81a0c90 con 0x7f0bf80ffbf0 2024-01-31T05:20:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bfdef0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf81017a0 0x7f0bf81a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bfdef0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf81017a0 0x7f0bf81a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54678/0 (socket says 172.21.15.45:54678) 2024-01-31T05:20:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bfdef0640 1 -- 172.21.15.45:0/1655244005 learned_addr learned my addr 172.21.15.45:0/1655244005 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bfceee640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bf8100de0 0x7f0bf81993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bfd6ef640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bf80ffbf0 0x7f0bf8198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bfdef0640 1 -- 172.21.15.45:0/1655244005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bf80ffbf0 msgr2=0x7f0bf8198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bfdef0640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bf80ffbf0 0x7f0bf8198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bfdef0640 1 -- 172.21.15.45:0/1655244005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bf8100de0 msgr2=0x7f0bf81993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bfdef0640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bf8100de0 0x7f0bf81993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bfdef0640 1 -- 172.21.15.45:0/1655244005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf4009490 con 0x7f0bf81017a0 2024-01-31T05:20:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.828+0000 7f0bfdef0640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf81017a0 0x7f0bf81a0420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0037040 tx=0x7f0bf00093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.829+0000 7f0bee7fc640 1 -- 172.21.15.45:0/1655244005 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0034c70 con 0x7f0bf81017a0 2024-01-31T05:20:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.829+0000 7f0bff953640 1 -- 172.21.15.45:0/1655244005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf0009900 con 0x7f0bf81017a0 2024-01-31T05:20:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.829+0000 7f0bee7fc640 1 -- 172.21.15.45:0/1655244005 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bf0003e30 con 0x7f0bf81017a0 2024-01-31T05:20:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.829+0000 7f0bee7fc640 1 -- 172.21.15.45:0/1655244005 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0004800 con 0x7f0bf81017a0 2024-01-31T05:20:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.829+0000 7f0bff953640 1 -- 172.21.15.45:0/1655244005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf81a1220 con 0x7f0bf81017a0 2024-01-31T05:20:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.830+0000 7f0bff953640 1 -- 172.21.15.45:0/1655244005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf810b490 con 0x7f0bf81017a0 2024-01-31T05:20:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.831+0000 7f0bee7fc640 1 -- 172.21.15.45:0/1655244005 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f0bf003b040 con 0x7f0bf81017a0 2024-01-31T05:20:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.832+0000 7f0bee7fc640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0be40a12e0 0x7f0be40a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.832+0000 7f0bfd6ef640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0be40a12e0 0x7f0be40a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.832+0000 7f0bee7fc640 1 -- 172.21.15.45:0/1655244005 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f0bf00ef1c0 con 0x7f0bf81017a0 2024-01-31T05:20:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.832+0000 7f0bfd6ef640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0be40a12e0 0x7f0be40a37a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8100c40 tx=0x7f0bf4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:38.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:38.836+0000 7f0bee7fc640 1 -- 172.21.15.45:0/1655244005 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f0bf00b8a00 con 0x7f0bf81017a0 2024-01-31T05:20:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:38 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2931819270' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-01-31T05:20:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:38 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/249701284' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-01-31T05:20:39.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.021+0000 7f0bff953640 1 -- 172.21.15.45:0/1655244005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0bf81a14b0 con 0x7f0bf81017a0 2024-01-31T05:20:39.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:38 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2931819270' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-01-31T05:20:39.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:38 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/249701284' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-01-31T05:20:39.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:38 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2931819270' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-01-31T05:20:39.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:38 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/249701284' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-01-31T05:20:39.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.693+0000 7f0bee7fc640 1 -- 172.21.15.45:0/1655244005 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0bf00bc020 con 0x7f0bf81017a0 2024-01-31T05:20:39.692 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-01-31T05:20:39.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.696+0000 7f0bff953640 1 -- 172.21.15.45:0/1655244005 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0be40a12e0 msgr2=0x7f0be40a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:39.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.696+0000 7f0bff953640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0be40a12e0 0x7f0be40a37a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8100c40 tx=0x7f0bf4005f90 comp rx=0 tx=0).stop 2024-01-31T05:20:39.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.696+0000 7f0bff953640 1 -- 172.21.15.45:0/1655244005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf81017a0 msgr2=0x7f0bf81a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:39.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.696+0000 7f0bff953640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf81017a0 0x7f0bf81a0420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0037040 tx=0x7f0bf00093d0 comp rx=0 tx=0).stop 2024-01-31T05:20:39.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.696+0000 7f0bff953640 1 -- 172.21.15.45:0/1655244005 shutdown_connections 2024-01-31T05:20:39.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.696+0000 7f0bff953640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0be40a12e0 0x7f0be40a37a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:39.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.696+0000 7f0bff953640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf81017a0 0x7f0bf81a0420 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:39.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.696+0000 7f0bff953640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bf8100de0 0x7f0bf81993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:39.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.696+0000 7f0bff953640 1 --2- 172.21.15.45:0/1655244005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bf80ffbf0 0x7f0bf8198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:39.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.696+0000 7f0bff953640 1 -- 172.21.15.45:0/1655244005 >> 172.21.15.45:0/1655244005 conn(0x7f0bf80fb180 msgr2=0x7f0bf80fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:39.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.697+0000 7f0bff953640 1 -- 172.21.15.45:0/1655244005 shutdown_connections 2024-01-31T05:20:39.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.697+0000 7f0bff953640 1 -- 172.21.15.45:0/1655244005 wait complete. 2024-01-31T05:20:39.718 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-31T05:20:39.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.829+0000 7f4dedfee640 1 -- 172.21.15.45:0/4280956333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de8101f10 msgr2=0x7f4de8102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:39.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.829+0000 7f4dedfee640 1 --2- 172.21.15.45:0/4280956333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de8101f10 0x7f4de8102390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009900 tx=0x7f4ddc02bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:39.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.829+0000 7f4dedfee640 1 -- 172.21.15.45:0/4280956333 shutdown_connections 2024-01-31T05:20:39.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.829+0000 7f4dedfee640 1 --2- 172.21.15.45:0/4280956333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de8101f10 0x7f4de8102390 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:39.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.829+0000 7f4dedfee640 1 --2- 172.21.15.45:0/4280956333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4de81028d0 0x7f4de8109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:39.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.829+0000 7f4dedfee640 1 --2- 172.21.15.45:0/4280956333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de8100d20 0x7f4de8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:39.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.829+0000 7f4dedfee640 1 -- 172.21.15.45:0/4280956333 >> 172.21.15.45:0/4280956333 conn(0x7f4de80fc290 msgr2=0x7f4de80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:39.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.829+0000 7f4dedfee640 1 -- 172.21.15.45:0/4280956333 shutdown_connections 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.829+0000 7f4dedfee640 1 -- 172.21.15.45:0/4280956333 wait complete. 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.830+0000 7f4dedfee640 1 Processor -- start 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.830+0000 7f4dedfee640 1 -- start start 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.830+0000 7f4dedfee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de8100d20 0x7f4de8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.830+0000 7f4dedfee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de8101f10 0x7f4de80730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.830+0000 7f4dedfee640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4de81028d0 0x7f4de8073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.830+0000 7f4dedfee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8073d80 con 0x7f4de8101f10 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.830+0000 7f4dedfee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8073ef0 con 0x7f4de8100d20 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.830+0000 7f4dedfee640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de81a0d20 con 0x7f4de81028d0 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.830+0000 7f4de6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de8101f10 0x7f4de80730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.830+0000 7f4de6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de8101f10 0x7f4de80730e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54694/0 (socket says 172.21.15.45:54694) 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.830+0000 7f4de6ffd640 1 -- 172.21.15.45:0/2169164012 learned_addr learned my addr 172.21.15.45:0/2169164012 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4de77fe640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de8100d20 0x7f4de8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4de7fff640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4de81028d0 0x7f4de8073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4de6ffd640 1 -- 172.21.15.45:0/2169164012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4de81028d0 msgr2=0x7f4de8073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4de6ffd640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4de81028d0 0x7f4de8073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4de6ffd640 1 -- 172.21.15.45:0/2169164012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de8100d20 msgr2=0x7f4de8074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4de6ffd640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de8100d20 0x7f4de8074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4de6ffd640 1 -- 172.21.15.45:0/2169164012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc009560 con 0x7f4de8101f10 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4de6ffd640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de8101f10 0x7f4de80730e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009430 tx=0x7f4ddc004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4de4ff9640 1 -- 172.21.15.45:0/2169164012 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc004350 con 0x7f4de8101f10 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4de4ff9640 1 -- 172.21.15.45:0/2169164012 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ddc0044b0 con 0x7f4de8101f10 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4dedfee640 1 -- 172.21.15.45:0/2169164012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de81a0f70 con 0x7f4de8101f10 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4de4ff9640 1 -- 172.21.15.45:0/2169164012 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc003e80 con 0x7f4de8101f10 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.831+0000 7f4dedfee640 1 -- 172.21.15.45:0/2169164012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de81a13f0 con 0x7f4de8101f10 2024-01-31T05:20:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.832+0000 7f4dedfee640 1 -- 172.21.15.45:0/2169164012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de8108060 con 0x7f4de8101f10 2024-01-31T05:20:39.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.833+0000 7f4de4ff9640 1 -- 172.21.15.45:0/2169164012 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f4ddc0035f0 con 0x7f4de8101f10 2024-01-31T05:20:39.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.834+0000 7f4de4ff9640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f4dd00a12e0 0x7f4dd00a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:39.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.834+0000 7f4de77fe640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f4dd00a12e0 0x7f4dd00a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:39.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.834+0000 7f4de4ff9640 1 -- 172.21.15.45:0/2169164012 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f4ddc0ef2f0 con 0x7f4de8101f10 2024-01-31T05:20:39.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.834+0000 7f4de77fe640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f4dd00a12e0 0x7f4dd00a37a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4de8101d70 tx=0x7f4dd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:39.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:39.838+0000 7f4de4ff9640 1 -- 172.21.15.45:0/2169164012 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f4ddc0b8920 con 0x7f4de8101f10 2024-01-31T05:20:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/249701284' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-01-31T05:20:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: osdmap e83: 12 total, 12 up, 12 in 2024-01-31T05:20:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:39.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:39.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:39.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:39.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:39.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:20:39.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:39.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:39 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1655244005' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-01-31T05:20:40.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.055+0000 7f4dedfee640 1 -- 172.21.15.45:0/2169164012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4de810c7a0 con 0x7f4de8101f10 2024-01-31T05:20:40.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:40.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/249701284' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-01-31T05:20:40.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: osdmap e83: 12 total, 12 up, 12 in 2024-01-31T05:20:40.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:20:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:39 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1655244005' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/249701284' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: osdmap e83: 12 total, 12 up, 12 in 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:20:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:39 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1655244005' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-01-31T05:20:40.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.695+0000 7f4de4ff9640 1 -- 172.21.15.45:0/2169164012 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4ddc0bbf40 con 0x7f4de8101f10 2024-01-31T05:20:40.694 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 2 and data pool 3 2024-01-31T05:20:40.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.699+0000 7f4dedfee640 1 -- 172.21.15.45:0/2169164012 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f4dd00a12e0 msgr2=0x7f4dd00a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:40.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.699+0000 7f4dedfee640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f4dd00a12e0 0x7f4dd00a37a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4de8101d70 tx=0x7f4dd8005f90 comp rx=0 tx=0).stop 2024-01-31T05:20:40.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.699+0000 7f4dedfee640 1 -- 172.21.15.45:0/2169164012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de8101f10 msgr2=0x7f4de80730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:40.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.699+0000 7f4dedfee640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de8101f10 0x7f4de80730e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009430 tx=0x7f4ddc004ec0 comp rx=0 tx=0).stop 2024-01-31T05:20:40.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.699+0000 7f4dedfee640 1 -- 172.21.15.45:0/2169164012 shutdown_connections 2024-01-31T05:20:40.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.699+0000 7f4dedfee640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f4dd00a12e0 0x7f4dd00a37a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:40.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.699+0000 7f4dedfee640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de8101f10 0x7f4de80730e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:40.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.699+0000 7f4dedfee640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4de81028d0 0x7f4de8073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:40.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.699+0000 7f4dedfee640 1 --2- 172.21.15.45:0/2169164012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de8100d20 0x7f4de8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:40.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.699+0000 7f4dedfee640 1 -- 172.21.15.45:0/2169164012 >> 172.21.15.45:0/2169164012 conn(0x7f4de80fc290 msgr2=0x7f4de8105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:40.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.699+0000 7f4dedfee640 1 -- 172.21.15.45:0/2169164012 shutdown_connections 2024-01-31T05:20:40.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.699+0000 7f4dedfee640 1 -- 172.21.15.45:0/2169164012 wait complete. 2024-01-31T05:20:40.722 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:20:40.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.833+0000 7fdcded42640 1 -- 172.21.15.45:0/481318290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd81028d0 msgr2=0x7fdcd8109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.833+0000 7fdcded42640 1 --2- 172.21.15.45:0/481318290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd81028d0 0x7fdcd8109160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009ca0 tx=0x7fdcd002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.833+0000 7fdcded42640 1 -- 172.21.15.45:0/481318290 shutdown_connections 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.833+0000 7fdcded42640 1 --2- 172.21.15.45:0/481318290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd81028d0 0x7fdcd8109160 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.833+0000 7fdcded42640 1 --2- 172.21.15.45:0/481318290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcd8101f10 0x7fdcd8102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.833+0000 7fdcded42640 1 --2- 172.21.15.45:0/481318290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcd8100d20 0x7fdcd8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.833+0000 7fdcded42640 1 -- 172.21.15.45:0/481318290 >> 172.21.15.45:0/481318290 conn(0x7fdcd80fc2d0 msgr2=0x7fdcd80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.833+0000 7fdcded42640 1 -- 172.21.15.45:0/481318290 shutdown_connections 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.833+0000 7fdcded42640 1 -- 172.21.15.45:0/481318290 wait complete. 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcded42640 1 Processor -- start 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcded42640 1 -- start start 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcded42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd8100d20 0x7fdcd8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcded42640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcd8101f10 0x7fdcd80730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcded42640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcd81028d0 0x7fdcd8073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcded42640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd8073d80 con 0x7fdcd8100d20 2024-01-31T05:20:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcded42640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd8073ef0 con 0x7fdcd81028d0 2024-01-31T05:20:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcded42640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd81a0cc0 con 0x7fdcd8101f10 2024-01-31T05:20:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcdcade640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd8100d20 0x7fdcd8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcdcade640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd8100d20 0x7fdcd8074aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54714/0 (socket says 172.21.15.45:54714) 2024-01-31T05:20:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcdcade640 1 -- 172.21.15.45:0/32937965 learned_addr learned my addr 172.21.15.45:0/32937965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcd7fff640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcd8101f10 0x7fdcd80730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcdcade640 1 -- 172.21.15.45:0/32937965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcd8101f10 msgr2=0x7fdcd80730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcdcade640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcd8101f10 0x7fdcd80730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcdcade640 1 -- 172.21.15.45:0/32937965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcd81028d0 msgr2=0x7fdcd8073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcdcade640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcd81028d0 0x7fdcd8073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:40.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.834+0000 7fdcdcade640 1 -- 172.21.15.45:0/32937965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd0009900 con 0x7fdcd8100d20 2024-01-31T05:20:40.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.835+0000 7fdcdcade640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd8100d20 0x7fdcd8074aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00c9a0 tx=0x7fdccc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:40.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.835+0000 7fdcd5ffb640 1 -- 172.21.15.45:0/32937965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004220 con 0x7fdcd8100d20 2024-01-31T05:20:40.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.835+0000 7fdcd5ffb640 1 -- 172.21.15.45:0/32937965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdccc007e70 con 0x7fdcd8100d20 2024-01-31T05:20:40.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.835+0000 7fdcded42640 1 -- 172.21.15.45:0/32937965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd81a0f70 con 0x7fdcd8100d20 2024-01-31T05:20:40.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.835+0000 7fdcd5ffb640 1 -- 172.21.15.45:0/32937965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc00f450 con 0x7fdcd8100d20 2024-01-31T05:20:40.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.835+0000 7fdcded42640 1 -- 172.21.15.45:0/32937965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd81a13a0 con 0x7fdcd8100d20 2024-01-31T05:20:40.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.836+0000 7fdcded42640 1 -- 172.21.15.45:0/32937965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd8108110 con 0x7fdcd8100d20 2024-01-31T05:20:40.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.837+0000 7fdcd5ffb640 1 -- 172.21.15.45:0/32937965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fdccc0079b0 con 0x7fdcd8100d20 2024-01-31T05:20:40.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.838+0000 7fdcd5ffb640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdcc80a12e0 0x7fdcc80a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:40.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.838+0000 7fdcd7fff640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdcc80a12e0 0x7fdcc80a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:40.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.838+0000 7fdcd5ffb640 1 -- 172.21.15.45:0/32937965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdccc0ccac0 con 0x7fdcd8100d20 2024-01-31T05:20:40.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.838+0000 7fdcd7fff640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdcc80a12e0 0x7fdcc80a37a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8074810 tx=0x7fdcc400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:40.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:40.842+0000 7fdcd5ffb640 1 -- 172.21.15.45:0/32937965 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fdccc095ff0 con 0x7fdcd8100d20 2024-01-31T05:20:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:40 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1655244005' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-01-31T05:20:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:40 smithi156 ceph-mon[35720]: osdmap e84: 12 total, 12 up, 12 in 2024-01-31T05:20:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:40 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2169164012' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-01-31T05:20:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:40 smithi156 ceph-mon[35720]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-01-31T05:20:41.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.023+0000 7fdcded42640 1 -- 172.21.15.45:0/32937965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcd802cc30 con 0x7fdcd8100d20 2024-01-31T05:20:41.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.025+0000 7fdcd5ffb640 1 -- 172.21.15.45:0/32937965 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22146 (secure 0 0 0) 0x7fdccc01d090 con 0x7fdcd8100d20 2024-01-31T05:20:41.024 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:41.024 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:20:40.057232+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:20:41.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.027+0000 7fdcded42640 1 -- 172.21.15.45:0/32937965 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdcc80a12e0 msgr2=0x7fdcc80a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.027+0000 7fdcded42640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdcc80a12e0 0x7fdcc80a37a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8074810 tx=0x7fdcc400b390 comp rx=0 tx=0).stop 2024-01-31T05:20:41.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.027+0000 7fdcded42640 1 -- 172.21.15.45:0/32937965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd8100d20 msgr2=0x7fdcd8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.027+0000 7fdcded42640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd8100d20 0x7fdcd8074aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00c9a0 tx=0x7fdccc00ce60 comp rx=0 tx=0).stop 2024-01-31T05:20:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.027+0000 7fdcded42640 1 -- 172.21.15.45:0/32937965 shutdown_connections 2024-01-31T05:20:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.027+0000 7fdcded42640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdcc80a12e0 0x7fdcc80a37a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.027+0000 7fdcded42640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcd81028d0 0x7fdcd8073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.027+0000 7fdcded42640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcd8101f10 0x7fdcd80730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.027+0000 7fdcded42640 1 --2- 172.21.15.45:0/32937965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd8100d20 0x7fdcd8074aa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.027+0000 7fdcded42640 1 -- 172.21.15.45:0/32937965 >> 172.21.15.45:0/32937965 conn(0x7fdcd80fc2d0 msgr2=0x7fdcd81052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.027+0000 7fdcded42640 1 -- 172.21.15.45:0/32937965 shutdown_connections 2024-01-31T05:20:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.027+0000 7fdcded42640 1 -- 172.21.15.45:0/32937965 wait complete. 2024-01-31T05:20:41.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:40 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1655244005' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-01-31T05:20:41.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:40 smithi045 ceph-mon[31585]: osdmap e84: 12 total, 12 up, 12 in 2024-01-31T05:20:41.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:40 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2169164012' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-01-31T05:20:41.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:40 smithi045 ceph-mon[31585]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-01-31T05:20:41.037 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:20:41.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:40 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1655244005' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-01-31T05:20:41.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:40 smithi186 ceph-mon[35659]: osdmap e84: 12 total, 12 up, 12 in 2024-01-31T05:20:41.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:40 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2169164012' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-01-31T05:20:41.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:40 smithi186 ceph-mon[35659]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-01-31T05:20:41.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.140+0000 7f942e31f640 1 -- 172.21.15.45:0/2363017561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9428100e00 msgr2=0x7f9428101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.140+0000 7f942e31f640 1 --2- 172.21.15.45:0/2363017561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9428100e00 0x7f9428101280 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f941c009900 tx=0x7f941c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:41.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.140+0000 7f942e31f640 1 -- 172.21.15.45:0/2363017561 shutdown_connections 2024-01-31T05:20:41.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.140+0000 7f942e31f640 1 --2- 172.21.15.45:0/2363017561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9428100e00 0x7f9428101280 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.140+0000 7f942e31f640 1 --2- 172.21.15.45:0/2363017561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94281017c0 0x7f9428108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.140+0000 7f942e31f640 1 --2- 172.21.15.45:0/2363017561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94280ffc10 0x7f9428100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.140+0000 7f942e31f640 1 -- 172.21.15.45:0/2363017561 >> 172.21.15.45:0/2363017561 conn(0x7f94280fb180 msgr2=0x7f94280fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.140+0000 7f942e31f640 1 -- 172.21.15.45:0/2363017561 shutdown_connections 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.140+0000 7f942e31f640 1 -- 172.21.15.45:0/2363017561 wait complete. 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.140+0000 7f942e31f640 1 Processor -- start 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.140+0000 7f942e31f640 1 -- start start 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f942e31f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94280ffc10 0x7f9428198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f942e31f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428100e00 0x7f9428199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f942e31f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94281017c0 0x7f94281a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f942e31f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94281a0a30 con 0x7f94280ffc10 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f942e31f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94281a0ba0 con 0x7f9428100e00 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f942e31f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94281a0d60 con 0x7f94281017c0 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f9427fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94280ffc10 0x7f9428198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f9427fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94280ffc10 0x7f9428198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54740/0 (socket says 172.21.15.45:54740) 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f9427fff640 1 -- 172.21.15.45:0/2047522748 learned_addr learned my addr 172.21.15.45:0/2047522748 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f94277fe640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428100e00 0x7f9428199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f942c8bc640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94281017c0 0x7f94281a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f9427fff640 1 -- 172.21.15.45:0/2047522748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94281017c0 msgr2=0x7f94281a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f9427fff640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94281017c0 0x7f94281a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f9427fff640 1 -- 172.21.15.45:0/2047522748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428100e00 msgr2=0x7f9428199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f9427fff640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428100e00 0x7f9428199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f9427fff640 1 -- 172.21.15.45:0/2047522748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c009560 con 0x7f94280ffc10 2024-01-31T05:20:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f9427fff640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94280ffc10 0x7f9428198f40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f941800d770 tx=0x7f941800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.141+0000 7f94257fa640 1 -- 172.21.15.45:0/2047522748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418009c30 con 0x7f94280ffc10 2024-01-31T05:20:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.142+0000 7f94257fa640 1 -- 172.21.15.45:0/2047522748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f941800de80 con 0x7f94280ffc10 2024-01-31T05:20:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.142+0000 7f94257fa640 1 -- 172.21.15.45:0/2047522748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418003e20 con 0x7f94280ffc10 2024-01-31T05:20:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.142+0000 7f942e31f640 1 -- 172.21.15.45:0/2047522748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94281a0fe0 con 0x7f94280ffc10 2024-01-31T05:20:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.142+0000 7f942e31f640 1 -- 172.21.15.45:0/2047522748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94281a14b0 con 0x7f94280ffc10 2024-01-31T05:20:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.143+0000 7f942e31f640 1 -- 172.21.15.45:0/2047522748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942810b4b0 con 0x7f94280ffc10 2024-01-31T05:20:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.144+0000 7f94257fa640 1 -- 172.21.15.45:0/2047522748 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f9418003f80 con 0x7f94280ffc10 2024-01-31T05:20:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.144+0000 7f94257fa640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f94100a1360 0x7f94100a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.144+0000 7f94257fa640 1 -- 172.21.15.45:0/2047522748 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f94180cc390 con 0x7f94280ffc10 2024-01-31T05:20:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.144+0000 7f94277fe640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f94100a1360 0x7f94100a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:41.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.145+0000 7f94277fe640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f94100a1360 0x7f94100a3820 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f941c005ab0 tx=0x7f941c02c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:41.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.148+0000 7f94257fa640 1 -- 172.21.15.45:0/2047522748 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f9418095960 con 0x7f94280ffc10 2024-01-31T05:20:41.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.327+0000 7f942e31f640 1 -- 172.21.15.45:0/2047522748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f942810b690 con 0x7f94280ffc10 2024-01-31T05:20:41.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.329+0000 7f94257fa640 1 -- 172.21.15.45:0/2047522748 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22146 (secure 0 0 0) 0x7f9418098f80 con 0x7f94280ffc10 2024-01-31T05:20:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:41.328 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:20:40.057232+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:20:41.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.330+0000 7f942e31f640 1 -- 172.21.15.45:0/2047522748 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f94100a1360 msgr2=0x7f94100a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.331+0000 7f942e31f640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f94100a1360 0x7f94100a3820 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f941c005ab0 tx=0x7f941c02c5d0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.331+0000 7f942e31f640 1 -- 172.21.15.45:0/2047522748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94280ffc10 msgr2=0x7f9428198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.331+0000 7f942e31f640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94280ffc10 0x7f9428198f40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f941800d770 tx=0x7f941800dc30 comp rx=0 tx=0).stop 2024-01-31T05:20:41.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.331+0000 7f942e31f640 1 -- 172.21.15.45:0/2047522748 shutdown_connections 2024-01-31T05:20:41.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.331+0000 7f942e31f640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f94100a1360 0x7f94100a3820 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.331+0000 7f942e31f640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94281017c0 0x7f94281a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.331+0000 7f942e31f640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9428100e00 0x7f9428199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.331+0000 7f942e31f640 1 --2- 172.21.15.45:0/2047522748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94280ffc10 0x7f9428198f40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.331+0000 7f942e31f640 1 -- 172.21.15.45:0/2047522748 >> 172.21.15.45:0/2047522748 conn(0x7f94280fb180 msgr2=0x7f9428067ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:41.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.331+0000 7f942e31f640 1 -- 172.21.15.45:0/2047522748 shutdown_connections 2024-01-31T05:20:41.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.331+0000 7f942e31f640 1 -- 172.21.15.45:0/2047522748 wait complete. 2024-01-31T05:20:41.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-31T05:20:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.446+0000 7f1d655d9640 1 -- 172.21.15.45:0/3751563814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d60101ef0 msgr2=0x7f1d60102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.446+0000 7f1d655d9640 1 --2- 172.21.15.45:0/3751563814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d60101ef0 0x7f1d60102370 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009900 tx=0x7f1d5402bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.446+0000 7f1d655d9640 1 -- 172.21.15.45:0/3751563814 shutdown_connections 2024-01-31T05:20:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.446+0000 7f1d655d9640 1 --2- 172.21.15.45:0/3751563814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d60101ef0 0x7f1d60102370 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.446+0000 7f1d655d9640 1 --2- 172.21.15.45:0/3751563814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d601028b0 0x7f1d60109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.446+0000 7f1d655d9640 1 --2- 172.21.15.45:0/3751563814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d60100d00 0x7f1d60101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.446+0000 7f1d655d9640 1 -- 172.21.15.45:0/3751563814 >> 172.21.15.45:0/3751563814 conn(0x7f1d600fc290 msgr2=0x7f1d600fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.447+0000 7f1d655d9640 1 -- 172.21.15.45:0/3751563814 shutdown_connections 2024-01-31T05:20:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.447+0000 7f1d655d9640 1 -- 172.21.15.45:0/3751563814 wait complete. 2024-01-31T05:20:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.447+0000 7f1d655d9640 1 Processor -- start 2024-01-31T05:20:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.447+0000 7f1d655d9640 1 -- start start 2024-01-31T05:20:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.447+0000 7f1d655d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d60100d00 0x7f1d60074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.447+0000 7f1d655d9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d60101ef0 0x7f1d600730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.447+0000 7f1d655d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d601028b0 0x7f1d60073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.447+0000 7f1d655d9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d60073d80 con 0x7f1d60100d00 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.447+0000 7f1d655d9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d60073ef0 con 0x7f1d60101ef0 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.447+0000 7f1d655d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601a0d90 con 0x7f1d601028b0 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d60100d00 0x7f1d60074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d60100d00 0x7f1d60074aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54766/0 (socket says 172.21.15.45:54766) 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5effd640 1 -- 172.21.15.45:0/2985595876 learned_addr learned my addr 172.21.15.45:0/2985595876 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5e7fc640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d60101ef0 0x7f1d600730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5f7fe640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d601028b0 0x7f1d60073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5effd640 1 -- 172.21.15.45:0/2985595876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d601028b0 msgr2=0x7f1d60073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5effd640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d601028b0 0x7f1d60073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5effd640 1 -- 172.21.15.45:0/2985595876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d60101ef0 msgr2=0x7f1d600730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5effd640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d60101ef0 0x7f1d600730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5effd640 1 -- 172.21.15.45:0/2985595876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d54009560 con 0x7f1d60100d00 2024-01-31T05:20:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5e7fc640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d60101ef0 0x7f1d600730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5f7fe640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d601028b0 0x7f1d60073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:20:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d5effd640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d60100d00 0x7f1d60074aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000c9a0 tx=0x7f1d5000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d47fff640 1 -- 172.21.15.45:0/2985595876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50004220 con 0x7f1d60100d00 2024-01-31T05:20:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d655d9640 1 -- 172.21.15.45:0/2985595876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d601a0fe0 con 0x7f1d60100d00 2024-01-31T05:20:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d47fff640 1 -- 172.21.15.45:0/2985595876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d50007e70 con 0x7f1d60100d00 2024-01-31T05:20:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d47fff640 1 -- 172.21.15.45:0/2985595876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5000f450 con 0x7f1d60100d00 2024-01-31T05:20:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.448+0000 7f1d655d9640 1 -- 172.21.15.45:0/2985595876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d601a14e0 con 0x7f1d60100d00 2024-01-31T05:20:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.449+0000 7f1d655d9640 1 -- 172.21.15.45:0/2985595876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d60108140 con 0x7f1d60100d00 2024-01-31T05:20:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.450+0000 7f1d47fff640 1 -- 172.21.15.45:0/2985595876 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f1d500079b0 con 0x7f1d60100d00 2024-01-31T05:20:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.451+0000 7f1d47fff640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1d480a12e0 0x7f1d480a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.451+0000 7f1d5e7fc640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1d480a12e0 0x7f1d480a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.451+0000 7f1d47fff640 1 -- 172.21.15.45:0/2985595876 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1d500ccb90 con 0x7f1d60100d00 2024-01-31T05:20:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.451+0000 7f1d5e7fc640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1d480a12e0 0x7f1d480a37a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d54006010 tx=0x7f1d5402c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.455+0000 7f1d47fff640 1 -- 172.21.15.45:0/2985595876 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1d500960c0 con 0x7f1d60100d00 2024-01-31T05:20:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.673+0000 7f1d655d9640 1 -- 172.21.15.45:0/2985595876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1d6002cbb0 con 0x7f1d60100d00 2024-01-31T05:20:41.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.708+0000 7f1d47fff640 1 -- 172.21.15.45:0/2985595876 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1d5001d0b0 con 0x7f1d60100d00 2024-01-31T05:20:41.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.711+0000 7f1d655d9640 1 -- 172.21.15.45:0/2985595876 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1d480a12e0 msgr2=0x7f1d480a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.711+0000 7f1d655d9640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1d480a12e0 0x7f1d480a37a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d54006010 tx=0x7f1d5402c5d0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.711+0000 7f1d655d9640 1 -- 172.21.15.45:0/2985595876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d60100d00 msgr2=0x7f1d60074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.711+0000 7f1d655d9640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d60100d00 0x7f1d60074aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000c9a0 tx=0x7f1d5000ce60 comp rx=0 tx=0).stop 2024-01-31T05:20:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.711+0000 7f1d655d9640 1 -- 172.21.15.45:0/2985595876 shutdown_connections 2024-01-31T05:20:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.711+0000 7f1d655d9640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1d480a12e0 0x7f1d480a37a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.711+0000 7f1d655d9640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d601028b0 0x7f1d60073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.711+0000 7f1d655d9640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d60101ef0 0x7f1d600730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.711+0000 7f1d655d9640 1 --2- 172.21.15.45:0/2985595876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d60100d00 0x7f1d60074aa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.711+0000 7f1d655d9640 1 -- 172.21.15.45:0/2985595876 >> 172.21.15.45:0/2985595876 conn(0x7f1d600fc290 msgr2=0x7f1d60105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.711+0000 7f1d655d9640 1 -- 172.21.15.45:0/2985595876 shutdown_connections 2024-01-31T05:20:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.711+0000 7f1d655d9640 1 -- 172.21.15.45:0/2985595876 wait complete. 2024-01-31T05:20:41.721 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-01-31T05:20:41.721 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-01-31T05:20:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.827+0000 7faacaad6640 1 -- 172.21.15.45:0/422528310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac41069d0 msgr2=0x7faac4108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.827+0000 7faacaad6640 1 --2- 172.21.15.45:0/422528310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac41069d0 0x7faac4108e20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faabc009ca0 tx=0x7faabc02c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.827+0000 7faacaad6640 1 -- 172.21.15.45:0/422528310 shutdown_connections 2024-01-31T05:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.827+0000 7faacaad6640 1 --2- 172.21.15.45:0/422528310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac41069d0 0x7faac4108e20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.827+0000 7faacaad6640 1 --2- 172.21.15.45:0/422528310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faac41040a0 0x7faac4106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.827+0000 7faacaad6640 1 --2- 172.21.15.45:0/422528310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faac4101770 0x7faac4103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.827+0000 7faacaad6640 1 -- 172.21.15.45:0/422528310 >> 172.21.15.45:0/422528310 conn(0x7faac40fb180 msgr2=0x7faac40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.827+0000 7faacaad6640 1 -- 172.21.15.45:0/422528310 shutdown_connections 2024-01-31T05:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.827+0000 7faacaad6640 1 -- 172.21.15.45:0/422528310 wait complete. 2024-01-31T05:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faacaad6640 1 Processor -- start 2024-01-31T05:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faacaad6640 1 -- start start 2024-01-31T05:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faacaad6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac4101770 0x7faac4196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faacaad6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faac41040a0 0x7faac41971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faacaad6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faac41069d0 0x7faac419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faacaad6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac419e780 con 0x7faac4101770 2024-01-31T05:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faacaad6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac419e8f0 con 0x7faac41069d0 2024-01-31T05:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faacaad6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac419eab0 con 0x7faac41040a0 2024-01-31T05:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faac8872640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac4101770 0x7faac4196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faac8872640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac4101770 0x7faac4196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54784/0 (socket says 172.21.15.45:54784) 2024-01-31T05:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faac8872640 1 -- 172.21.15.45:0/2653427298 learned_addr learned my addr 172.21.15.45:0/2653427298 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faac3fff640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faac41040a0 0x7faac41971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faac9073640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faac41069d0 0x7faac419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faac8872640 1 -- 172.21.15.45:0/2653427298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faac41040a0 msgr2=0x7faac41971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.828+0000 7faac8872640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faac41040a0 0x7faac41971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.829+0000 7faac8872640 1 -- 172.21.15.45:0/2653427298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faac41069d0 msgr2=0x7faac419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:41.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.829+0000 7faac8872640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faac41069d0 0x7faac419e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:41.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.829+0000 7faac8872640 1 -- 172.21.15.45:0/2653427298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc009900 con 0x7faac4101770 2024-01-31T05:20:41.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.829+0000 7faac8872640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac4101770 0x7faac4196c70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faab400c920 tx=0x7faab400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:41.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.829+0000 7faac1ffb640 1 -- 172.21.15.45:0/2653427298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4007bb0 con 0x7faac4101770 2024-01-31T05:20:41.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.829+0000 7faacaad6640 1 -- 172.21.15.45:0/2653427298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac419ed90 con 0x7faac4101770 2024-01-31T05:20:41.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.829+0000 7faac1ffb640 1 -- 172.21.15.45:0/2653427298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faab4007d10 con 0x7faac4101770 2024-01-31T05:20:41.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.829+0000 7faac1ffb640 1 -- 172.21.15.45:0/2653427298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab40186a0 con 0x7faac4101770 2024-01-31T05:20:41.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.829+0000 7faacaad6640 1 -- 172.21.15.45:0/2653427298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac419f260 con 0x7faac4101770 2024-01-31T05:20:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.830+0000 7faacaad6640 1 -- 172.21.15.45:0/2653427298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac40fcc70 con 0x7faac4101770 2024-01-31T05:20:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.831+0000 7faac1ffb640 1 -- 172.21.15.45:0/2653427298 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7faab400f450 con 0x7faac4101770 2024-01-31T05:20:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.832+0000 7faac1ffb640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faaac0a12e0 0x7faaac0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:41.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.832+0000 7faac3fff640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faaac0a12e0 0x7faaac0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:41.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.832+0000 7faac1ffb640 1 -- 172.21.15.45:0/2653427298 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faab40ccc40 con 0x7faac4101770 2024-01-31T05:20:41.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.832+0000 7faac3fff640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faaac0a12e0 0x7faaac0a37a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faac4198170 tx=0x7faab8009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:41.836+0000 7faac1ffb640 1 -- 172.21.15.45:0/2653427298 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7faab4096170 con 0x7faac4101770 2024-01-31T05:20:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:41 smithi156 ceph-mon[35720]: pgmap v281: 97 pgs: 6 creating+peering, 7 active+clean, 84 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:41 smithi156 ceph-mon[35720]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-01-31T05:20:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:41 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2169164012' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-01-31T05:20:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:41 smithi156 ceph-mon[35720]: osdmap e85: 12 total, 12 up, 12 in 2024-01-31T05:20:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:41 smithi156 ceph-mon[35720]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-01-31T05:20:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:41 smithi156 ceph-mon[35720]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-01-31T05:20:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:41 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/32937965' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:41 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2047522748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:41.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:41 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2985595876' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-01-31T05:20:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.055+0000 7faacaad6640 1 -- 172.21.15.45:0/2653427298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7faac402cb10 con 0x7faac4101770 2024-01-31T05:20:42.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:41 smithi186 ceph-mon[35659]: pgmap v281: 97 pgs: 6 creating+peering, 7 active+clean, 84 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:42.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:41 smithi186 ceph-mon[35659]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-01-31T05:20:42.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:41 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2169164012' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-01-31T05:20:42.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:41 smithi186 ceph-mon[35659]: osdmap e85: 12 total, 12 up, 12 in 2024-01-31T05:20:42.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:41 smithi186 ceph-mon[35659]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-01-31T05:20:42.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:41 smithi186 ceph-mon[35659]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-01-31T05:20:42.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:41 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/32937965' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:42.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:41 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2047522748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:42.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:41 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2985595876' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-01-31T05:20:42.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:41 smithi045 ceph-mon[31585]: pgmap v281: 97 pgs: 6 creating+peering, 7 active+clean, 84 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:20:42.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:41 smithi045 ceph-mon[31585]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-01-31T05:20:42.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:41 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2169164012' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-01-31T05:20:42.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:41 smithi045 ceph-mon[31585]: osdmap e85: 12 total, 12 up, 12 in 2024-01-31T05:20:42.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:41 smithi045 ceph-mon[31585]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-01-31T05:20:42.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:41 smithi045 ceph-mon[31585]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-01-31T05:20:42.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:41 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/32937965' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:42.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:41 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2047522748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:42.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:41 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2985595876' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-01-31T05:20:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.453+0000 7faac1ffb640 1 -- 172.21.15.45:0/2653427298 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) v1 ==== 105+0+0 (secure 0 0 0) 0x7faab4099790 con 0x7faac4101770 2024-01-31T05:20:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.457+0000 7faacaad6640 1 -- 172.21.15.45:0/2653427298 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faaac0a12e0 msgr2=0x7faaac0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.457+0000 7faacaad6640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faaac0a12e0 0x7faaac0a37a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faac4198170 tx=0x7faab8009110 comp rx=0 tx=0).stop 2024-01-31T05:20:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.457+0000 7faacaad6640 1 -- 172.21.15.45:0/2653427298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac4101770 msgr2=0x7faac4196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.457+0000 7faacaad6640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac4101770 0x7faac4196c70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faab400c920 tx=0x7faab400cd20 comp rx=0 tx=0).stop 2024-01-31T05:20:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.457+0000 7faacaad6640 1 -- 172.21.15.45:0/2653427298 shutdown_connections 2024-01-31T05:20:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.457+0000 7faacaad6640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faaac0a12e0 0x7faaac0a37a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:42.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.457+0000 7faacaad6640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faac41069d0 0x7faac419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:42.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.457+0000 7faacaad6640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faac41040a0 0x7faac41971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:42.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.457+0000 7faacaad6640 1 --2- 172.21.15.45:0/2653427298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac4101770 0x7faac4196c70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:42.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.457+0000 7faacaad6640 1 -- 172.21.15.45:0/2653427298 >> 172.21.15.45:0/2653427298 conn(0x7faac40fb180 msgr2=0x7faac41020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:42.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.457+0000 7faacaad6640 1 -- 172.21.15.45:0/2653427298 shutdown_connections 2024-01-31T05:20:42.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.457+0000 7faacaad6640 1 -- 172.21.15.45:0/2653427298 wait complete. 2024-01-31T05:20:42.472 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-01-31T05:20:42.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.582+0000 7fa72006e640 1 -- 172.21.15.45:0/517618019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7181040c0 msgr2=0x7fa7181064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.582+0000 7fa72006e640 1 --2- 172.21.15.45:0/517618019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7181040c0 0x7fa7181064b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa708009900 tx=0x7fa70802bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.582+0000 7fa72006e640 1 -- 172.21.15.45:0/517618019 shutdown_connections 2024-01-31T05:20:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.582+0000 7fa72006e640 1 --2- 172.21.15.45:0/517618019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7181040c0 0x7fa7181064b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.582+0000 7fa72006e640 1 --2- 172.21.15.45:0/517618019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7181069f0 0x7fa718108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.582+0000 7fa72006e640 1 --2- 172.21.15.45:0/517618019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa718101790 0x7fa718103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.582+0000 7fa72006e640 1 -- 172.21.15.45:0/517618019 >> 172.21.15.45:0/517618019 conn(0x7fa7180fb1a0 msgr2=0x7fa7180fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.582+0000 7fa72006e640 1 -- 172.21.15.45:0/517618019 shutdown_connections 2024-01-31T05:20:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.582+0000 7fa72006e640 1 -- 172.21.15.45:0/517618019 wait complete. 2024-01-31T05:20:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa72006e640 1 Processor -- start 2024-01-31T05:20:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa72006e640 1 -- start start 2024-01-31T05:20:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa72006e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa718101790 0x7fa718101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa72006e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7181040c0 0x7fa7180ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa72006e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7181069f0 0x7fa7180ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa72006e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7181003e0 con 0x7fa7181069f0 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa72006e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa718100550 con 0x7fa7181040c0 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa72006e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7181a0d70 con 0x7fa718101790 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa71e60b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7181069f0 0x7fa7180ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa71e60b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7181069f0 0x7fa7180ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54808/0 (socket says 172.21.15.45:54808) 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa71e60b640 1 -- 172.21.15.45:0/2716232908 learned_addr learned my addr 172.21.15.45:0/2716232908 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa71d609640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7181040c0 0x7fa7180ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa71de0a640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa718101790 0x7fa718101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa71e60b640 1 -- 172.21.15.45:0/2716232908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa718101790 msgr2=0x7fa718101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa71e60b640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa718101790 0x7fa718101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa71e60b640 1 -- 172.21.15.45:0/2716232908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7181040c0 msgr2=0x7fa7180ff740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa71e60b640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7181040c0 0x7fa7180ff740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa71e60b640 1 -- 172.21.15.45:0/2716232908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa708009560 con 0x7fa7181069f0 2024-01-31T05:20:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa71d609640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7181040c0 0x7fa7180ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa71e60b640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7181069f0 0x7fa7180ffc80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa71000cd10 tx=0x7fa71000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa70effd640 1 -- 172.21.15.45:0/2716232908 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71000f450 con 0x7fa7181069f0 2024-01-31T05:20:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.583+0000 7fa70effd640 1 -- 172.21.15.45:0/2716232908 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa71000fa70 con 0x7fa7181069f0 2024-01-31T05:20:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.584+0000 7fa72006e640 1 -- 172.21.15.45:0/2716232908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7181a0fc0 con 0x7fa7181069f0 2024-01-31T05:20:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.584+0000 7fa70effd640 1 -- 172.21.15.45:0/2716232908 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710018b00 con 0x7fa7181069f0 2024-01-31T05:20:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.584+0000 7fa72006e640 1 -- 172.21.15.45:0/2716232908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7181a1520 con 0x7fa7181069f0 2024-01-31T05:20:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.585+0000 7fa72006e640 1 -- 172.21.15.45:0/2716232908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7180fcc90 con 0x7fa7181069f0 2024-01-31T05:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.586+0000 7fa70effd640 1 -- 172.21.15.45:0/2716232908 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fa71000fbe0 con 0x7fa7181069f0 2024-01-31T05:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.586+0000 7fa70effd640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa7040a1360 0x7fa7040a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.586+0000 7fa71de0a640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa7040a1360 0x7fa7040a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.586+0000 7fa70effd640 1 -- 172.21.15.45:0/2716232908 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa7100cce80 con 0x7fa7181069f0 2024-01-31T05:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.587+0000 7fa71de0a640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa7040a1360 0x7fa7040a3820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa714009890 tx=0x7fa714008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:42.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.591+0000 7fa70effd640 1 -- 172.21.15.45:0/2716232908 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fa710096450 con 0x7fa7181069f0 2024-01-31T05:20:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:42.810+0000 7fa72006e640 1 -- 172.21.15.45:0/2716232908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fa71810c630 con 0x7fa7181069f0 2024-01-31T05:20:42.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:42 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2985595876' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-01-31T05:20:42.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:42 smithi045 ceph-mon[31585]: osdmap e86: 12 total, 12 up, 12 in 2024-01-31T05:20:42.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:42 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] up:active 2024-01-31T05:20:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:42 smithi045 ceph-mon[31585]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-31T05:20:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:42 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2653427298' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-01-31T05:20:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:42 smithi045 ceph-mon[31585]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-01-31T05:20:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:42 smithi045 ceph-mon[31585]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-01-31T05:20:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:42 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2653427298' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-01-31T05:20:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:42 smithi045 ceph-mon[31585]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-01-31T05:20:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:42 smithi045 ceph-mon[31585]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-01-31T05:20:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:42 smithi045 ceph-mon[31585]: pgmap v284: 97 pgs: 8 creating+activating, 10 creating+peering, 64 active+clean, 15 unknown; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-01-31T05:20:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:42 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2985595876' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-01-31T05:20:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:42 smithi156 ceph-mon[35720]: osdmap e86: 12 total, 12 up, 12 in 2024-01-31T05:20:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:42 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] up:active 2024-01-31T05:20:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:42 smithi156 ceph-mon[35720]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-31T05:20:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:42 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2653427298' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-01-31T05:20:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:42 smithi156 ceph-mon[35720]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-01-31T05:20:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:42 smithi156 ceph-mon[35720]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-01-31T05:20:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:42 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2653427298' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-01-31T05:20:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:42 smithi156 ceph-mon[35720]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-01-31T05:20:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:42 smithi156 ceph-mon[35720]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-01-31T05:20:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:42 smithi156 ceph-mon[35720]: pgmap v284: 97 pgs: 8 creating+activating, 10 creating+peering, 64 active+clean, 15 unknown; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-01-31T05:20:43.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:42 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2985595876' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-01-31T05:20:43.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:42 smithi186 ceph-mon[35659]: osdmap e86: 12 total, 12 up, 12 in 2024-01-31T05:20:43.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:42 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] up:active 2024-01-31T05:20:43.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:42 smithi186 ceph-mon[35659]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-31T05:20:43.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:42 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2653427298' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-01-31T05:20:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:42 smithi186 ceph-mon[35659]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-01-31T05:20:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:42 smithi186 ceph-mon[35659]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-01-31T05:20:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:42 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2653427298' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-01-31T05:20:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:42 smithi186 ceph-mon[35659]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-01-31T05:20:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:42 smithi186 ceph-mon[35659]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-01-31T05:20:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:42 smithi186 ceph-mon[35659]: pgmap v284: 97 pgs: 8 creating+activating, 10 creating+peering, 64 active+clean, 15 unknown; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-01-31T05:20:43.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.465+0000 7fa70effd640 1 -- 172.21.15.45:0/2716232908 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v17) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa710099a70 con 0x7fa7181069f0 2024-01-31T05:20:43.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.468+0000 7fa72006e640 1 -- 172.21.15.45:0/2716232908 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa7040a1360 msgr2=0x7fa7040a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:43.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.468+0000 7fa72006e640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa7040a1360 0x7fa7040a3820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa714009890 tx=0x7fa714008040 comp rx=0 tx=0).stop 2024-01-31T05:20:43.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.468+0000 7fa72006e640 1 -- 172.21.15.45:0/2716232908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7181069f0 msgr2=0x7fa7180ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:43.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.468+0000 7fa72006e640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7181069f0 0x7fa7180ffc80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa71000cd10 tx=0x7fa71000a620 comp rx=0 tx=0).stop 2024-01-31T05:20:43.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.468+0000 7fa72006e640 1 -- 172.21.15.45:0/2716232908 shutdown_connections 2024-01-31T05:20:43.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.468+0000 7fa72006e640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa7040a1360 0x7fa7040a3820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:43.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.468+0000 7fa72006e640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7181069f0 0x7fa7180ffc80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:43.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.468+0000 7fa72006e640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7181040c0 0x7fa7180ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:43.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.468+0000 7fa72006e640 1 --2- 172.21.15.45:0/2716232908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa718101790 0x7fa718101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:43.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.468+0000 7fa72006e640 1 -- 172.21.15.45:0/2716232908 >> 172.21.15.45:0/2716232908 conn(0x7fa7180fb1a0 msgr2=0x7fa7181020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:43.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.468+0000 7fa72006e640 1 -- 172.21.15.45:0/2716232908 shutdown_connections 2024-01-31T05:20:43.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.468+0000 7fa72006e640 1 -- 172.21.15.45:0/2716232908 wait complete. 2024-01-31T05:20:43.483 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-01-31T05:20:43.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.595+0000 7f44ca099640 1 -- 172.21.15.45:0/361864578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c40ffc00 msgr2=0x7f44c4100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:43.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.595+0000 7f44ca099640 1 --2- 172.21.15.45:0/361864578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c40ffc00 0x7f44c4100020 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009900 tx=0x7f44b402bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:43.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.595+0000 7f44ca099640 1 -- 172.21.15.45:0/361864578 shutdown_connections 2024-01-31T05:20:43.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.595+0000 7f44ca099640 1 --2- 172.21.15.45:0/361864578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c41017b0 0x7f44c4108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:43.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.595+0000 7f44ca099640 1 --2- 172.21.15.45:0/361864578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44c4100df0 0x7f44c4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:43.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.595+0000 7f44ca099640 1 --2- 172.21.15.45:0/361864578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c40ffc00 0x7f44c4100020 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:43.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.595+0000 7f44ca099640 1 -- 172.21.15.45:0/361864578 >> 172.21.15.45:0/361864578 conn(0x7f44c40fb190 msgr2=0x7f44c40fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:43.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.595+0000 7f44ca099640 1 -- 172.21.15.45:0/361864578 shutdown_connections 2024-01-31T05:20:43.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.595+0000 7f44ca099640 1 -- 172.21.15.45:0/361864578 wait complete. 2024-01-31T05:20:43.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.595+0000 7f44ca099640 1 Processor -- start 2024-01-31T05:20:43.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44ca099640 1 -- start start 2024-01-31T05:20:43.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44ca099640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c40ffc00 0x7f44c4196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:43.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44ca099640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44c4100df0 0x7f44c41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:43.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44ca099640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c41017b0 0x7f44c419e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:43.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44ca099640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c419e7c0 con 0x7f44c41017b0 2024-01-31T05:20:43.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44ca099640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c419e930 con 0x7f44c4100df0 2024-01-31T05:20:43.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44ca099640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c419eaf0 con 0x7f44c40ffc00 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44c3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c41017b0 0x7f44c419e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44c3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c41017b0 0x7f44c419e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54834/0 (socket says 172.21.15.45:54834) 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44c3fff640 1 -- 172.21.15.45:0/171023682 learned_addr learned my addr 172.21.15.45:0/171023682 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44c37fe640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c40ffc00 0x7f44c4196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44c2ffd640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44c4100df0 0x7f44c41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44c3fff640 1 -- 172.21.15.45:0/171023682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c40ffc00 msgr2=0x7f44c4196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44c3fff640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c40ffc00 0x7f44c4196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44c3fff640 1 -- 172.21.15.45:0/171023682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44c4100df0 msgr2=0x7f44c41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44c3fff640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44c4100df0 0x7f44c41971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44c3fff640 1 -- 172.21.15.45:0/171023682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b4009560 con 0x7f44c41017b0 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.596+0000 7f44c3fff640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c41017b0 0x7f44c419e250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00b970 tx=0x7f44bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.597+0000 7f44c0ff9640 1 -- 172.21.15.45:0/171023682 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00cb40 con 0x7f44c41017b0 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.597+0000 7f44c0ff9640 1 -- 172.21.15.45:0/171023682 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44bc00cca0 con 0x7f44c41017b0 2024-01-31T05:20:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.597+0000 7f44ca099640 1 -- 172.21.15.45:0/171023682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c419edd0 con 0x7f44c41017b0 2024-01-31T05:20:43.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.597+0000 7f44c0ff9640 1 -- 172.21.15.45:0/171023682 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc007770 con 0x7f44c41017b0 2024-01-31T05:20:43.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.597+0000 7f44ca099640 1 -- 172.21.15.45:0/171023682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c419f2d0 con 0x7f44c41017b0 2024-01-31T05:20:43.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.598+0000 7f44ca099640 1 -- 172.21.15.45:0/171023682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c4106f90 con 0x7f44c41017b0 2024-01-31T05:20:43.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.599+0000 7f44c0ff9640 1 -- 172.21.15.45:0/171023682 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f44bc014440 con 0x7f44c41017b0 2024-01-31T05:20:43.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.600+0000 7f44c0ff9640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f44ac0a12e0 0x7f44ac0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:43.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.600+0000 7f44c37fe640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f44ac0a12e0 0x7f44ac0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:43.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.600+0000 7f44c0ff9640 1 -- 172.21.15.45:0/171023682 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f44bc0cc6e0 con 0x7f44c41017b0 2024-01-31T05:20:43.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.601+0000 7f44c37fe640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f44ac0a12e0 0x7f44ac0a37a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f44b4000c00 tx=0x7f44b402c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:43.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.604+0000 7f44c0ff9640 1 -- 172.21.15.45:0/171023682 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f44bc095d60 con 0x7f44c41017b0 2024-01-31T05:20:43.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:43.824+0000 7f44ca099640 1 -- 172.21.15.45:0/171023682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f44c410c810 con 0x7f44c41017b0 2024-01-31T05:20:43.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:43 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2716232908' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-01-31T05:20:43.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:43 smithi045 ceph-mon[31585]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-01-31T05:20:43.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:43 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2716232908' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-01-31T05:20:43.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:43 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] up:active 2024-01-31T05:20:43.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:43 smithi045 ceph-mon[31585]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-01-31T05:20:43.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:43 smithi045 ceph-mon[31585]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-01-31T05:20:43.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:43 smithi045 ceph-mon[31585]: Cluster is now healthy 2024-01-31T05:20:43.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:43 smithi045 ceph-mon[31585]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-01-31T05:20:43.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:43 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-01-31T05:20:43.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:43 smithi045 ceph-mon[31585]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-01-31T05:20:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:43 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2716232908' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-01-31T05:20:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:43 smithi156 ceph-mon[35720]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-01-31T05:20:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:43 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2716232908' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-01-31T05:20:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:43 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] up:active 2024-01-31T05:20:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:43 smithi156 ceph-mon[35720]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-01-31T05:20:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:43 smithi156 ceph-mon[35720]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-01-31T05:20:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:43 smithi156 ceph-mon[35720]: Cluster is now healthy 2024-01-31T05:20:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:43 smithi156 ceph-mon[35720]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-01-31T05:20:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:43 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-01-31T05:20:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:43 smithi156 ceph-mon[35720]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-01-31T05:20:44.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:43 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2716232908' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-01-31T05:20:44.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:43 smithi186 ceph-mon[35659]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-01-31T05:20:44.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:43 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2716232908' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-01-31T05:20:44.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:43 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] up:active 2024-01-31T05:20:44.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:43 smithi186 ceph-mon[35659]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-01-31T05:20:44.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:43 smithi186 ceph-mon[35659]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-01-31T05:20:44.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:43 smithi186 ceph-mon[35659]: Cluster is now healthy 2024-01-31T05:20:44.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:43 smithi186 ceph-mon[35659]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-01-31T05:20:44.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:43 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-01-31T05:20:44.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:43 smithi186 ceph-mon[35659]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-01-31T05:20:44.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.471+0000 7f44c0ff9640 1 -- 172.21.15.45:0/171023682 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v19) v1 ==== 115+0+0 (secure 0 0 0) 0x7f44bc099380 con 0x7f44c41017b0 2024-01-31T05:20:44.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.474+0000 7f44ca099640 1 -- 172.21.15.45:0/171023682 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f44ac0a12e0 msgr2=0x7f44ac0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:44.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.474+0000 7f44ca099640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f44ac0a12e0 0x7f44ac0a37a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f44b4000c00 tx=0x7f44b402c5d0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.474+0000 7f44ca099640 1 -- 172.21.15.45:0/171023682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c41017b0 msgr2=0x7f44c419e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:44.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.474+0000 7f44ca099640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c41017b0 0x7f44c419e250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00b970 tx=0x7f44bc00be30 comp rx=0 tx=0).stop 2024-01-31T05:20:44.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.474+0000 7f44ca099640 1 -- 172.21.15.45:0/171023682 shutdown_connections 2024-01-31T05:20:44.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.474+0000 7f44ca099640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f44ac0a12e0 0x7f44ac0a37a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.474+0000 7f44ca099640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c41017b0 0x7f44c419e250 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.474+0000 7f44ca099640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44c4100df0 0x7f44c41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.474+0000 7f44ca099640 1 --2- 172.21.15.45:0/171023682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44c40ffc00 0x7f44c4196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.474+0000 7f44ca099640 1 -- 172.21.15.45:0/171023682 >> 172.21.15.45:0/171023682 conn(0x7f44c40fb190 msgr2=0x7f44c4104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:44.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.474+0000 7f44ca099640 1 -- 172.21.15.45:0/171023682 shutdown_connections 2024-01-31T05:20:44.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.474+0000 7f44ca099640 1 -- 172.21.15.45:0/171023682 wait complete. 2024-01-31T05:20:44.488 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-01-31T05:20:44.488 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-01-31T05:20:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.600+0000 7fa5586d6640 1 -- 172.21.15.45:0/3070092407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa550100d00 msgr2=0x7fa550101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.600+0000 7fa5586d6640 1 --2- 172.21.15.45:0/3070092407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa550100d00 0x7fa550101120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009900 tx=0x7fa54c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.600+0000 7fa5586d6640 1 -- 172.21.15.45:0/3070092407 shutdown_connections 2024-01-31T05:20:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.600+0000 7fa5586d6640 1 --2- 172.21.15.45:0/3070092407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5501028b0 0x7fa550109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.600+0000 7fa5586d6640 1 --2- 172.21.15.45:0/3070092407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa550101ef0 0x7fa550102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.600+0000 7fa5586d6640 1 --2- 172.21.15.45:0/3070092407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa550100d00 0x7fa550101120 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.600+0000 7fa5586d6640 1 -- 172.21.15.45:0/3070092407 >> 172.21.15.45:0/3070092407 conn(0x7fa5500fc290 msgr2=0x7fa5500fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.600+0000 7fa5586d6640 1 -- 172.21.15.45:0/3070092407 shutdown_connections 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.600+0000 7fa5586d6640 1 -- 172.21.15.45:0/3070092407 wait complete. 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa5586d6640 1 Processor -- start 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa5586d6640 1 -- start start 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa5586d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa550100d00 0x7fa550196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa5586d6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa550101ef0 0x7fa550197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa5586d6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5501028b0 0x7fa55019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa5586d6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55019e940 con 0x7fa550100d00 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa5586d6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55019eab0 con 0x7fa550101ef0 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa5586d6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55019ec70 con 0x7fa5501028b0 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa556472640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa550100d00 0x7fa550196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa556472640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa550100d00 0x7fa550196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54862/0 (socket says 172.21.15.45:54862) 2024-01-31T05:20:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa556472640 1 -- 172.21.15.45:0/2180402761 learned_addr learned my addr 172.21.15.45:0/2180402761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa555c71640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa550101ef0 0x7fa550197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa556c73640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5501028b0 0x7fa55019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa556472640 1 -- 172.21.15.45:0/2180402761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5501028b0 msgr2=0x7fa55019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa556472640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5501028b0 0x7fa55019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa556472640 1 -- 172.21.15.45:0/2180402761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa550101ef0 msgr2=0x7fa550197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa556472640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa550101ef0 0x7fa550197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa556472640 1 -- 172.21.15.45:0/2180402761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c009560 con 0x7fa550100d00 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa555c71640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa550101ef0 0x7fa550197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa556c73640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5501028b0 0x7fa55019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.601+0000 7fa556472640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa550100d00 0x7fa550196e50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009230 tx=0x7fa54c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.602+0000 7fa5477fe640 1 -- 172.21.15.45:0/2180402761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c0044c0 con 0x7fa550100d00 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.602+0000 7fa5477fe640 1 -- 172.21.15.45:0/2180402761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa54c0054c0 con 0x7fa550100d00 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.602+0000 7fa5586d6640 1 -- 172.21.15.45:0/2180402761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55019eef0 con 0x7fa550100d00 2024-01-31T05:20:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.602+0000 7fa5477fe640 1 -- 172.21.15.45:0/2180402761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c03e450 con 0x7fa550100d00 2024-01-31T05:20:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.602+0000 7fa5586d6640 1 -- 172.21.15.45:0/2180402761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55019f3f0 con 0x7fa550100d00 2024-01-31T05:20:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.603+0000 7fa5586d6640 1 -- 172.21.15.45:0/2180402761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5501081b0 con 0x7fa550100d00 2024-01-31T05:20:44.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.604+0000 7fa5477fe640 1 -- 172.21.15.45:0/2180402761 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fa54c0036f0 con 0x7fa550100d00 2024-01-31T05:20:44.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.604+0000 7fa5477fe640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa53c0a1360 0x7fa53c0a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:44.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.604+0000 7fa555c71640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa53c0a1360 0x7fa53c0a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:44.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.604+0000 7fa5477fe640 1 -- 172.21.15.45:0/2180402761 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa54c0ef8d0 con 0x7fa550100d00 2024-01-31T05:20:44.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.605+0000 7fa555c71640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa53c0a1360 0x7fa53c0a3820 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa540006fd0 tx=0x7fa540008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.608+0000 7fa5477fe640 1 -- 172.21.15.45:0/2180402761 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fa54c0b8ea0 con 0x7fa550100d00 2024-01-31T05:20:44.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.814+0000 7fa5586d6640 1 -- 172.21.15.45:0/2180402761 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fa55002cb80 con 0x7fa53c0a1360 2024-01-31T05:20:44.819 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:44 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/171023682' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-01-31T05:20:44.819 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:44 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/171023682' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-01-31T05:20:44.819 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:44 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] up:active 2024-01-31T05:20:44.819 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:44 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-01-31T05:20:44.819 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:44 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:20:44.819 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:44 smithi045 ceph-mon[31585]: pgmap v285: 97 pgs: 8 creating+activating, 10 creating+peering, 79 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-01-31T05:20:44.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.834+0000 7fa5477fe640 1 -- 172.21.15.45:0/2180402761 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa55002cb80 con 0x7fa53c0a1360 2024-01-31T05:20:44.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.836+0000 7fa5586d6640 1 -- 172.21.15.45:0/2180402761 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa53c0a1360 msgr2=0x7fa53c0a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:44.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.836+0000 7fa5586d6640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa53c0a1360 0x7fa53c0a3820 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa540006fd0 tx=0x7fa540008040 comp rx=0 tx=0).stop 2024-01-31T05:20:44.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.836+0000 7fa5586d6640 1 -- 172.21.15.45:0/2180402761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa550100d00 msgr2=0x7fa550196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:44.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.836+0000 7fa5586d6640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa550100d00 0x7fa550196e50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009230 tx=0x7fa54c0040c0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.836+0000 7fa5586d6640 1 -- 172.21.15.45:0/2180402761 shutdown_connections 2024-01-31T05:20:44.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.836+0000 7fa5586d6640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa53c0a1360 0x7fa53c0a3820 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.836+0000 7fa5586d6640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5501028b0 0x7fa55019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.836+0000 7fa5586d6640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa550101ef0 0x7fa550197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.836+0000 7fa5586d6640 1 --2- 172.21.15.45:0/2180402761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa550100d00 0x7fa550196e50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.836+0000 7fa5586d6640 1 -- 172.21.15.45:0/2180402761 >> 172.21.15.45:0/2180402761 conn(0x7fa5500fc290 msgr2=0x7fa550105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:44.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.836+0000 7fa5586d6640 1 -- 172.21.15.45:0/2180402761 shutdown_connections 2024-01-31T05:20:44.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.836+0000 7fa5586d6640 1 -- 172.21.15.45:0/2180402761 wait complete. 2024-01-31T05:20:44.846 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-01-31T05:20:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.954+0000 7f0c2ae78640 1 -- 172.21.15.45:0/4087606079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c241017b0 msgr2=0x7f0c24108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.954+0000 7f0c2ae78640 1 --2- 172.21.15.45:0/4087606079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c241017b0 0x7f0c24108040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009900 tx=0x7f0c1402bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.954+0000 7f0c2ae78640 1 -- 172.21.15.45:0/4087606079 shutdown_connections 2024-01-31T05:20:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.954+0000 7f0c2ae78640 1 --2- 172.21.15.45:0/4087606079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c241017b0 0x7f0c24108040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.954+0000 7f0c2ae78640 1 --2- 172.21.15.45:0/4087606079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c24100df0 0x7f0c24101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.954+0000 7f0c2ae78640 1 --2- 172.21.15.45:0/4087606079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c240ffc00 0x7f0c24100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.954+0000 7f0c2ae78640 1 -- 172.21.15.45:0/4087606079 >> 172.21.15.45:0/4087606079 conn(0x7f0c240fb190 msgr2=0x7f0c240fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.954+0000 7f0c2ae78640 1 -- 172.21.15.45:0/4087606079 shutdown_connections 2024-01-31T05:20:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.954+0000 7f0c2ae78640 1 -- 172.21.15.45:0/4087606079 wait complete. 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c2ae78640 1 Processor -- start 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c2ae78640 1 -- start start 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c2ae78640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c240ffc00 0x7f0c24195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c2ae78640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c24100df0 0x7f0c24196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c2ae78640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c241017b0 0x7f0c2419d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c2ae78640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2419d710 con 0x7f0c241017b0 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c2ae78640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2419d880 con 0x7f0c24100df0 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c2ae78640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2419da40 con 0x7f0c240ffc00 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c29415640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c241017b0 0x7f0c2419d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c29415640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c241017b0 0x7f0c2419d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54902/0 (socket says 172.21.15.45:54902) 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c29415640 1 -- 172.21.15.45:0/2709925897 learned_addr learned my addr 172.21.15.45:0/2709925897 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c28c14640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c240ffc00 0x7f0c24195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c23fff640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c24100df0 0x7f0c24196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c29415640 1 -- 172.21.15.45:0/2709925897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c240ffc00 msgr2=0x7f0c24195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c29415640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c240ffc00 0x7f0c24195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c29415640 1 -- 172.21.15.45:0/2709925897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c24100df0 msgr2=0x7f0c24196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c29415640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c24100df0 0x7f0c24196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c29415640 1 -- 172.21.15.45:0/2709925897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14009560 con 0x7f0c241017b0 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c28c14640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c240ffc00 0x7f0c24195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.955+0000 7f0c23fff640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c24100df0 0x7f0c24196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.956+0000 7f0c29415640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c241017b0 0x7f0c2419d1a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c1402c7e0 tx=0x7f0c14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.956+0000 7f0c21ffb640 1 -- 172.21.15.45:0/2709925897 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14005000 con 0x7f0c241017b0 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.956+0000 7f0c21ffb640 1 -- 172.21.15.45:0/2709925897 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c14005620 con 0x7f0c241017b0 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.956+0000 7f0c2ae78640 1 -- 172.21.15.45:0/2709925897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2419dcc0 con 0x7f0c241017b0 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.956+0000 7f0c21ffb640 1 -- 172.21.15.45:0/2709925897 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1403e6b0 con 0x7f0c241017b0 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.956+0000 7f0c2ae78640 1 -- 172.21.15.45:0/2709925897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2419e1d0 con 0x7f0c241017b0 2024-01-31T05:20:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.957+0000 7f0c2ae78640 1 -- 172.21.15.45:0/2709925897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c24107040 con 0x7f0c241017b0 2024-01-31T05:20:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.958+0000 7f0c21ffb640 1 -- 172.21.15.45:0/2709925897 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f0c14005160 con 0x7f0c241017b0 2024-01-31T05:20:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.958+0000 7f0c21ffb640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0c0c0a1360 0x7f0c0c0a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.959+0000 7f0c21ffb640 1 -- 172.21.15.45:0/2709925897 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0c140efd30 con 0x7f0c241017b0 2024-01-31T05:20:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.959+0000 7f0c28c14640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0c0c0a1360 0x7f0c0c0a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.959+0000 7f0c28c14640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0c0c0a1360 0x7f0c0c0a3820 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0c24197100 tx=0x7f0c1c0073e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:44.962+0000 7f0c21ffb640 1 -- 172.21.15.45:0/2709925897 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f0c140b91d0 con 0x7f0c241017b0 2024-01-31T05:20:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:44 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/171023682' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-01-31T05:20:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:44 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/171023682' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-01-31T05:20:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:44 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] up:active 2024-01-31T05:20:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:44 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-01-31T05:20:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:44 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:20:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:44 smithi156 ceph-mon[35720]: pgmap v285: 97 pgs: 8 creating+activating, 10 creating+peering, 79 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-01-31T05:20:45.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:44 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/171023682' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-01-31T05:20:45.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:44 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/171023682' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-01-31T05:20:45.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:44 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] up:active 2024-01-31T05:20:45.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:44 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-01-31T05:20:45.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:44 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:20:45.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:44 smithi186 ceph-mon[35659]: pgmap v285: 97 pgs: 8 creating+activating, 10 creating+peering, 79 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-01-31T05:20:45.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:20:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:20:44.818+0000 7f4ecee5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:20:45.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:20:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:20:44.819+0000 7f4ecee5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:20:45.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:20:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:20:44.819+0000 7f4ecee5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:20:45.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:20:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:20:44.819+0000 7f4ecee5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:20:45.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:20:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:20:44.819+0000 7f4ecee5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:20:45.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.168+0000 7f0c2ae78640 1 -- 172.21.15.45:0/2709925897 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c2402cc20 con 0x7f0c0c0a1360 2024-01-31T05:20:45.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.192+0000 7f0c21ffb640 1 -- 172.21.15.45:0/2709925897 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0c2402cc20 con 0x7f0c0c0a1360 2024-01-31T05:20:45.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.194+0000 7f0c2ae78640 1 -- 172.21.15.45:0/2709925897 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0c0c0a1360 msgr2=0x7f0c0c0a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:45.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.194+0000 7f0c2ae78640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0c0c0a1360 0x7f0c0c0a3820 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0c24197100 tx=0x7f0c1c0073e0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.194+0000 7f0c2ae78640 1 -- 172.21.15.45:0/2709925897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c241017b0 msgr2=0x7f0c2419d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:45.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.194+0000 7f0c2ae78640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c241017b0 0x7f0c2419d1a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c1402c7e0 tx=0x7f0c14004270 comp rx=0 tx=0).stop 2024-01-31T05:20:45.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.194+0000 7f0c2ae78640 1 -- 172.21.15.45:0/2709925897 shutdown_connections 2024-01-31T05:20:45.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.194+0000 7f0c2ae78640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0c0c0a1360 0x7f0c0c0a3820 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.194+0000 7f0c2ae78640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c241017b0 0x7f0c2419d1a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.194+0000 7f0c2ae78640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c24100df0 0x7f0c24196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.194+0000 7f0c2ae78640 1 --2- 172.21.15.45:0/2709925897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c240ffc00 0x7f0c24195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.194+0000 7f0c2ae78640 1 -- 172.21.15.45:0/2709925897 >> 172.21.15.45:0/2709925897 conn(0x7f0c240fb190 msgr2=0x7f0c24104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:45.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.194+0000 7f0c2ae78640 1 -- 172.21.15.45:0/2709925897 shutdown_connections 2024-01-31T05:20:45.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.194+0000 7f0c2ae78640 1 -- 172.21.15.45:0/2709925897 wait complete. 2024-01-31T05:20:45.203 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-01-31T05:20:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.312+0000 7f9548fd9640 1 -- 172.21.15.45:0/2997010998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9544101fb0 msgr2=0x7f9544102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.312+0000 7f9548fd9640 1 --2- 172.21.15.45:0/2997010998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9544101fb0 0x7f9544102430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f952c009900 tx=0x7f952c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.312+0000 7f9548fd9640 1 -- 172.21.15.45:0/2997010998 shutdown_connections 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.312+0000 7f9548fd9640 1 --2- 172.21.15.45:0/2997010998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9544101fb0 0x7f9544102430 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.312+0000 7f9548fd9640 1 --2- 172.21.15.45:0/2997010998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9544102970 0x7f9544109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.312+0000 7f9548fd9640 1 --2- 172.21.15.45:0/2997010998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95440fefe0 0x7f95440ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.312+0000 7f9548fd9640 1 -- 172.21.15.45:0/2997010998 >> 172.21.15.45:0/2997010998 conn(0x7f95440fa8f0 msgr2=0x7f95440fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.312+0000 7f9548fd9640 1 -- 172.21.15.45:0/2997010998 shutdown_connections 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.312+0000 7f9548fd9640 1 -- 172.21.15.45:0/2997010998 wait complete. 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9548fd9640 1 Processor -- start 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9548fd9640 1 -- start start 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9548fd9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95440fefe0 0x7f9544196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9548fd9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9544101fb0 0x7f9544197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9548fd9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9544102970 0x7f954419e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9548fd9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954419e910 con 0x7f9544102970 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9548fd9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954419ea80 con 0x7f95440fefe0 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9548fd9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954419ec40 con 0x7f9544101fb0 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9542d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9544102970 0x7f954419e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9542d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9544102970 0x7f954419e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54922/0 (socket says 172.21.15.45:54922) 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9542d9d640 1 -- 172.21.15.45:0/402381695 learned_addr learned my addr 172.21.15.45:0/402381695 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f953bfff640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9544101fb0 0x7f9544197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f954259c640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95440fefe0 0x7f9544196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9542d9d640 1 -- 172.21.15.45:0/402381695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9544101fb0 msgr2=0x7f9544197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9542d9d640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9544101fb0 0x7f9544197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9542d9d640 1 -- 172.21.15.45:0/402381695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95440fefe0 msgr2=0x7f9544196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9542d9d640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95440fefe0 0x7f9544196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f9542d9d640 1 -- 172.21.15.45:0/402381695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c009560 con 0x7f9544102970 2024-01-31T05:20:45.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.313+0000 7f954259c640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95440fefe0 0x7f9544196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:20:45.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.314+0000 7f9542d9d640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9544102970 0x7f954419e3d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f953c00b970 tx=0x7f953c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:45.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.314+0000 7f953b7fe640 1 -- 172.21.15.45:0/402381695 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00cb40 con 0x7f9544102970 2024-01-31T05:20:45.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.314+0000 7f953b7fe640 1 -- 172.21.15.45:0/402381695 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f953c00cca0 con 0x7f9544102970 2024-01-31T05:20:45.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.314+0000 7f9548fd9640 1 -- 172.21.15.45:0/402381695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954419ef20 con 0x7f9544102970 2024-01-31T05:20:45.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.314+0000 7f953b7fe640 1 -- 172.21.15.45:0/402381695 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c007770 con 0x7f9544102970 2024-01-31T05:20:45.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.314+0000 7f9548fd9640 1 -- 172.21.15.45:0/402381695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954419f3d0 con 0x7f9544102970 2024-01-31T05:20:45.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.315+0000 7f9548fd9640 1 -- 172.21.15.45:0/402381695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9544108200 con 0x7f9544102970 2024-01-31T05:20:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.316+0000 7f953b7fe640 1 -- 172.21.15.45:0/402381695 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f953c014440 con 0x7f9544102970 2024-01-31T05:20:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.316+0000 7f953b7fe640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f95300a12e0 0x7f95300a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.316+0000 7f953b7fe640 1 -- 172.21.15.45:0/402381695 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f953c0cc6e0 con 0x7f9544102970 2024-01-31T05:20:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.316+0000 7f954259c640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f95300a12e0 0x7f95300a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.317+0000 7f954259c640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f95300a12e0 0x7f95300a37a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9534006fd0 tx=0x7f9534008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.320+0000 7f953b7fe640 1 -- 172.21.15.45:0/402381695 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f953c095d60 con 0x7f9544102970 2024-01-31T05:20:45.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.528+0000 7f9548fd9640 1 -- 172.21.15.45:0/402381695 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f954402cbb0 con 0x7f95300a12e0 2024-01-31T05:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.532+0000 7f953b7fe640 1 -- 172.21.15.45:0/402381695 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f954402cbb0 con 0x7f95300a12e0 2024-01-31T05:20:45.532 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_0/48196921-553b-4aec-be9a-4131d125ebc2 2024-01-31T05:20:45.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.535+0000 7f9548fd9640 1 -- 172.21.15.45:0/402381695 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f95300a12e0 msgr2=0x7f95300a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:45.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.535+0000 7f9548fd9640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f95300a12e0 0x7f95300a37a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9534006fd0 tx=0x7f9534008040 comp rx=0 tx=0).stop 2024-01-31T05:20:45.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.535+0000 7f9548fd9640 1 -- 172.21.15.45:0/402381695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9544102970 msgr2=0x7f954419e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:45.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.535+0000 7f9548fd9640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9544102970 0x7f954419e3d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f953c00b970 tx=0x7f953c00be30 comp rx=0 tx=0).stop 2024-01-31T05:20:45.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.535+0000 7f9548fd9640 1 -- 172.21.15.45:0/402381695 shutdown_connections 2024-01-31T05:20:45.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.535+0000 7f9548fd9640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f95300a12e0 0x7f95300a37a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.535+0000 7f9548fd9640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9544102970 0x7f954419e3d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.535+0000 7f9548fd9640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9544101fb0 0x7f9544197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.535+0000 7f9548fd9640 1 --2- 172.21.15.45:0/402381695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95440fefe0 0x7f9544196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.535+0000 7f9548fd9640 1 -- 172.21.15.45:0/402381695 >> 172.21.15.45:0/402381695 conn(0x7f95440fa8f0 msgr2=0x7f95441053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:45.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.535+0000 7f9548fd9640 1 -- 172.21.15.45:0/402381695 shutdown_connections 2024-01-31T05:20:45.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.535+0000 7f9548fd9640 1 -- 172.21.15.45:0/402381695 wait complete. 2024-01-31T05:20:45.545 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-01-31T05:20:45.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.652+0000 7f3df8de1640 1 -- 172.21.15.45:0/2246089601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4100de0 msgr2=0x7f3df4101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:45.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.652+0000 7f3df8de1640 1 --2- 172.21.15.45:0/2246089601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4100de0 0x7f3df4101260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009900 tx=0x7f3de802bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:45.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.652+0000 7f3df8de1640 1 -- 172.21.15.45:0/2246089601 shutdown_connections 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.652+0000 7f3df8de1640 1 --2- 172.21.15.45:0/2246089601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4100de0 0x7f3df4101260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.652+0000 7f3df8de1640 1 --2- 172.21.15.45:0/2246089601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df41017a0 0x7f3df4108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.652+0000 7f3df8de1640 1 --2- 172.21.15.45:0/2246089601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df40ffbf0 0x7f3df4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.652+0000 7f3df8de1640 1 -- 172.21.15.45:0/2246089601 >> 172.21.15.45:0/2246089601 conn(0x7f3df40fb180 msgr2=0x7f3df40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.653+0000 7f3df8de1640 1 -- 172.21.15.45:0/2246089601 shutdown_connections 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.653+0000 7f3df8de1640 1 -- 172.21.15.45:0/2246089601 wait complete. 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.653+0000 7f3df8de1640 1 Processor -- start 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.653+0000 7f3df8de1640 1 -- start start 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.653+0000 7f3df8de1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df40ffbf0 0x7f3df4103b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.653+0000 7f3df8de1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4100de0 0x7f3df41021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.653+0000 7f3df8de1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df41017a0 0x7f3df4102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.653+0000 7f3df8de1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df4102e60 con 0x7f3df4100de0 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.653+0000 7f3df8de1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df4102fd0 con 0x7f3df40ffbf0 2024-01-31T05:20:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.653+0000 7f3df8de1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41a0d20 con 0x7f3df41017a0 2024-01-31T05:20:45.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4100de0 0x7f3df41021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:45.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4100de0 0x7f3df41021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54934/0 (socket says 172.21.15.45:54934) 2024-01-31T05:20:45.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df1d9b640 1 -- 172.21.15.45:0/662367269 learned_addr learned my addr 172.21.15.45:0/662367269 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:45.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df2d9d640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df41017a0 0x7f3df4102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:45.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df259c640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df40ffbf0 0x7f3df4103b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:45.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df1d9b640 1 -- 172.21.15.45:0/662367269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df41017a0 msgr2=0x7f3df4102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:45.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df1d9b640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df41017a0 0x7f3df4102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df1d9b640 1 -- 172.21.15.45:0/662367269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df40ffbf0 msgr2=0x7f3df4103b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df1d9b640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df40ffbf0 0x7f3df4103b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df1d9b640 1 -- 172.21.15.45:0/662367269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8009560 con 0x7f3df4100de0 2024-01-31T05:20:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df259c640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df40ffbf0 0x7f3df4103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:20:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df2d9d640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df41017a0 0x7f3df4102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df1d9b640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4100de0 0x7f3df41021c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009430 tx=0x7f3de8004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3de37fe640 1 -- 172.21.15.45:0/662367269 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de80043d0 con 0x7f3df4100de0 2024-01-31T05:20:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3de37fe640 1 -- 172.21.15.45:0/662367269 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de80054c0 con 0x7f3df4100de0 2024-01-31T05:20:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df8de1640 1 -- 172.21.15.45:0/662367269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df41a0f70 con 0x7f3df4100de0 2024-01-31T05:20:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3de37fe640 1 -- 172.21.15.45:0/662367269 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8003e80 con 0x7f3df4100de0 2024-01-31T05:20:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.654+0000 7f3df8de1640 1 -- 172.21.15.45:0/662367269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df41a13f0 con 0x7f3df4100de0 2024-01-31T05:20:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.655+0000 7f3df8de1640 1 -- 172.21.15.45:0/662367269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df410b4b0 con 0x7f3df4100de0 2024-01-31T05:20:45.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.657+0000 7f3de37fe640 1 -- 172.21.15.45:0/662367269 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f3de80035f0 con 0x7f3df4100de0 2024-01-31T05:20:45.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.657+0000 7f3de37fe640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3ddc0a12e0 0x7f3ddc0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:45.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.657+0000 7f3df259c640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3ddc0a12e0 0x7f3ddc0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:45.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.657+0000 7f3de37fe640 1 -- 172.21.15.45:0/662367269 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3de80ef570 con 0x7f3df4100de0 2024-01-31T05:20:45.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.658+0000 7f3df259c640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3ddc0a12e0 0x7f3ddc0a37a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3df4100c40 tx=0x7f3dec005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:45.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.661+0000 7f3de37fe640 1 -- 172.21.15.45:0/662367269 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f3de80b8b40 con 0x7f3df4100de0 2024-01-31T05:20:45.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.725+0000 7f3de37fe640 1 -- 172.21.15.45:0/662367269 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f3de80b8f10 con 0x7f3df4100de0 2024-01-31T05:20:45.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.874+0000 7f3df8de1640 1 -- 172.21.15.45:0/662367269 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f3df4068590 con 0x7f3ddc0a12e0 2024-01-31T05:20:45.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.898+0000 7f3de37fe640 1 -- 172.21.15.45:0/662367269 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3df4068590 con 0x7f3ddc0a12e0 2024-01-31T05:20:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.900+0000 7f3df8de1640 1 -- 172.21.15.45:0/662367269 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3ddc0a12e0 msgr2=0x7f3ddc0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.900+0000 7f3df8de1640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3ddc0a12e0 0x7f3ddc0a37a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3df4100c40 tx=0x7f3dec005d30 comp rx=0 tx=0).stop 2024-01-31T05:20:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.900+0000 7f3df8de1640 1 -- 172.21.15.45:0/662367269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4100de0 msgr2=0x7f3df41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.900+0000 7f3df8de1640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4100de0 0x7f3df41021c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009430 tx=0x7f3de8004ee0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.900+0000 7f3df8de1640 1 -- 172.21.15.45:0/662367269 shutdown_connections 2024-01-31T05:20:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.900+0000 7f3df8de1640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3ddc0a12e0 0x7f3ddc0a37a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.900+0000 7f3df8de1640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df4100de0 0x7f3df41021c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.900+0000 7f3df8de1640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df41017a0 0x7f3df4102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.900+0000 7f3df8de1640 1 --2- 172.21.15.45:0/662367269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df40ffbf0 0x7f3df4103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.900+0000 7f3df8de1640 1 -- 172.21.15.45:0/662367269 >> 172.21.15.45:0/662367269 conn(0x7f3df40fb180 msgr2=0x7f3df4067e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.900+0000 7f3df8de1640 1 -- 172.21.15.45:0/662367269 shutdown_connections 2024-01-31T05:20:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:45.900+0000 7f3df8de1640 1 -- 172.21.15.45:0/662367269 wait complete. 2024-01-31T05:20:45.910 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-01-31T05:20:45.976 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:20:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b[66600]: 2024-01-31T05:20:45.679+0000 7f3e96d5a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-31T05:20:45.976 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:20:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b[66600]: 2024-01-31T05:20:45.679+0000 7f3e96d5a700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-01-31T05:20:45.976 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:20:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b[66600]: 2024-01-31T05:20:45.679+0000 7f3e96d5a700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-01-31T05:20:46.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.018+0000 7f5db6b7c640 1 -- 172.21.15.45:0/869587330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db00ffc10 msgr2=0x7f5db0100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.018+0000 7f5db6b7c640 1 --2- 172.21.15.45:0/869587330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db00ffc10 0x7f5db0100030 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009900 tx=0x7f5da402bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.018+0000 7f5db6b7c640 1 -- 172.21.15.45:0/869587330 shutdown_connections 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.018+0000 7f5db6b7c640 1 --2- 172.21.15.45:0/869587330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db01017c0 0x7f5db0108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.018+0000 7f5db6b7c640 1 --2- 172.21.15.45:0/869587330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db0100e00 0x7f5db0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.018+0000 7f5db6b7c640 1 --2- 172.21.15.45:0/869587330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db00ffc10 0x7f5db0100030 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.018+0000 7f5db6b7c640 1 -- 172.21.15.45:0/869587330 >> 172.21.15.45:0/869587330 conn(0x7f5db00fb1a0 msgr2=0x7f5db00fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.018+0000 7f5db6b7c640 1 -- 172.21.15.45:0/869587330 shutdown_connections 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.018+0000 7f5db6b7c640 1 -- 172.21.15.45:0/869587330 wait complete. 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db6b7c640 1 Processor -- start 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db6b7c640 1 -- start start 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db6b7c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db00ffc10 0x7f5db0196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db6b7c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db0100e00 0x7f5db0197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db6b7c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db01017c0 0x7f5db019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db6b7c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db019e9b0 con 0x7f5db01017c0 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db6b7c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db019eb20 con 0x7f5db00ffc10 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db6b7c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db019ece0 con 0x7f5db0100e00 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db5119640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db01017c0 0x7f5db019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db5119640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db01017c0 0x7f5db019e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54958/0 (socket says 172.21.15.45:54958) 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db5119640 1 -- 172.21.15.45:0/1896962657 learned_addr learned my addr 172.21.15.45:0/1896962657 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5daffff640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db0100e00 0x7f5db0197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db4918640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db00ffc10 0x7f5db0196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db5119640 1 -- 172.21.15.45:0/1896962657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db0100e00 msgr2=0x7f5db0197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db5119640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db0100e00 0x7f5db0197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db5119640 1 -- 172.21.15.45:0/1896962657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db00ffc10 msgr2=0x7f5db0196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db5119640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db00ffc10 0x7f5db0196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db5119640 1 -- 172.21.15.45:0/1896962657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da4009560 con 0x7f5db01017c0 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5daffff640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db0100e00 0x7f5db0197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.019+0000 7f5db5119640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db01017c0 0x7f5db019e470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5da800b970 tx=0x7f5da800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.020+0000 7f5dadffb640 1 -- 172.21.15.45:0/1896962657 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da800cb40 con 0x7f5db01017c0 2024-01-31T05:20:46.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.020+0000 7f5dadffb640 1 -- 172.21.15.45:0/1896962657 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5da800cca0 con 0x7f5db01017c0 2024-01-31T05:20:46.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.020+0000 7f5db6b7c640 1 -- 172.21.15.45:0/1896962657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db019efc0 con 0x7f5db01017c0 2024-01-31T05:20:46.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.020+0000 7f5dadffb640 1 -- 172.21.15.45:0/1896962657 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8007770 con 0x7f5db01017c0 2024-01-31T05:20:46.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.020+0000 7f5db6b7c640 1 -- 172.21.15.45:0/1896962657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db019f470 con 0x7f5db01017c0 2024-01-31T05:20:46.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.021+0000 7f5db6b7c640 1 -- 172.21.15.45:0/1896962657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db010b5a0 con 0x7f5db01017c0 2024-01-31T05:20:46.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.022+0000 7f5dadffb640 1 -- 172.21.15.45:0/1896962657 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f5da8014440 con 0x7f5db01017c0 2024-01-31T05:20:46.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.022+0000 7f5dadffb640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5da00a1370 0x7f5da00a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:46.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.022+0000 7f5db4918640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5da00a1370 0x7f5da00a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:46.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.022+0000 7f5dadffb640 1 -- 172.21.15.45:0/1896962657 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5da80cc490 con 0x7f5db01017c0 2024-01-31T05:20:46.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.023+0000 7f5db4918640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5da00a1370 0x7f5da00a3830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5da402c6b0 tx=0x7f5da402c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:46.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.027+0000 7f5dadffb640 1 -- 172.21.15.45:0/1896962657 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5da804f600 con 0x7f5db01017c0 2024-01-31T05:20:46.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.235+0000 7f5db6b7c640 1 -- 172.21.15.45:0/1896962657 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5db0069a80 con 0x7f5da00a1370 2024-01-31T05:20:46.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.239+0000 7f5dadffb640 1 -- 172.21.15.45:0/1896962657 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f5db0069a80 con 0x7f5da00a1370 2024-01-31T05:20:46.238 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_1/0677e5c0-3673-41ea-a0ca-4130f847896f 2024-01-31T05:20:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.241+0000 7f5db6b7c640 1 -- 172.21.15.45:0/1896962657 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5da00a1370 msgr2=0x7f5da00a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.241+0000 7f5db6b7c640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5da00a1370 0x7f5da00a3830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5da402c6b0 tx=0x7f5da402c5d0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.241+0000 7f5db6b7c640 1 -- 172.21.15.45:0/1896962657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db01017c0 msgr2=0x7f5db019e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.241+0000 7f5db6b7c640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db01017c0 0x7f5db019e470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5da800b970 tx=0x7f5da800be30 comp rx=0 tx=0).stop 2024-01-31T05:20:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.241+0000 7f5db6b7c640 1 -- 172.21.15.45:0/1896962657 shutdown_connections 2024-01-31T05:20:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.241+0000 7f5db6b7c640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5da00a1370 0x7f5da00a3830 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.241+0000 7f5db6b7c640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db01017c0 0x7f5db019e470 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.241+0000 7f5db6b7c640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db0100e00 0x7f5db0197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.241+0000 7f5db6b7c640 1 --2- 172.21.15.45:0/1896962657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db00ffc10 0x7f5db0196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.241+0000 7f5db6b7c640 1 -- 172.21.15.45:0/1896962657 >> 172.21.15.45:0/1896962657 conn(0x7f5db00fb1a0 msgr2=0x7f5db0069370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.241+0000 7f5db6b7c640 1 -- 172.21.15.45:0/1896962657 shutdown_connections 2024-01-31T05:20:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.241+0000 7f5db6b7c640 1 -- 172.21.15.45:0/1896962657 wait complete. 2024-01-31T05:20:46.251 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:20:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.354+0000 7f5f6c589640 1 -- 172.21.15.45:0/3251132475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f641017e0 msgr2=0x7f5f64108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.354+0000 7f5f6c589640 1 --2- 172.21.15.45:0/3251132475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f641017e0 0x7f5f64108070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009ca0 tx=0x7f5f5c02c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.354+0000 7f5f6c589640 1 -- 172.21.15.45:0/3251132475 shutdown_connections 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.354+0000 7f5f6c589640 1 --2- 172.21.15.45:0/3251132475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f641017e0 0x7f5f64108070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.354+0000 7f5f6c589640 1 --2- 172.21.15.45:0/3251132475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f64100e20 0x7f5f641012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.354+0000 7f5f6c589640 1 --2- 172.21.15.45:0/3251132475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f640ffc30 0x7f5f64100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.354+0000 7f5f6c589640 1 -- 172.21.15.45:0/3251132475 >> 172.21.15.45:0/3251132475 conn(0x7f5f640fb1a0 msgr2=0x7f5f640fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.354+0000 7f5f6c589640 1 -- 172.21.15.45:0/3251132475 shutdown_connections 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.354+0000 7f5f6c589640 1 -- 172.21.15.45:0/3251132475 wait complete. 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6c589640 1 Processor -- start 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6c589640 1 -- start start 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6c589640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f640ffc30 0x7f5f64195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6c589640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f64100e20 0x7f5f64196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6c589640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f641017e0 0x7f5f6419d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6c589640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6419d640 con 0x7f5f641017e0 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6c589640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6419d7b0 con 0x7f5f64100e20 2024-01-31T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6c589640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6419d970 con 0x7f5f640ffc30 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6ab26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f641017e0 0x7f5f6419d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6ab26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f641017e0 0x7f5f6419d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54982/0 (socket says 172.21.15.45:54982) 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6ab26640 1 -- 172.21.15.45:0/4159742351 learned_addr learned my addr 172.21.15.45:0/4159742351 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6a325640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f640ffc30 0x7f5f64195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f69b24640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f64100e20 0x7f5f64196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6ab26640 1 -- 172.21.15.45:0/4159742351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f640ffc30 msgr2=0x7f5f64195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6ab26640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f640ffc30 0x7f5f64195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6ab26640 1 -- 172.21.15.45:0/4159742351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f64100e20 msgr2=0x7f5f64196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6ab26640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f64100e20 0x7f5f64196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.355+0000 7f5f6ab26640 1 -- 172.21.15.45:0/4159742351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c009900 con 0x7f5f641017e0 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.356+0000 7f5f6ab26640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f641017e0 0x7f5f6419d100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c037040 tx=0x7f5f5c032810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.356+0000 7f5f5b7fe640 1 -- 172.21.15.45:0/4159742351 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c034e80 con 0x7f5f641017e0 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.356+0000 7f5f5b7fe640 1 -- 172.21.15.45:0/4159742351 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f5c003e30 con 0x7f5f641017e0 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.356+0000 7f5f6c589640 1 -- 172.21.15.45:0/4159742351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6419dbf0 con 0x7f5f641017e0 2024-01-31T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.356+0000 7f5f5b7fe640 1 -- 172.21.15.45:0/4159742351 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c03d3f0 con 0x7f5f641017e0 2024-01-31T05:20:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.356+0000 7f5f6c589640 1 -- 172.21.15.45:0/4159742351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6419e0a0 con 0x7f5f641017e0 2024-01-31T05:20:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.357+0000 7f5f6c589640 1 -- 172.21.15.45:0/4159742351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f64106fa0 con 0x7f5f641017e0 2024-01-31T05:20:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.358+0000 7f5f5b7fe640 1 -- 172.21.15.45:0/4159742351 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f5f5c03b040 con 0x7f5f641017e0 2024-01-31T05:20:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.358+0000 7f5f5b7fe640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5f500a13f0 0x7f5f500a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.358+0000 7f5f6a325640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5f500a13f0 0x7f5f500a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.358+0000 7f5f5b7fe640 1 -- 172.21.15.45:0/4159742351 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5f5c0ef630 con 0x7f5f641017e0 2024-01-31T05:20:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.359+0000 7f5f6a325640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5f500a13f0 0x7f5f500a38b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5f64100c80 tx=0x7f5f6000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:46.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.362+0000 7f5f5b7fe640 1 -- 172.21.15.45:0/4159742351 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5f5c0b8bc0 con 0x7f5f641017e0 2024-01-31T05:20:46.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.578+0000 7f5f6c589640 1 -- 172.21.15.45:0/4159742351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f64106a10 con 0x7f5f641017e0 2024-01-31T05:20:46.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.579+0000 7f5f5b7fe640 1 -- 172.21.15.45:0/4159742351 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f5f5c0bc1e0 con 0x7f5f641017e0 2024-01-31T05:20:46.578 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:46.578 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1801709710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1801709710},{"type":"v1","addr":"172.21.15.45:6839","nonce":1801709710}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/1148586810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1148586810},{"type":"v1","addr":"172.21.15.45:6841","nonce":1148586810}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-31T05:20:40.057668+0000","modified":"2024-01-31T05:20:44.476398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3782584640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3782584640},{"type":"v1","addr":"172.21.15.156:6837","nonce":3782584640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/3821255404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3821255404},{"type":"v1","addr":"172.21.15.186:6839","nonce":3821255404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/3137398384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3137398384},{"type":"v1","addr":"172.21.15.156:6839","nonce":3137398384}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:20:46.578 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 20 2024-01-31T05:20:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.580+0000 7f5f6c589640 1 -- 172.21.15.45:0/4159742351 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5f500a13f0 msgr2=0x7f5f500a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.580+0000 7f5f6c589640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5f500a13f0 0x7f5f500a38b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5f64100c80 tx=0x7f5f6000b390 comp rx=0 tx=0).stop 2024-01-31T05:20:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.580+0000 7f5f6c589640 1 -- 172.21.15.45:0/4159742351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f641017e0 msgr2=0x7f5f6419d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.580+0000 7f5f6c589640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f641017e0 0x7f5f6419d100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c037040 tx=0x7f5f5c032810 comp rx=0 tx=0).stop 2024-01-31T05:20:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.580+0000 7f5f6c589640 1 -- 172.21.15.45:0/4159742351 shutdown_connections 2024-01-31T05:20:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.581+0000 7f5f6c589640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5f500a13f0 0x7f5f500a38b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.581+0000 7f5f6c589640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f641017e0 0x7f5f6419d100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.581+0000 7f5f6c589640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f64100e20 0x7f5f64196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.581+0000 7f5f6c589640 1 --2- 172.21.15.45:0/4159742351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f640ffc30 0x7f5f64195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.581+0000 7f5f6c589640 1 -- 172.21.15.45:0/4159742351 >> 172.21.15.45:0/4159742351 conn(0x7f5f640fb1a0 msgr2=0x7f5f64104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:46.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.581+0000 7f5f6c589640 1 -- 172.21.15.45:0/4159742351 shutdown_connections 2024-01-31T05:20:46.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.581+0000 7f5f6c589640 1 -- 172.21.15.45:0/4159742351 wait complete. 2024-01-31T05:20:46.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:46 smithi045 ceph-mon[31585]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:46.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:46 smithi045 ceph-mon[31585]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:46.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:46 smithi045 ceph-mon[31585]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:46.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:46 smithi045 ceph-mon[31585]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-01-31T05:20:46.591 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:20:46.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.694+0000 7fdafd4b1640 1 -- 172.21.15.45:0/198802063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8100d30 msgr2=0x7fdaf8101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.694+0000 7fdafd4b1640 1 --2- 172.21.15.45:0/198802063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8100d30 0x7fdaf8101150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009ca0 tx=0x7fdae802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.694+0000 7fdafd4b1640 1 -- 172.21.15.45:0/198802063 shutdown_connections 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.694+0000 7fdafd4b1640 1 --2- 172.21.15.45:0/198802063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf81028e0 0x7fdaf8109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.694+0000 7fdafd4b1640 1 --2- 172.21.15.45:0/198802063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaf8101f20 0x7fdaf81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.694+0000 7fdafd4b1640 1 --2- 172.21.15.45:0/198802063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8100d30 0x7fdaf8101150 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.694+0000 7fdafd4b1640 1 -- 172.21.15.45:0/198802063 >> 172.21.15.45:0/198802063 conn(0x7fdaf80fc2a0 msgr2=0x7fdaf80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.694+0000 7fdafd4b1640 1 -- 172.21.15.45:0/198802063 shutdown_connections 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.694+0000 7fdafd4b1640 1 -- 172.21.15.45:0/198802063 wait complete. 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdafd4b1640 1 Processor -- start 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdafd4b1640 1 -- start start 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdafd4b1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf8100d30 0x7fdaf8196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdafd4b1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8101f20 0x7fdaf8197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdafd4b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaf81028e0 0x7fdaf819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdafd4b1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf819e820 con 0x7fdaf8101f20 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdafd4b1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf819e990 con 0x7fdaf81028e0 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdafd4b1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf819eb50 con 0x7fdaf8100d30 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdaf67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8101f20 0x7fdaf8197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdaf67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8101f20 0x7fdaf8197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54998/0 (socket says 172.21.15.45:54998) 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdaf67fc640 1 -- 172.21.15.45:0/2861565696 learned_addr learned my addr 172.21.15.45:0/2861565696 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdaf77fe640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaf81028e0 0x7fdaf819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.695+0000 7fdaf6ffd640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf8100d30 0x7fdaf8196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdaf67fc640 1 -- 172.21.15.45:0/2861565696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf8100d30 msgr2=0x7fdaf8196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdaf67fc640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf8100d30 0x7fdaf8196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdaf67fc640 1 -- 172.21.15.45:0/2861565696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaf81028e0 msgr2=0x7fdaf819e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdaf67fc640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaf81028e0 0x7fdaf819e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdaf67fc640 1 -- 172.21.15.45:0/2861565696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae8009900 con 0x7fdaf8101f20 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdaf6ffd640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf8100d30 0x7fdaf8196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdaf77fe640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaf81028e0 0x7fdaf819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:20:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdaf67fc640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8101f20 0x7fdaf8197280 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdae000d950 tx=0x7fdae000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdaeffff640 1 -- 172.21.15.45:0/2861565696 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0009c30 con 0x7fdaf8101f20 2024-01-31T05:20:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdafd4b1640 1 -- 172.21.15.45:0/2861565696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf819ee30 con 0x7fdaf8101f20 2024-01-31T05:20:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdaeffff640 1 -- 172.21.15.45:0/2861565696 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdae0009d90 con 0x7fdaf8101f20 2024-01-31T05:20:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdaeffff640 1 -- 172.21.15.45:0/2861565696 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae000f740 con 0x7fdaf8101f20 2024-01-31T05:20:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.696+0000 7fdafd4b1640 1 -- 172.21.15.45:0/2861565696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf819f260 con 0x7fdaf8101f20 2024-01-31T05:20:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.697+0000 7fdafd4b1640 1 -- 172.21.15.45:0/2861565696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf81080c0 con 0x7fdaf8101f20 2024-01-31T05:20:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.698+0000 7fdaeffff640 1 -- 172.21.15.45:0/2861565696 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fdae0010a20 con 0x7fdaf8101f20 2024-01-31T05:20:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.699+0000 7fdaeffff640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdae40a1370 0x7fdae40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.699+0000 7fdaf6ffd640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdae40a1370 0x7fdae40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.699+0000 7fdaeffff640 1 -- 172.21.15.45:0/2861565696 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdae00cc8d0 con 0x7fdaf8101f20 2024-01-31T05:20:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.699+0000 7fdaf6ffd640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdae40a1370 0x7fdae40a3830 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8198210 tx=0x7fdae8032690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:46.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.703+0000 7fdaeffff640 1 -- 172.21.15.45:0/2861565696 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fdae0095d10 con 0x7fdaf8101f20 2024-01-31T05:20:46.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:46 smithi186 ceph-mon[35659]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:46.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:46 smithi186 ceph-mon[35659]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:46.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:46 smithi186 ceph-mon[35659]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:46.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:46 smithi186 ceph-mon[35659]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-01-31T05:20:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.884+0000 7fdafd4b1640 1 -- 172.21.15.45:0/2861565696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaf810c840 con 0x7fdaf8101f20 2024-01-31T05:20:46.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.886+0000 7fdaeffff640 1 -- 172.21.15.45:0/2861565696 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22133 (secure 0 0 0) 0x7fdae0097c40 con 0x7fdaf8101f20 2024-01-31T05:20:46.885 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:46.885 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":86,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:20:41.704316+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:20:46.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.887+0000 7fdafd4b1640 1 -- 172.21.15.45:0/2861565696 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdae40a1370 msgr2=0x7fdae40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.887+0000 7fdafd4b1640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdae40a1370 0x7fdae40a3830 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8198210 tx=0x7fdae8032690 comp rx=0 tx=0).stop 2024-01-31T05:20:46.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.887+0000 7fdafd4b1640 1 -- 172.21.15.45:0/2861565696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8101f20 msgr2=0x7fdaf8197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:46.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.887+0000 7fdafd4b1640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8101f20 0x7fdaf8197280 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdae000d950 tx=0x7fdae000de10 comp rx=0 tx=0).stop 2024-01-31T05:20:46.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.888+0000 7fdafd4b1640 1 -- 172.21.15.45:0/2861565696 shutdown_connections 2024-01-31T05:20:46.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.888+0000 7fdafd4b1640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdae40a1370 0x7fdae40a3830 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.888+0000 7fdafd4b1640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8101f20 0x7fdaf8197280 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.888+0000 7fdafd4b1640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaf81028e0 0x7fdaf819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.888+0000 7fdafd4b1640 1 --2- 172.21.15.45:0/2861565696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf8100d30 0x7fdaf8196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:46.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.888+0000 7fdafd4b1640 1 -- 172.21.15.45:0/2861565696 >> 172.21.15.45:0/2861565696 conn(0x7fdaf80fc2a0 msgr2=0x7fdaf8105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:46.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.888+0000 7fdafd4b1640 1 -- 172.21.15.45:0/2861565696 shutdown_connections 2024-01-31T05:20:46.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:46.888+0000 7fdafd4b1640 1 -- 172.21.15.45:0/2861565696 wait complete. 2024-01-31T05:20:46.898 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-31T05:20:46.898 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:20:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:46 smithi156 ceph-mon[35720]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:46 smithi156 ceph-mon[35720]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:46 smithi156 ceph-mon[35720]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:46 smithi156 ceph-mon[35720]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-01-31T05:20:47.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.001+0000 7fb80079d640 1 -- 172.21.15.45:0/681146051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f8101fb0 msgr2=0x7fb7f8102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:47.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.001+0000 7fb80079d640 1 --2- 172.21.15.45:0/681146051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f8101fb0 0x7fb7f8102430 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009900 tx=0x7fb7e802bc30 comp rx=0 tx=0).stop 2024-01-31T05:20:47.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.001+0000 7fb80079d640 1 -- 172.21.15.45:0/681146051 shutdown_connections 2024-01-31T05:20:47.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.001+0000 7fb80079d640 1 --2- 172.21.15.45:0/681146051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f8101fb0 0x7fb7f8102430 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.001+0000 7fb80079d640 1 --2- 172.21.15.45:0/681146051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f8102970 0x7fb7f8109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.001+0000 7fb80079d640 1 --2- 172.21.15.45:0/681146051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7f80fefe0 0x7fb7f80ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.001+0000 7fb80079d640 1 -- 172.21.15.45:0/681146051 >> 172.21.15.45:0/681146051 conn(0x7fb7f80fa910 msgr2=0x7fb7f80fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.001+0000 7fb80079d640 1 -- 172.21.15.45:0/681146051 shutdown_connections 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.001+0000 7fb80079d640 1 -- 172.21.15.45:0/681146051 wait complete. 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb80079d640 1 Processor -- start 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb80079d640 1 -- start start 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb80079d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f80fefe0 0x7fb7f8196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb80079d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7f8101fb0 0x7fb7f8197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb80079d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f8102970 0x7fb7f819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb80079d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f819e720 con 0x7fb7f8102970 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb80079d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f819e890 con 0x7fb7f8101fb0 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb80079d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f819ea50 con 0x7fb7f80fefe0 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb7fed3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f8102970 0x7fb7f819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb7fed3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f8102970 0x7fb7f819e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55010/0 (socket says 172.21.15.45:55010) 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb7fed3a640 1 -- 172.21.15.45:0/1084179347 learned_addr learned my addr 172.21.15.45:0/1084179347 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:47.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb7fe539640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f80fefe0 0x7fb7f8196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:47.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb7fed3a640 1 -- 172.21.15.45:0/1084179347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f80fefe0 msgr2=0x7fb7f8196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:47.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb7fed3a640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f80fefe0 0x7fb7f8196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb7fed3a640 1 -- 172.21.15.45:0/1084179347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7f8101fb0 msgr2=0x7fb7f8197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:47.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb7fed3a640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7f8101fb0 0x7fb7f8197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb7fed3a640 1 -- 172.21.15.45:0/1084179347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e8009560 con 0x7fb7f8102970 2024-01-31T05:20:47.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb7fed3a640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f8102970 0x7fb7f819e1b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000eb10 tx=0x7fb7f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:47.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb7ef7fe640 1 -- 172.21.15.45:0/1084179347 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0004b20 con 0x7fb7f8102970 2024-01-31T05:20:47.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.002+0000 7fb7ef7fe640 1 -- 172.21.15.45:0/1084179347 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7f0004c80 con 0x7fb7f8102970 2024-01-31T05:20:47.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.003+0000 7fb7ef7fe640 1 -- 172.21.15.45:0/1084179347 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0010690 con 0x7fb7f8102970 2024-01-31T05:20:47.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.003+0000 7fb80079d640 1 -- 172.21.15.45:0/1084179347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f819ed30 con 0x7fb7f8102970 2024-01-31T05:20:47.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.003+0000 7fb80079d640 1 -- 172.21.15.45:0/1084179347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f819f1e0 con 0x7fb7f8102970 2024-01-31T05:20:47.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.003+0000 7fb80079d640 1 -- 172.21.15.45:0/1084179347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f8108160 con 0x7fb7f8102970 2024-01-31T05:20:47.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.004+0000 7fb7ef7fe640 1 -- 172.21.15.45:0/1084179347 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fb7f0005020 con 0x7fb7f8102970 2024-01-31T05:20:47.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.005+0000 7fb7ef7fe640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb7e40a1370 0x7fb7e40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:47.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.005+0000 7fb7fe539640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb7e40a1370 0x7fb7e40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:47.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.005+0000 7fb7ef7fe640 1 -- 172.21.15.45:0/1084179347 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb7f0014070 con 0x7fb7f8102970 2024-01-31T05:20:47.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.006+0000 7fb7fe539640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb7e40a1370 0x7fb7e40a3830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40058c0 tx=0x7fb7f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:47.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.009+0000 7fb7ef7fe640 1 -- 172.21.15.45:0/1084179347 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb7f0096690 con 0x7fb7f8102970 2024-01-31T05:20:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.188+0000 7fb80079d640 1 -- 172.21.15.45:0/1084179347 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7f802cd50 con 0x7fb7e40a1370 2024-01-31T05:20:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.199+0000 7fb7ef7fe640 1 -- 172.21.15.45:0/1084179347 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336163 (secure 0 0 0) 0x7fb7f802cd50 con 0x7fb7e40a1370 2024-01-31T05:20:47.203 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:47.205 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-01-31T05:20:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.203+0000 7fb80079d640 1 -- 172.21.15.45:0/1084179347 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb7e40a1370 msgr2=0x7fb7e40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.203+0000 7fb80079d640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb7e40a1370 0x7fb7e40a3830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40058c0 tx=0x7fb7f4005850 comp rx=0 tx=0).stop 2024-01-31T05:20:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.203+0000 7fb80079d640 1 -- 172.21.15.45:0/1084179347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f8102970 msgr2=0x7fb7f819e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.203+0000 7fb80079d640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f8102970 0x7fb7f819e1b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000eb10 tx=0x7fb7f000c2d0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.203+0000 7fb80079d640 1 -- 172.21.15.45:0/1084179347 shutdown_connections 2024-01-31T05:20:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.203+0000 7fb80079d640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb7e40a1370 0x7fb7e40a3830 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.203+0000 7fb80079d640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f8102970 0x7fb7f819e1b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.203+0000 7fb80079d640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7f8101fb0 0x7fb7f8197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.203+0000 7fb80079d640 1 --2- 172.21.15.45:0/1084179347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7f80fefe0 0x7fb7f8196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.203+0000 7fb80079d640 1 -- 172.21.15.45:0/1084179347 >> 172.21.15.45:0/1084179347 conn(0x7fb7f80fa910 msgr2=0x7fb7f8105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.203+0000 7fb80079d640 1 -- 172.21.15.45:0/1084179347 shutdown_connections 2024-01-31T05:20:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.203+0000 7fb80079d640 1 -- 172.21.15.45:0/1084179347 wait complete. 2024-01-31T05:20:47.213 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-01-31T05:20:46.591268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613190,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":153,"num_read_kb":152,"num_write":301,"num_write_kb":2995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":268,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":399376,"kb_used_data":12724,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124542448,"statfs":{"total":1151940427776,"available":1151531466752,"internally_reserved":0,"allocated":13029376,"data_stored":7088608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22822,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":17,"num_read_kb":33,"num_write":120,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.896534"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989980+0000","last_change":"2024-01-31T05:20:40.720465+0000","last_active":"2024-01-31T05:20:41.989980+0000","last_peered":"2024-01-31T05:20:41.989980+0000","last_clean":"2024-01-31T05:20:41.989980+0000","last_became_active":"2024-01-31T05:20:40.717615+0000","last_became_peered":"2024-01-31T05:20:40.717615+0000","last_unstale":"2024-01-31T05:20:41.989980+0000","last_undegraded":"2024-01-31T05:20:41.989980+0000","last_fullsized":"2024-01-31T05:20:41.989980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:30:01.564811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.730410+0000","last_change":"2024-01-31T05:20:40.729011+0000","last_active":"2024-01-31T05:20:40.730410+0000","last_peered":"2024-01-31T05:20:40.730410+0000","last_clean":"2024-01-31T05:20:40.730410+0000","last_became_active":"2024-01-31T05:20:40.728021+0000","last_became_peered":"2024-01-31T05:20:40.728021+0000","last_unstale":"2024-01-31T05:20:40.730410+0000","last_undegraded":"2024-01-31T05:20:40.730410+0000","last_fullsized":"2024-01-31T05:20:40.730410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:10:19.013530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.730853+0000","last_change":"2024-01-31T05:20:40.729136+0000","last_active":"2024-01-31T05:20:40.730853+0000","last_peered":"2024-01-31T05:20:40.730853+0000","last_clean":"2024-01-31T05:20:40.730853+0000","last_became_active":"2024-01-31T05:20:40.728273+0000","last_became_peered":"2024-01-31T05:20:40.728273+0000","last_unstale":"2024-01-31T05:20:40.730853+0000","last_undegraded":"2024-01-31T05:20:40.730853+0000","last_fullsized":"2024-01-31T05:20:40.730853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:46:43.462275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.887470+0000","last_change":"2024-01-31T05:20:40.721057+0000","last_active":"2024-01-31T05:20:41.887470+0000","last_peered":"2024-01-31T05:20:41.887470+0000","last_clean":"2024-01-31T05:20:41.887470+0000","last_became_active":"2024-01-31T05:20:40.720595+0000","last_became_peered":"2024-01-31T05:20:40.720595+0000","last_unstale":"2024-01-31T05:20:41.887470+0000","last_undegraded":"2024-01-31T05:20:41.887470+0000","last_fullsized":"2024-01-31T05:20:41.887470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:07:49.519349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990332+0000","last_change":"2024-01-31T05:20:40.724551+0000","last_active":"2024-01-31T05:20:41.990332+0000","last_peered":"2024-01-31T05:20:41.990332+0000","last_clean":"2024-01-31T05:20:41.990332+0000","last_became_active":"2024-01-31T05:20:40.724172+0000","last_became_peered":"2024-01-31T05:20:40.724172+0000","last_unstale":"2024-01-31T05:20:41.990332+0000","last_undegraded":"2024-01-31T05:20:41.990332+0000","last_fullsized":"2024-01-31T05:20:41.990332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:45:28.748481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989682+0000","last_change":"2024-01-31T05:20:40.721982+0000","last_active":"2024-01-31T05:20:41.989682+0000","last_peered":"2024-01-31T05:20:41.989682+0000","last_clean":"2024-01-31T05:20:41.989682+0000","last_became_active":"2024-01-31T05:20:40.721649+0000","last_became_peered":"2024-01-31T05:20:40.721649+0000","last_unstale":"2024-01-31T05:20:41.989682+0000","last_undegraded":"2024-01-31T05:20:41.989682+0000","last_fullsized":"2024-01-31T05:20:41.989682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:47:32.776711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.988984+0000","last_change":"2024-01-31T05:20:40.718340+0000","last_active":"2024-01-31T05:20:41.988984+0000","last_peered":"2024-01-31T05:20:41.988984+0000","last_clean":"2024-01-31T05:20:41.988984+0000","last_became_active":"2024-01-31T05:20:40.718017+0000","last_became_peered":"2024-01-31T05:20:40.718017+0000","last_unstale":"2024-01-31T05:20:41.988984+0000","last_undegraded":"2024-01-31T05:20:41.988984+0000","last_fullsized":"2024-01-31T05:20:41.988984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:17:45.606831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.711507+0000","last_change":"2024-01-31T05:20:40.720271+0000","last_active":"2024-01-31T05:20:41.711507+0000","last_peered":"2024-01-31T05:20:41.711507+0000","last_clean":"2024-01-31T05:20:41.711507+0000","last_became_active":"2024-01-31T05:20:40.719960+0000","last_became_peered":"2024-01-31T05:20:40.719960+0000","last_unstale":"2024-01-31T05:20:41.711507+0000","last_undegraded":"2024-01-31T05:20:41.711507+0000","last_fullsized":"2024-01-31T05:20:41.711507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:59:16.243268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712859+0000","last_change":"2024-01-31T05:20:40.725779+0000","last_active":"2024-01-31T05:20:41.712859+0000","last_peered":"2024-01-31T05:20:41.712859+0000","last_clean":"2024-01-31T05:20:41.712859+0000","last_became_active":"2024-01-31T05:20:40.725423+0000","last_became_peered":"2024-01-31T05:20:40.725423+0000","last_unstale":"2024-01-31T05:20:41.712859+0000","last_undegraded":"2024-01-31T05:20:41.712859+0000","last_fullsized":"2024-01-31T05:20:41.712859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:11:46.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990266+0000","last_change":"2024-01-31T05:20:40.723144+0000","last_active":"2024-01-31T05:20:41.990266+0000","last_peered":"2024-01-31T05:20:41.990266+0000","last_clean":"2024-01-31T05:20:41.990266+0000","last_became_active":"2024-01-31T05:20:40.722806+0000","last_became_peered":"2024-01-31T05:20:40.722806+0000","last_unstale":"2024-01-31T05:20:41.990266+0000","last_undegraded":"2024-01-31T05:20:41.990266+0000","last_fullsized":"2024-01-31T05:20:41.990266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:33.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989730+0000","last_change":"2024-01-31T05:20:40.721083+0000","last_active":"2024-01-31T05:20:41.989730+0000","last_peered":"2024-01-31T05:20:41.989730+0000","last_clean":"2024-01-31T05:20:41.989730+0000","last_became_active":"2024-01-31T05:20:40.720773+0000","last_became_peered":"2024-01-31T05:20:40.720773+0000","last_unstale":"2024-01-31T05:20:41.989730+0000","last_undegraded":"2024-01-31T05:20:41.989730+0000","last_fullsized":"2024-01-31T05:20:41.989730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:39:08.114126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.887435+0000","last_change":"2024-01-31T05:20:40.723630+0000","last_active":"2024-01-31T05:20:41.887435+0000","last_peered":"2024-01-31T05:20:41.887435+0000","last_clean":"2024-01-31T05:20:41.887435+0000","last_became_active":"2024-01-31T05:20:40.723340+0000","last_became_peered":"2024-01-31T05:20:40.723340+0000","last_unstale":"2024-01-31T05:20:41.887435+0000","last_undegraded":"2024-01-31T05:20:41.887435+0000","last_fullsized":"2024-01-31T05:20:41.887435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:53:21.199111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.713122+0000","last_change":"2024-01-31T05:20:40.712602+0000","last_active":"2024-01-31T05:20:40.713122+0000","last_peered":"2024-01-31T05:20:40.713122+0000","last_clean":"2024-01-31T05:20:40.713122+0000","last_became_active":"2024-01-31T05:20:40.712261+0000","last_became_peered":"2024-01-31T05:20:40.712261+0000","last_unstale":"2024-01-31T05:20:40.713122+0000","last_undegraded":"2024-01-31T05:20:40.713122+0000","last_fullsized":"2024-01-31T05:20:40.713122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:40:20.754437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.726917+0000","last_change":"2024-01-31T05:20:40.723759+0000","last_active":"2024-01-31T05:20:40.726917+0000","last_peered":"2024-01-31T05:20:40.726917+0000","last_clean":"2024-01-31T05:20:40.726917+0000","last_became_active":"2024-01-31T05:20:40.723421+0000","last_became_peered":"2024-01-31T05:20:40.723421+0000","last_unstale":"2024-01-31T05:20:40.726917+0000","last_undegraded":"2024-01-31T05:20:40.726917+0000","last_fullsized":"2024-01-31T05:20:40.726917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:35.268012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.727170+0000","last_change":"2024-01-31T05:20:40.726482+0000","last_active":"2024-01-31T05:20:40.727170+0000","last_peered":"2024-01-31T05:20:40.727170+0000","last_clean":"2024-01-31T05:20:40.727170+0000","last_became_active":"2024-01-31T05:20:40.726086+0000","last_became_peered":"2024-01-31T05:20:40.726086+0000","last_unstale":"2024-01-31T05:20:40.727170+0000","last_undegraded":"2024-01-31T05:20:40.727170+0000","last_fullsized":"2024-01-31T05:20:40.727170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:27:53.695166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712234+0000","last_change":"2024-01-31T05:20:40.728363+0000","last_active":"2024-01-31T05:20:41.712234+0000","last_peered":"2024-01-31T05:20:41.712234+0000","last_clean":"2024-01-31T05:20:41.712234+0000","last_became_active":"2024-01-31T05:20:40.727795+0000","last_became_peered":"2024-01-31T05:20:40.727795+0000","last_unstale":"2024-01-31T05:20:41.712234+0000","last_undegraded":"2024-01-31T05:20:41.712234+0000","last_fullsized":"2024-01-31T05:20:41.712234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:43:08.123159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989887+0000","last_change":"2024-01-31T05:20:40.720902+0000","last_active":"2024-01-31T05:20:41.989887+0000","last_peered":"2024-01-31T05:20:41.989887+0000","last_clean":"2024-01-31T05:20:41.989887+0000","last_became_active":"2024-01-31T05:20:40.720554+0000","last_became_peered":"2024-01-31T05:20:40.720554+0000","last_unstale":"2024-01-31T05:20:41.989887+0000","last_undegraded":"2024-01-31T05:20:41.989887+0000","last_fullsized":"2024-01-31T05:20:41.989887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:56:49.441368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990090+0000","last_change":"2024-01-31T05:20:40.710504+0000","last_active":"2024-01-31T05:20:41.990090+0000","last_peered":"2024-01-31T05:20:41.990090+0000","last_clean":"2024-01-31T05:20:41.990090+0000","last_became_active":"2024-01-31T05:20:40.710162+0000","last_became_peered":"2024-01-31T05:20:40.710162+0000","last_unstale":"2024-01-31T05:20:41.990090+0000","last_undegraded":"2024-01-31T05:20:41.990090+0000","last_fullsized":"2024-01-31T05:20:41.990090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:49:49.930573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989921+0000","last_change":"2024-01-31T05:20:40.729023+0000","last_active":"2024-01-31T05:20:41.989921+0000","last_peered":"2024-01-31T05:20:41.989921+0000","last_clean":"2024-01-31T05:20:41.989921+0000","last_became_active":"2024-01-31T05:20:40.728672+0000","last_became_peered":"2024-01-31T05:20:40.728672+0000","last_unstale":"2024-01-31T05:20:41.989921+0000","last_undegraded":"2024-01-31T05:20:41.989921+0000","last_fullsized":"2024-01-31T05:20:41.989921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:56:41.578440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.484132+0000","last_change":"2024-01-31T05:20:39.716771+0000","last_active":"2024-01-31T05:20:43.484132+0000","last_peered":"2024-01-31T05:20:43.484132+0000","last_clean":"2024-01-31T05:20:43.484132+0000","last_became_active":"2024-01-31T05:20:39.716439+0000","last_became_peered":"2024-01-31T05:20:39.716439+0000","last_unstale":"2024-01-31T05:20:43.484132+0000","last_undegraded":"2024-01-31T05:20:43.484132+0000","last_fullsized":"2024-01-31T05:20:43.484132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:14:24.317293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.711931+0000","last_change":"2024-01-31T05:20:40.730695+0000","last_active":"2024-01-31T05:20:41.711931+0000","last_peered":"2024-01-31T05:20:41.711931+0000","last_clean":"2024-01-31T05:20:41.711931+0000","last_became_active":"2024-01-31T05:20:40.730343+0000","last_became_peered":"2024-01-31T05:20:40.730343+0000","last_unstale":"2024-01-31T05:20:41.711931+0000","last_undegraded":"2024-01-31T05:20:41.711931+0000","last_fullsized":"2024-01-31T05:20:41.711931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:32.308239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712063+0000","last_change":"2024-01-31T05:20:39.711082+0000","last_active":"2024-01-31T05:20:41.712063+0000","last_peered":"2024-01-31T05:20:41.712063+0000","last_clean":"2024-01-31T05:20:41.712063+0000","last_became_active":"2024-01-31T05:20:39.710741+0000","last_became_peered":"2024-01-31T05:20:39.710741+0000","last_unstale":"2024-01-31T05:20:41.712063+0000","last_undegraded":"2024-01-31T05:20:41.712063+0000","last_fullsized":"2024-01-31T05:20:41.712063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:20:01.934837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.730946+0000","last_change":"2024-01-31T05:20:40.729269+0000","last_active":"2024-01-31T05:20:40.730946+0000","last_peered":"2024-01-31T05:20:40.730946+0000","last_clean":"2024-01-31T05:20:40.730946+0000","last_became_active":"2024-01-31T05:20:40.728910+0000","last_became_peered":"2024-01-31T05:20:40.728910+0000","last_unstale":"2024-01-31T05:20:40.730946+0000","last_undegraded":"2024-01-31T05:20:40.730946+0000","last_fullsized":"2024-01-31T05:20:40.730946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:28:16.951673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.482200+0000","last_change":"2024-01-31T05:20:39.710336+0000","last_active":"2024-01-31T05:20:43.482200+0000","last_peered":"2024-01-31T05:20:43.482200+0000","last_clean":"2024-01-31T05:20:43.482200+0000","last_became_active":"2024-01-31T05:20:39.710034+0000","last_became_peered":"2024-01-31T05:20:39.710034+0000","last_unstale":"2024-01-31T05:20:43.482200+0000","last_undegraded":"2024-01-31T05:20:43.482200+0000","last_fullsized":"2024-01-31T05:20:43.482200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:27:46.964866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712457+0000","last_change":"2024-01-31T05:20:40.727934+0000","last_active":"2024-01-31T05:20:41.712457+0000","last_peered":"2024-01-31T05:20:41.712457+0000","last_clean":"2024-01-31T05:20:41.712457+0000","last_became_active":"2024-01-31T05:20:40.727517+0000","last_became_peered":"2024-01-31T05:20:40.727517+0000","last_unstale":"2024-01-31T05:20:41.712457+0000","last_undegraded":"2024-01-31T05:20:41.712457+0000","last_fullsized":"2024-01-31T05:20:41.712457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:35:46.329596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990495+0000","last_change":"2024-01-31T05:20:39.709541+0000","last_active":"2024-01-31T05:20:41.990495+0000","last_peered":"2024-01-31T05:20:41.990495+0000","last_clean":"2024-01-31T05:20:41.990495+0000","last_became_active":"2024-01-31T05:20:39.708918+0000","last_became_peered":"2024-01-31T05:20:39.708918+0000","last_unstale":"2024-01-31T05:20:41.990495+0000","last_undegraded":"2024-01-31T05:20:41.990495+0000","last_fullsized":"2024-01-31T05:20:41.990495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:55:51.545144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712901+0000","last_change":"2024-01-31T05:20:40.729677+0000","last_active":"2024-01-31T05:20:41.712901+0000","last_peered":"2024-01-31T05:20:41.712901+0000","last_clean":"2024-01-31T05:20:41.712901+0000","last_became_active":"2024-01-31T05:20:40.727880+0000","last_became_peered":"2024-01-31T05:20:40.727880+0000","last_unstale":"2024-01-31T05:20:41.712901+0000","last_undegraded":"2024-01-31T05:20:41.712901+0000","last_fullsized":"2024-01-31T05:20:41.712901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:49:55.935842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712568+0000","last_change":"2024-01-31T05:20:39.711310+0000","last_active":"2024-01-31T05:20:41.712568+0000","last_peered":"2024-01-31T05:20:41.712568+0000","last_clean":"2024-01-31T05:20:41.712568+0000","last_became_active":"2024-01-31T05:20:39.710108+0000","last_became_peered":"2024-01-31T05:20:39.710108+0000","last_unstale":"2024-01-31T05:20:41.712568+0000","last_undegraded":"2024-01-31T05:20:41.712568+0000","last_fullsized":"2024-01-31T05:20:41.712568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:03:25.550777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990347+0000","last_change":"2024-01-31T05:20:40.722095+0000","last_active":"2024-01-31T05:20:41.990347+0000","last_peered":"2024-01-31T05:20:41.990347+0000","last_clean":"2024-01-31T05:20:41.990347+0000","last_became_active":"2024-01-31T05:20:40.721778+0000","last_became_peered":"2024-01-31T05:20:40.721778+0000","last_unstale":"2024-01-31T05:20:41.990347+0000","last_undegraded":"2024-01-31T05:20:41.990347+0000","last_fullsized":"2024-01-31T05:20:41.990347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:48:04.284745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.483270+0000","last_change":"2024-01-31T05:20:39.705704+0000","last_active":"2024-01-31T05:20:43.483270+0000","last_peered":"2024-01-31T05:20:43.483270+0000","last_clean":"2024-01-31T05:20:43.483270+0000","last_became_active":"2024-01-31T05:20:39.705304+0000","last_became_peered":"2024-01-31T05:20:39.705304+0000","last_unstale":"2024-01-31T05:20:43.483270+0000","last_undegraded":"2024-01-31T05:20:43.483270+0000","last_fullsized":"2024-01-31T05:20:43.483270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:01:40.116085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712765+0000","last_change":"2024-01-31T05:20:40.728028+0000","last_active":"2024-01-31T05:20:41.712765+0000","last_peered":"2024-01-31T05:20:41.712765+0000","last_clean":"2024-01-31T05:20:41.712765+0000","last_became_active":"2024-01-31T05:20:40.727197+0000","last_became_peered":"2024-01-31T05:20:40.727197+0000","last_unstale":"2024-01-31T05:20:41.712765+0000","last_undegraded":"2024-01-31T05:20:41.712765+0000","last_fullsized":"2024-01-31T05:20:41.712765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:12:19.749802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.698962+0000","last_change":"2024-01-31T05:20:39.705157+0000","last_active":"2024-01-31T05:20:40.698962+0000","last_peered":"2024-01-31T05:20:40.698962+0000","last_clean":"2024-01-31T05:20:40.698962+0000","last_became_active":"2024-01-31T05:20:39.704794+0000","last_became_peered":"2024-01-31T05:20:39.704794+0000","last_unstale":"2024-01-31T05:20:40.698962+0000","last_undegraded":"2024-01-31T05:20:40.698962+0000","last_fullsized":"2024-01-31T05:20:40.698962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:21:20.314515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.711868+0000","last_change":"2024-01-31T05:20:40.723906+0000","last_active":"2024-01-31T05:20:41.711868+0000","last_peered":"2024-01-31T05:20:41.711868+0000","last_clean":"2024-01-31T05:20:41.711868+0000","last_became_active":"2024-01-31T05:20:40.723612+0000","last_became_peered":"2024-01-31T05:20:40.723612+0000","last_unstale":"2024-01-31T05:20:41.711868+0000","last_undegraded":"2024-01-31T05:20:41.711868+0000","last_fullsized":"2024-01-31T05:20:41.711868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:42:42.509983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712228+0000","last_change":"2024-01-31T05:20:39.709315+0000","last_active":"2024-01-31T05:20:41.712228+0000","last_peered":"2024-01-31T05:20:41.712228+0000","last_clean":"2024-01-31T05:20:41.712228+0000","last_became_active":"2024-01-31T05:20:39.709033+0000","last_became_peered":"2024-01-31T05:20:39.709033+0000","last_unstale":"2024-01-31T05:20:41.712228+0000","last_undegraded":"2024-01-31T05:20:41.712228+0000","last_fullsized":"2024-01-31T05:20:41.712228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:43:29.540029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989378+0000","last_change":"2024-01-31T05:20:40.717396+0000","last_active":"2024-01-31T05:20:41.989378+0000","last_peered":"2024-01-31T05:20:41.989378+0000","last_clean":"2024-01-31T05:20:41.989378+0000","last_became_active":"2024-01-31T05:20:40.717076+0000","last_became_peered":"2024-01-31T05:20:40.717076+0000","last_unstale":"2024-01-31T05:20:41.989378+0000","last_undegraded":"2024-01-31T05:20:41.989378+0000","last_fullsized":"2024-01-31T05:20:41.989378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:42:51.667070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:44.484240+0000","last_change":"2024-01-31T05:20:39.716940+0000","last_active":"2024-01-31T05:20:44.484240+0000","last_peered":"2024-01-31T05:20:44.484240+0000","last_clean":"2024-01-31T05:20:44.484240+0000","last_became_active":"2024-01-31T05:20:39.716691+0000","last_became_peered":"2024-01-31T05:20:39.716691+0000","last_unstale":"2024-01-31T05:20:44.484240+0000","last_undegraded":"2024-01-31T05:20:44.484240+0000","last_fullsized":"2024-01-31T05:20:44.484240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:38:23.786807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712747+0000","last_change":"2024-01-31T05:20:40.727618+0000","last_active":"2024-01-31T05:20:41.712747+0000","last_peered":"2024-01-31T05:20:41.712747+0000","last_clean":"2024-01-31T05:20:41.712747+0000","last_became_active":"2024-01-31T05:20:40.727282+0000","last_became_peered":"2024-01-31T05:20:40.727282+0000","last_unstale":"2024-01-31T05:20:41.712747+0000","last_undegraded":"2024-01-31T05:20:41.712747+0000","last_fullsized":"2024-01-31T05:20:41.712747+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:07:12.416575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.470336+0000","last_change":"2024-01-31T05:20:39.709584+0000","last_active":"2024-01-31T05:20:42.470336+0000","last_peered":"2024-01-31T05:20:42.470336+0000","last_clean":"2024-01-31T05:20:42.470336+0000","last_became_active":"2024-01-31T05:20:39.708660+0000","last_became_peered":"2024-01-31T05:20:39.708660+0000","last_unstale":"2024-01-31T05:20:42.470336+0000","last_undegraded":"2024-01-31T05:20:42.470336+0000","last_fullsized":"2024-01-31T05:20:42.470336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:21:06.450913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.734563+0000","last_change":"2024-01-31T05:20:40.734040+0000","last_active":"2024-01-31T05:20:40.734563+0000","last_peered":"2024-01-31T05:20:40.734563+0000","last_clean":"2024-01-31T05:20:40.734563+0000","last_became_active":"2024-01-31T05:20:40.733307+0000","last_became_peered":"2024-01-31T05:20:40.733307+0000","last_unstale":"2024-01-31T05:20:40.734563+0000","last_undegraded":"2024-01-31T05:20:40.734563+0000","last_fullsized":"2024-01-31T05:20:40.734563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:32:22.657358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712952+0000","last_change":"2024-01-31T05:20:39.708004+0000","last_active":"2024-01-31T05:20:41.712952+0000","last_peered":"2024-01-31T05:20:41.712952+0000","last_clean":"2024-01-31T05:20:41.712952+0000","last_became_active":"2024-01-31T05:20:39.707681+0000","last_became_peered":"2024-01-31T05:20:39.707681+0000","last_unstale":"2024-01-31T05:20:41.712952+0000","last_undegraded":"2024-01-31T05:20:41.712952+0000","last_fullsized":"2024-01-31T05:20:41.712952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:24:10.003803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712652+0000","last_change":"2024-01-31T05:20:40.714449+0000","last_active":"2024-01-31T05:20:41.712652+0000","last_peered":"2024-01-31T05:20:41.712652+0000","last_clean":"2024-01-31T05:20:41.712652+0000","last_became_active":"2024-01-31T05:20:40.714064+0000","last_became_peered":"2024-01-31T05:20:40.714064+0000","last_unstale":"2024-01-31T05:20:41.712652+0000","last_undegraded":"2024-01-31T05:20:41.712652+0000","last_fullsized":"2024-01-31T05:20:41.712652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:17:04.803852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:45.487844+0000","last_change":"2024-01-31T05:20:39.709050+0000","last_active":"2024-01-31T05:20:45.487844+0000","last_peered":"2024-01-31T05:20:45.487844+0000","last_clean":"2024-01-31T05:20:45.487844+0000","last_became_active":"2024-01-31T05:20:39.708674+0000","last_became_peered":"2024-01-31T05:20:39.708674+0000","last_unstale":"2024-01-31T05:20:45.487844+0000","last_undegraded":"2024-01-31T05:20:45.487844+0000","last_fullsized":"2024-01-31T05:20:45.487844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:08:34.994683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20894,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":23,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990562+0000","last_change":"2024-01-31T05:20:40.727191+0000","last_active":"2024-01-31T05:20:41.990562+0000","last_peered":"2024-01-31T05:20:41.990562+0000","last_clean":"2024-01-31T05:20:41.990562+0000","last_became_active":"2024-01-31T05:20:40.725114+0000","last_became_peered":"2024-01-31T05:20:40.725114+0000","last_unstale":"2024-01-31T05:20:41.990562+0000","last_undegraded":"2024-01-31T05:20:41.990562+0000","last_fullsized":"2024-01-31T05:20:41.990562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:00.758702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.482923+0000","last_change":"2024-01-31T05:20:39.719349+0000","last_active":"2024-01-31T05:20:43.482923+0000","last_peered":"2024-01-31T05:20:43.482923+0000","last_clean":"2024-01-31T05:20:43.482923+0000","last_became_active":"2024-01-31T05:20:39.718918+0000","last_became_peered":"2024-01-31T05:20:39.718918+0000","last_unstale":"2024-01-31T05:20:43.482923+0000","last_undegraded":"2024-01-31T05:20:43.482923+0000","last_fullsized":"2024-01-31T05:20:43.482923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:47:48.521638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.717336+0000","last_change":"2024-01-31T05:20:40.716868+0000","last_active":"2024-01-31T05:20:40.717336+0000","last_peered":"2024-01-31T05:20:40.717336+0000","last_clean":"2024-01-31T05:20:40.717336+0000","last_became_active":"2024-01-31T05:20:40.716519+0000","last_became_peered":"2024-01-31T05:20:40.716519+0000","last_unstale":"2024-01-31T05:20:40.717336+0000","last_undegraded":"2024-01-31T05:20:40.717336+0000","last_fullsized":"2024-01-31T05:20:40.717336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:30:42.910337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712615+0000","last_change":"2024-01-31T05:20:39.716631+0000","last_active":"2024-01-31T05:20:41.712615+0000","last_peered":"2024-01-31T05:20:41.712615+0000","last_clean":"2024-01-31T05:20:41.712615+0000","last_became_active":"2024-01-31T05:20:39.716331+0000","last_became_peered":"2024-01-31T05:20:39.716331+0000","last_unstale":"2024-01-31T05:20:41.712615+0000","last_undegraded":"2024-01-31T05:20:41.712615+0000","last_fullsized":"2024-01-31T05:20:41.712615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:05:59.555000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989669+0000","last_change":"2024-01-31T05:20:40.721548+0000","last_active":"2024-01-31T05:20:41.989669+0000","last_peered":"2024-01-31T05:20:41.989669+0000","last_clean":"2024-01-31T05:20:41.989669+0000","last_became_active":"2024-01-31T05:20:40.720296+0000","last_became_peered":"2024-01-31T05:20:40.720296+0000","last_unstale":"2024-01-31T05:20:41.989669+0000","last_undegraded":"2024-01-31T05:20:41.989669+0000","last_fullsized":"2024-01-31T05:20:41.989669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:04:08.406863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.887065+0000","last_change":"2024-01-31T05:20:39.707644+0000","last_active":"2024-01-31T05:20:41.887065+0000","last_peered":"2024-01-31T05:20:41.887065+0000","last_clean":"2024-01-31T05:20:41.887065+0000","last_became_active":"2024-01-31T05:20:39.707334+0000","last_became_peered":"2024-01-31T05:20:39.707334+0000","last_unstale":"2024-01-31T05:20:41.887065+0000","last_undegraded":"2024-01-31T05:20:41.887065+0000","last_fullsized":"2024-01-31T05:20:41.887065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:13:25.630415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.988922+0000","last_change":"2024-01-31T05:20:40.715835+0000","last_active":"2024-01-31T05:20:41.988922+0000","last_peered":"2024-01-31T05:20:41.988922+0000","last_clean":"2024-01-31T05:20:41.988922+0000","last_became_active":"2024-01-31T05:20:40.715467+0000","last_became_peered":"2024-01-31T05:20:40.715467+0000","last_unstale":"2024-01-31T05:20:41.988922+0000","last_undegraded":"2024-01-31T05:20:41.988922+0000","last_fullsized":"2024-01-31T05:20:41.988922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:40:33.583238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.483883+0000","last_change":"2024-01-31T05:20:39.717929+0000","last_active":"2024-01-31T05:20:43.483883+0000","last_peered":"2024-01-31T05:20:43.483883+0000","last_clean":"2024-01-31T05:20:43.483883+0000","last_became_active":"2024-01-31T05:20:39.717644+0000","last_became_peered":"2024-01-31T05:20:39.717644+0000","last_unstale":"2024-01-31T05:20:43.483883+0000","last_undegraded":"2024-01-31T05:20:43.483883+0000","last_fullsized":"2024-01-31T05:20:43.483883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:24:41.296706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712394+0000","last_change":"2024-01-31T05:20:40.726182+0000","last_active":"2024-01-31T05:20:41.712394+0000","last_peered":"2024-01-31T05:20:41.712394+0000","last_clean":"2024-01-31T05:20:41.712394+0000","last_became_active":"2024-01-31T05:20:40.725858+0000","last_became_peered":"2024-01-31T05:20:40.725858+0000","last_unstale":"2024-01-31T05:20:41.712394+0000","last_undegraded":"2024-01-31T05:20:41.712394+0000","last_fullsized":"2024-01-31T05:20:41.712394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:27.209744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990875+0000","last_change":"2024-01-31T05:20:39.706132+0000","last_active":"2024-01-31T05:20:41.990875+0000","last_peered":"2024-01-31T05:20:41.990875+0000","last_clean":"2024-01-31T05:20:41.990875+0000","last_became_active":"2024-01-31T05:20:39.705791+0000","last_became_peered":"2024-01-31T05:20:39.705791+0000","last_unstale":"2024-01-31T05:20:41.990875+0000","last_undegraded":"2024-01-31T05:20:41.990875+0000","last_fullsized":"2024-01-31T05:20:41.990875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:54:57.857011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989542+0000","last_change":"2024-01-31T05:20:40.720232+0000","last_active":"2024-01-31T05:20:41.989542+0000","last_peered":"2024-01-31T05:20:41.989542+0000","last_clean":"2024-01-31T05:20:41.989542+0000","last_became_active":"2024-01-31T05:20:40.719917+0000","last_became_peered":"2024-01-31T05:20:40.719917+0000","last_unstale":"2024-01-31T05:20:41.989542+0000","last_undegraded":"2024-01-31T05:20:41.989542+0000","last_fullsized":"2024-01-31T05:20:41.989542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:24:25.931378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989744+0000","last_change":"2024-01-31T05:20:39.710195+0000","last_active":"2024-01-31T05:20:41.989744+0000","last_peered":"2024-01-31T05:20:41.989744+0000","last_clean":"2024-01-31T05:20:41.989744+0000","last_became_active":"2024-01-31T05:20:39.709787+0000","last_became_peered":"2024-01-31T05:20:39.709787+0000","last_unstale":"2024-01-31T05:20:41.989744+0000","last_undegraded":"2024-01-31T05:20:41.989744+0000","last_fullsized":"2024-01-31T05:20:41.989744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:28:36.283468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990259+0000","last_change":"2024-01-31T05:20:40.716243+0000","last_active":"2024-01-31T05:20:41.990259+0000","last_peered":"2024-01-31T05:20:41.990259+0000","last_clean":"2024-01-31T05:20:41.990259+0000","last_became_active":"2024-01-31T05:20:40.715927+0000","last_became_peered":"2024-01-31T05:20:40.715927+0000","last_unstale":"2024-01-31T05:20:41.990259+0000","last_undegraded":"2024-01-31T05:20:41.990259+0000","last_fullsized":"2024-01-31T05:20:41.990259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:20:19.289014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989133+0000","last_change":"2024-01-31T05:20:39.712333+0000","last_active":"2024-01-31T05:20:41.989133+0000","last_peered":"2024-01-31T05:20:41.989133+0000","last_clean":"2024-01-31T05:20:41.989133+0000","last_became_active":"2024-01-31T05:20:39.712008+0000","last_became_peered":"2024-01-31T05:20:39.712008+0000","last_unstale":"2024-01-31T05:20:41.989133+0000","last_undegraded":"2024-01-31T05:20:41.989133+0000","last_fullsized":"2024-01-31T05:20:41.989133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:04:52.962130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990046+0000","last_change":"2024-01-31T05:20:40.725131+0000","last_active":"2024-01-31T05:20:41.990046+0000","last_peered":"2024-01-31T05:20:41.990046+0000","last_clean":"2024-01-31T05:20:41.990046+0000","last_became_active":"2024-01-31T05:20:40.724797+0000","last_became_peered":"2024-01-31T05:20:40.724797+0000","last_unstale":"2024-01-31T05:20:41.990046+0000","last_undegraded":"2024-01-31T05:20:41.990046+0000","last_fullsized":"2024-01-31T05:20:41.990046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:23:26.535023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712887+0000","last_change":"2024-01-31T05:20:40.726857+0000","last_active":"2024-01-31T05:20:41.712887+0000","last_peered":"2024-01-31T05:20:41.712887+0000","last_clean":"2024-01-31T05:20:41.712887+0000","last_became_active":"2024-01-31T05:20:40.726545+0000","last_became_peered":"2024-01-31T05:20:40.726545+0000","last_unstale":"2024-01-31T05:20:41.712887+0000","last_undegraded":"2024-01-31T05:20:41.712887+0000","last_fullsized":"2024-01-31T05:20:41.712887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:43:55.852474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.698763+0000","last_change":"2024-01-31T05:20:39.713920+0000","last_active":"2024-01-31T05:20:40.698763+0000","last_peered":"2024-01-31T05:20:40.698763+0000","last_clean":"2024-01-31T05:20:40.698763+0000","last_became_active":"2024-01-31T05:20:39.713582+0000","last_became_peered":"2024-01-31T05:20:39.713582+0000","last_unstale":"2024-01-31T05:20:40.698763+0000","last_undegraded":"2024-01-31T05:20:40.698763+0000","last_fullsized":"2024-01-31T05:20:40.698763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:20:18.932993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712051+0000","last_change":"2024-01-31T05:20:40.720442+0000","last_active":"2024-01-31T05:20:41.712051+0000","last_peered":"2024-01-31T05:20:41.712051+0000","last_clean":"2024-01-31T05:20:41.712051+0000","last_became_active":"2024-01-31T05:20:40.720114+0000","last_became_peered":"2024-01-31T05:20:40.720114+0000","last_unstale":"2024-01-31T05:20:41.712051+0000","last_undegraded":"2024-01-31T05:20:41.712051+0000","last_fullsized":"2024-01-31T05:20:41.712051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:11:01.536820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.466410+0000","last_change":"2024-01-31T05:20:39.706791+0000","last_active":"2024-01-31T05:20:42.466410+0000","last_peered":"2024-01-31T05:20:42.466410+0000","last_clean":"2024-01-31T05:20:42.466410+0000","last_became_active":"2024-01-31T05:20:39.706447+0000","last_became_peered":"2024-01-31T05:20:39.706447+0000","last_unstale":"2024-01-31T05:20:42.466410+0000","last_undegraded":"2024-01-31T05:20:42.466410+0000","last_fullsized":"2024-01-31T05:20:42.466410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:41:05.118138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.988972+0000","last_change":"2024-01-31T05:19:28.446661+0000","last_active":"2024-01-31T05:20:41.988972+0000","last_peered":"2024-01-31T05:20:41.988972+0000","last_clean":"2024-01-31T05:20:41.988972+0000","last_became_active":"2024-01-31T05:19:28.446238+0000","last_became_peered":"2024-01-31T05:19:28.446238+0000","last_unstale":"2024-01-31T05:20:41.988972+0000","last_undegraded":"2024-01-31T05:20:41.988972+0000","last_fullsized":"2024-01-31T05:20:41.988972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:16:11.415995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:16:11.415995+0000","last_clean_scrub_stamp":"2024-01-31T05:16:11.415995+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:56:53.665942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.715768+0000","last_change":"2024-01-31T05:20:40.714343+0000","last_active":"2024-01-31T05:20:40.715768+0000","last_peered":"2024-01-31T05:20:40.715768+0000","last_clean":"2024-01-31T05:20:40.715768+0000","last_became_active":"2024-01-31T05:20:40.713820+0000","last_became_peered":"2024-01-31T05:20:40.713820+0000","last_unstale":"2024-01-31T05:20:40.715768+0000","last_undegraded":"2024-01-31T05:20:40.715768+0000","last_fullsized":"2024-01-31T05:20:40.715768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:46:29.228809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.482100+0000","last_change":"2024-01-31T05:20:39.712068+0000","last_active":"2024-01-31T05:20:43.482100+0000","last_peered":"2024-01-31T05:20:43.482100+0000","last_clean":"2024-01-31T05:20:43.482100+0000","last_became_active":"2024-01-31T05:20:39.711641+0000","last_became_peered":"2024-01-31T05:20:39.711641+0000","last_unstale":"2024-01-31T05:20:43.482100+0000","last_undegraded":"2024-01-31T05:20:43.482100+0000","last_fullsized":"2024-01-31T05:20:43.482100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:27.199351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.471461+0000","last_change":"2024-01-31T05:20:39.716003+0000","last_active":"2024-01-31T05:20:42.471461+0000","last_peered":"2024-01-31T05:20:42.471461+0000","last_clean":"2024-01-31T05:20:42.471461+0000","last_became_active":"2024-01-31T05:20:39.715653+0000","last_became_peered":"2024-01-31T05:20:39.715653+0000","last_unstale":"2024-01-31T05:20:42.471461+0000","last_undegraded":"2024-01-31T05:20:42.471461+0000","last_fullsized":"2024-01-31T05:20:42.471461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:45:56.886077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.713156+0000","last_change":"2024-01-31T05:20:40.712298+0000","last_active":"2024-01-31T05:20:40.713156+0000","last_peered":"2024-01-31T05:20:40.713156+0000","last_clean":"2024-01-31T05:20:40.713156+0000","last_became_active":"2024-01-31T05:20:40.711907+0000","last_became_peered":"2024-01-31T05:20:40.711907+0000","last_unstale":"2024-01-31T05:20:40.713156+0000","last_undegraded":"2024-01-31T05:20:40.713156+0000","last_fullsized":"2024-01-31T05:20:40.713156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:17:52.282149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:45.000870+0000","last_change":"2024-01-31T05:20:39.720639+0000","last_active":"2024-01-31T05:20:45.000870+0000","last_peered":"2024-01-31T05:20:45.000870+0000","last_clean":"2024-01-31T05:20:45.000870+0000","last_became_active":"2024-01-31T05:20:39.720302+0000","last_became_peered":"2024-01-31T05:20:39.720302+0000","last_unstale":"2024-01-31T05:20:45.000870+0000","last_undegraded":"2024-01-31T05:20:45.000870+0000","last_fullsized":"2024-01-31T05:20:45.000870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:11:31.507908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712008+0000","last_change":"2024-01-31T05:20:40.725324+0000","last_active":"2024-01-31T05:20:41.712008+0000","last_peered":"2024-01-31T05:20:41.712008+0000","last_clean":"2024-01-31T05:20:41.712008+0000","last_became_active":"2024-01-31T05:20:40.725027+0000","last_became_peered":"2024-01-31T05:20:40.725027+0000","last_unstale":"2024-01-31T05:20:41.712008+0000","last_undegraded":"2024-01-31T05:20:41.712008+0000","last_fullsized":"2024-01-31T05:20:41.712008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:06:24.129276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.471871+0000","last_change":"2024-01-31T05:20:39.705805+0000","last_active":"2024-01-31T05:20:42.471871+0000","last_peered":"2024-01-31T05:20:42.471871+0000","last_clean":"2024-01-31T05:20:42.471871+0000","last_became_active":"2024-01-31T05:20:39.705447+0000","last_became_peered":"2024-01-31T05:20:39.705447+0000","last_unstale":"2024-01-31T05:20:42.471871+0000","last_undegraded":"2024-01-31T05:20:42.471871+0000","last_fullsized":"2024-01-31T05:20:42.471871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:24:47.856773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.734557+0000","last_change":"2024-01-31T05:20:40.733818+0000","last_active":"2024-01-31T05:20:40.734557+0000","last_peered":"2024-01-31T05:20:40.734557+0000","last_clean":"2024-01-31T05:20:40.734557+0000","last_became_active":"2024-01-31T05:20:40.733272+0000","last_became_peered":"2024-01-31T05:20:40.733272+0000","last_unstale":"2024-01-31T05:20:40.734557+0000","last_undegraded":"2024-01-31T05:20:40.734557+0000","last_fullsized":"2024-01-31T05:20:40.734557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:27:19.767212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.471935+0000","last_change":"2024-01-31T05:20:39.707920+0000","last_active":"2024-01-31T05:20:42.471935+0000","last_peered":"2024-01-31T05:20:42.471935+0000","last_clean":"2024-01-31T05:20:42.471935+0000","last_became_active":"2024-01-31T05:20:39.707612+0000","last_became_peered":"2024-01-31T05:20:39.707612+0000","last_unstale":"2024-01-31T05:20:42.471935+0000","last_undegraded":"2024-01-31T05:20:42.471935+0000","last_fullsized":"2024-01-31T05:20:42.471935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:57.522664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989302+0000","last_change":"2024-01-31T05:20:40.722376+0000","last_active":"2024-01-31T05:20:41.989302+0000","last_peered":"2024-01-31T05:20:41.989302+0000","last_clean":"2024-01-31T05:20:41.989302+0000","last_became_active":"2024-01-31T05:20:40.722055+0000","last_became_peered":"2024-01-31T05:20:40.722055+0000","last_unstale":"2024-01-31T05:20:41.989302+0000","last_undegraded":"2024-01-31T05:20:41.989302+0000","last_fullsized":"2024-01-31T05:20:41.989302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:11:47.198866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.466727+0000","last_change":"2024-01-31T05:20:39.714498+0000","last_active":"2024-01-31T05:20:42.466727+0000","last_peered":"2024-01-31T05:20:42.466727+0000","last_clean":"2024-01-31T05:20:42.466727+0000","last_became_active":"2024-01-31T05:20:39.714179+0000","last_became_peered":"2024-01-31T05:20:39.714179+0000","last_unstale":"2024-01-31T05:20:42.466727+0000","last_undegraded":"2024-01-31T05:20:42.466727+0000","last_fullsized":"2024-01-31T05:20:42.466727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:42:55.913696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989706+0000","last_change":"2024-01-31T05:20:40.716878+0000","last_active":"2024-01-31T05:20:41.989706+0000","last_peered":"2024-01-31T05:20:41.989706+0000","last_clean":"2024-01-31T05:20:41.989706+0000","last_became_active":"2024-01-31T05:20:40.715998+0000","last_became_peered":"2024-01-31T05:20:40.715998+0000","last_unstale":"2024-01-31T05:20:41.989706+0000","last_undegraded":"2024-01-31T05:20:41.989706+0000","last_fullsized":"2024-01-31T05:20:41.989706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:59:31.030147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989446+0000","last_change":"2024-01-31T05:20:39.719267+0000","last_active":"2024-01-31T05:20:41.989446+0000","last_peered":"2024-01-31T05:20:41.989446+0000","last_clean":"2024-01-31T05:20:41.989446+0000","last_became_active":"2024-01-31T05:20:39.718964+0000","last_became_peered":"2024-01-31T05:20:39.718964+0000","last_unstale":"2024-01-31T05:20:41.989446+0000","last_undegraded":"2024-01-31T05:20:41.989446+0000","last_fullsized":"2024-01-31T05:20:41.989446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:31.150796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.711786+0000","last_change":"2024-01-31T05:20:40.727530+0000","last_active":"2024-01-31T05:20:41.711786+0000","last_peered":"2024-01-31T05:20:41.711786+0000","last_clean":"2024-01-31T05:20:41.711786+0000","last_became_active":"2024-01-31T05:20:40.727016+0000","last_became_peered":"2024-01-31T05:20:40.727016+0000","last_unstale":"2024-01-31T05:20:41.711786+0000","last_undegraded":"2024-01-31T05:20:41.711786+0000","last_fullsized":"2024-01-31T05:20:41.711786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:15:25.851713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.472142+0000","last_change":"2024-01-31T05:20:39.711083+0000","last_active":"2024-01-31T05:20:42.472142+0000","last_peered":"2024-01-31T05:20:42.472142+0000","last_clean":"2024-01-31T05:20:42.472142+0000","last_became_active":"2024-01-31T05:20:39.710732+0000","last_became_peered":"2024-01-31T05:20:39.710732+0000","last_unstale":"2024-01-31T05:20:42.472142+0000","last_undegraded":"2024-01-31T05:20:42.472142+0000","last_fullsized":"2024-01-31T05:20:42.472142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:43.846910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989238+0000","last_change":"2024-01-31T05:20:40.723667+0000","last_active":"2024-01-31T05:20:41.989238+0000","last_peered":"2024-01-31T05:20:41.989238+0000","last_clean":"2024-01-31T05:20:41.989238+0000","last_became_active":"2024-01-31T05:20:40.723343+0000","last_became_peered":"2024-01-31T05:20:40.723343+0000","last_unstale":"2024-01-31T05:20:41.989238+0000","last_undegraded":"2024-01-31T05:20:41.989238+0000","last_fullsized":"2024-01-31T05:20:41.989238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:12.983137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.483898+0000","last_change":"2024-01-31T05:20:39.717263+0000","last_active":"2024-01-31T05:20:43.483898+0000","last_peered":"2024-01-31T05:20:43.483898+0000","last_clean":"2024-01-31T05:20:43.483898+0000","last_became_active":"2024-01-31T05:20:39.716666+0000","last_became_peered":"2024-01-31T05:20:39.716666+0000","last_unstale":"2024-01-31T05:20:43.483898+0000","last_undegraded":"2024-01-31T05:20:43.483898+0000","last_fullsized":"2024-01-31T05:20:43.483898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:10:41.789503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989721+0000","last_change":"2024-01-31T05:20:40.721728+0000","last_active":"2024-01-31T05:20:41.989721+0000","last_peered":"2024-01-31T05:20:41.989721+0000","last_clean":"2024-01-31T05:20:41.989721+0000","last_became_active":"2024-01-31T05:20:40.717082+0000","last_became_peered":"2024-01-31T05:20:40.717082+0000","last_unstale":"2024-01-31T05:20:41.989721+0000","last_undegraded":"2024-01-31T05:20:41.989721+0000","last_fullsized":"2024-01-31T05:20:41.989721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:15:55.823060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712785+0000","last_change":"2024-01-31T05:20:39.715450+0000","last_active":"2024-01-31T05:20:41.712785+0000","last_peered":"2024-01-31T05:20:41.712785+0000","last_clean":"2024-01-31T05:20:41.712785+0000","last_became_active":"2024-01-31T05:20:39.715140+0000","last_became_peered":"2024-01-31T05:20:39.715140+0000","last_unstale":"2024-01-31T05:20:41.712785+0000","last_undegraded":"2024-01-31T05:20:41.712785+0000","last_fullsized":"2024-01-31T05:20:41.712785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:34:01.299228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989819+0000","last_change":"2024-01-31T05:20:40.723855+0000","last_active":"2024-01-31T05:20:41.989819+0000","last_peered":"2024-01-31T05:20:41.989819+0000","last_clean":"2024-01-31T05:20:41.989819+0000","last_became_active":"2024-01-31T05:20:40.723532+0000","last_became_peered":"2024-01-31T05:20:40.723532+0000","last_unstale":"2024-01-31T05:20:41.989819+0000","last_undegraded":"2024-01-31T05:20:41.989819+0000","last_fullsized":"2024-01-31T05:20:41.989819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:48:59.700145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.731877+0000","last_change":"2024-01-31T05:20:39.713941+0000","last_active":"2024-01-31T05:20:40.731877+0000","last_peered":"2024-01-31T05:20:40.731877+0000","last_clean":"2024-01-31T05:20:40.731877+0000","last_became_active":"2024-01-31T05:20:39.713610+0000","last_became_peered":"2024-01-31T05:20:39.713610+0000","last_unstale":"2024-01-31T05:20:40.731877+0000","last_undegraded":"2024-01-31T05:20:40.731877+0000","last_fullsized":"2024-01-31T05:20:40.731877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:48:13.722639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989566+0000","last_change":"2024-01-31T05:20:40.716485+0000","last_active":"2024-01-31T05:20:41.989566+0000","last_peered":"2024-01-31T05:20:41.989566+0000","last_clean":"2024-01-31T05:20:41.989566+0000","last_became_active":"2024-01-31T05:20:40.716177+0000","last_became_peered":"2024-01-31T05:20:40.716177+0000","last_unstale":"2024-01-31T05:20:41.989566+0000","last_undegraded":"2024-01-31T05:20:41.989566+0000","last_fullsized":"2024-01-31T05:20:41.989566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:25:37.359617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989885+0000","last_change":"2024-01-31T05:20:40.710376+0000","last_active":"2024-01-31T05:20:41.989885+0000","last_peered":"2024-01-31T05:20:41.989885+0000","last_clean":"2024-01-31T05:20:41.989885+0000","last_became_active":"2024-01-31T05:20:40.710017+0000","last_became_peered":"2024-01-31T05:20:40.710017+0000","last_unstale":"2024-01-31T05:20:41.989885+0000","last_undegraded":"2024-01-31T05:20:41.989885+0000","last_fullsized":"2024-01-31T05:20:41.989885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:48:37.519561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989391+0000","last_change":"2024-01-31T05:20:40.727375+0000","last_active":"2024-01-31T05:20:41.989391+0000","last_peered":"2024-01-31T05:20:41.989391+0000","last_clean":"2024-01-31T05:20:41.989391+0000","last_became_active":"2024-01-31T05:20:40.725995+0000","last_became_peered":"2024-01-31T05:20:40.725995+0000","last_unstale":"2024-01-31T05:20:41.989391+0000","last_undegraded":"2024-01-31T05:20:41.989391+0000","last_fullsized":"2024-01-31T05:20:41.989391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:13:03.392972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989342+0000","last_change":"2024-01-31T05:20:40.725974+0000","last_active":"2024-01-31T05:20:41.989342+0000","last_peered":"2024-01-31T05:20:41.989342+0000","last_clean":"2024-01-31T05:20:41.989342+0000","last_became_active":"2024-01-31T05:20:40.725586+0000","last_became_peered":"2024-01-31T05:20:40.725586+0000","last_unstale":"2024-01-31T05:20:41.989342+0000","last_undegraded":"2024-01-31T05:20:41.989342+0000","last_fullsized":"2024-01-31T05:20:41.989342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:03:06.413502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.734533+0000","last_change":"2024-01-31T05:20:40.733524+0000","last_active":"2024-01-31T05:20:40.734533+0000","last_peered":"2024-01-31T05:20:40.734533+0000","last_clean":"2024-01-31T05:20:40.734533+0000","last_became_active":"2024-01-31T05:20:40.733162+0000","last_became_peered":"2024-01-31T05:20:40.733162+0000","last_unstale":"2024-01-31T05:20:40.734533+0000","last_undegraded":"2024-01-31T05:20:40.734533+0000","last_fullsized":"2024-01-31T05:20:40.734533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:00:56.321364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.717753+0000","last_change":"2024-01-31T05:20:40.715460+0000","last_active":"2024-01-31T05:20:40.717753+0000","last_peered":"2024-01-31T05:20:40.717753+0000","last_clean":"2024-01-31T05:20:40.717753+0000","last_became_active":"2024-01-31T05:20:40.715103+0000","last_became_peered":"2024-01-31T05:20:40.715103+0000","last_unstale":"2024-01-31T05:20:40.717753+0000","last_undegraded":"2024-01-31T05:20:40.717753+0000","last_fullsized":"2024-01-31T05:20:40.717753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:39:31.084636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989806+0000","last_change":"2024-01-31T05:20:40.714317+0000","last_active":"2024-01-31T05:20:41.989806+0000","last_peered":"2024-01-31T05:20:41.989806+0000","last_clean":"2024-01-31T05:20:41.989806+0000","last_became_active":"2024-01-31T05:20:40.714008+0000","last_became_peered":"2024-01-31T05:20:40.714008+0000","last_unstale":"2024-01-31T05:20:41.989806+0000","last_undegraded":"2024-01-31T05:20:41.989806+0000","last_fullsized":"2024-01-31T05:20:41.989806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:36:39.740278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990314+0000","last_change":"2024-01-31T05:20:40.710901+0000","last_active":"2024-01-31T05:20:41.990314+0000","last_peered":"2024-01-31T05:20:41.990314+0000","last_clean":"2024-01-31T05:20:41.990314+0000","last_became_active":"2024-01-31T05:20:40.710412+0000","last_became_peered":"2024-01-31T05:20:40.710412+0000","last_unstale":"2024-01-31T05:20:41.990314+0000","last_undegraded":"2024-01-31T05:20:41.990314+0000","last_fullsized":"2024-01-31T05:20:41.990314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:39.966723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.711934+0000","last_change":"2024-01-31T05:20:40.732753+0000","last_active":"2024-01-31T05:20:41.711934+0000","last_peered":"2024-01-31T05:20:41.711934+0000","last_clean":"2024-01-31T05:20:41.711934+0000","last_became_active":"2024-01-31T05:20:40.732330+0000","last_became_peered":"2024-01-31T05:20:40.732330+0000","last_unstale":"2024-01-31T05:20:41.711934+0000","last_undegraded":"2024-01-31T05:20:41.711934+0000","last_fullsized":"2024-01-31T05:20:41.711934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:54:01.933393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712468+0000","last_change":"2024-01-31T05:20:40.723129+0000","last_active":"2024-01-31T05:20:41.712468+0000","last_peered":"2024-01-31T05:20:41.712468+0000","last_clean":"2024-01-31T05:20:41.712468+0000","last_became_active":"2024-01-31T05:20:40.722796+0000","last_became_peered":"2024-01-31T05:20:40.722796+0000","last_unstale":"2024-01-31T05:20:41.712468+0000","last_undegraded":"2024-01-31T05:20:41.712468+0000","last_fullsized":"2024-01-31T05:20:41.712468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:18.120495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.711542+0000","last_change":"2024-01-31T05:20:40.722612+0000","last_active":"2024-01-31T05:20:41.711542+0000","last_peered":"2024-01-31T05:20:41.711542+0000","last_clean":"2024-01-31T05:20:41.711542+0000","last_became_active":"2024-01-31T05:20:40.722289+0000","last_became_peered":"2024-01-31T05:20:40.722289+0000","last_unstale":"2024-01-31T05:20:41.711542+0000","last_undegraded":"2024-01-31T05:20:41.711542+0000","last_fullsized":"2024-01-31T05:20:41.711542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:30:24.130753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712456+0000","last_change":"2024-01-31T05:20:40.720798+0000","last_active":"2024-01-31T05:20:41.712456+0000","last_peered":"2024-01-31T05:20:41.712456+0000","last_clean":"2024-01-31T05:20:41.712456+0000","last_became_active":"2024-01-31T05:20:40.720393+0000","last_became_peered":"2024-01-31T05:20:40.720393+0000","last_unstale":"2024-01-31T05:20:41.712456+0000","last_undegraded":"2024-01-31T05:20:41.712456+0000","last_fullsized":"2024-01-31T05:20:41.712456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:05:51.442874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.716112+0000","last_change":"2024-01-31T05:20:40.715469+0000","last_active":"2024-01-31T05:20:40.716112+0000","last_peered":"2024-01-31T05:20:40.716112+0000","last_clean":"2024-01-31T05:20:40.716112+0000","last_became_active":"2024-01-31T05:20:40.715124+0000","last_became_peered":"2024-01-31T05:20:40.715124+0000","last_unstale":"2024-01-31T05:20:40.716112+0000","last_undegraded":"2024-01-31T05:20:40.716112+0000","last_fullsized":"2024-01-31T05:20:40.716112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:39.759250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989505+0000","last_change":"2024-01-31T05:20:40.719815+0000","last_active":"2024-01-31T05:20:41.989505+0000","last_peered":"2024-01-31T05:20:41.989505+0000","last_clean":"2024-01-31T05:20:41.989505+0000","last_became_active":"2024-01-31T05:20:40.719438+0000","last_became_peered":"2024-01-31T05:20:40.719438+0000","last_unstale":"2024-01-31T05:20:41.989505+0000","last_undegraded":"2024-01-31T05:20:41.989505+0000","last_fullsized":"2024-01-31T05:20:41.989505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:24:44.814291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22588,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":17,"num_read_kb":33,"num_write":118,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":57345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":441954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36880,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708272,"statfs":{"total":95995035648,"available":95957270528,"internally_reserved":0,"allocated":1581056,"data_stored":1054329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.71799999999999997}]},{"osd":3,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.50600000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.751}]},{"osd":6,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.626},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.61399999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.439},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.67200000000000004}]},{"osd":8,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.60099999999999998}]},{"osd":9,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.68999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40420,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704732,"statfs":{"total":95995035648,"available":95953645568,"internally_reserved":0,"allocated":942080,"data_stored":442190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.374}]},{"osd":1,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68100000000000005}]},{"osd":2,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.626}]},{"osd":3,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.496},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.66400000000000003}]},{"osd":6,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.46200000000000002}]},{"osd":7,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.72499999999999998}]},{"osd":8,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.73499999999999999}]},{"osd":10,"last update":"Wed Jan 31 05:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.75800000000000001}]}]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.37,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.474,"5min":0.77900000000000003,"15min":0.66300000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":17.460999999999999,"5min":17.460999999999999,"15min":17.460999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73499999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.41899999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70199999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.29699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95999999999999996,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77900000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.50900000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.622,"15min":0.628},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79600000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.90500000000000003}]},{"osd":5,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.76100000000000001,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.59999999999999998,"15min":0.61399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.72599999999999998}]},{"osd":6,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":0.83899999999999997,"15min":0.72699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":17.439,"5min":17.439,"15min":17.439},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53300000000000003}]},{"osd":7,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85499999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.548,"5min":0.91700000000000004,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":17.420000000000002,"5min":17.420000000000002,"15min":17.420000000000002},"last":0.86299999999999999}]},{"osd":9,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5,"15min":1.5},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":17.396999999999998,"5min":17.396999999999998,"15min":17.396999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":17.489999999999998,"5min":17.489999999999998,"15min":17.489999999999998},"last":0.76900000000000002}]},{"osd":10,"last update":"Wed Jan 31 05:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.83799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1033532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.49299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.59199999999999997}]},{"osd":1,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40500000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.432,"15min":0.433},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.62,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.82699999999999996}]},{"osd":3,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.036,"5min":0.69699999999999995,"15min":0.64100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":7.883,"5min":7.883,"15min":7.883},"last":0.75900000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58199999999999996}]},{"osd":5,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64800000000000002,"15min":0.64400000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.877}]},{"osd":6,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":0.75700000000000001,"15min":0.70099999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":7.9299999999999997,"5min":7.9299999999999997,"15min":7.9299999999999997},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.89500000000000002}]},{"osd":8,"last update":"Wed Jan 31 05:20:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69299999999999995,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.92900000000000005}]},{"osd":9,"last update":"Wed Jan 31 05:20:02 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":7.9020000000000001,"5min":7.9020000000000001,"15min":7.9020000000000001},"last":0.98599999999999999}]},{"osd":10,"last update":"Wed Jan 31 05:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.0369999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":6,"up_from":47,"seq":201863462949,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40392,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704760,"statfs":{"total":95995035648,"available":95953674240,"internally_reserved":0,"allocated":909312,"data_stored":432802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40100000000000002,"15min":0.39700000000000002},"min":{"1min":0.25,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.34799999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.41199999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.48599999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.496}]},{"osd":3,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.5,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75900000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46200000000000002}]},{"osd":4,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.629,"15min":0.628},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67600000000000005,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.60199999999999998}]},{"osd":5,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.64300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.44500000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.67500000000000004}]},{"osd":8,"last update":"Wed Jan 31 05:20:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93300000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.55000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.622}]},{"osd":9,"last update":"Wed Jan 31 05:20:02 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58799999999999997}]},{"osd":10,"last update":"Wed Jan 31 05:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.78800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":449583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:19:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.50800000000000001,"15min":0.48999999999999999},"min":{"1min":0.29999999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.501,"15min":0.495},"min":{"1min":0.33800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.748,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed Jan 31 05:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53000000000000003,"15min":0.50600000000000001},"min":{"1min":0.44700000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83599999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51100000000000001,"15min":0.499},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68100000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.62}]},{"osd":3,"last update":"Wed Jan 31 05:20:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69899999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.497},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.54800000000000004},"min":{"1min":0.38200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.84899999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.56200000000000006}]},{"osd":4,"last update":"Wed Jan 31 05:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.41899999999999998,"15min":0.39100000000000001},"min":{"1min":0.29799999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.42299999999999999,"15min":0.39700000000000002},"min":{"1min":0.33500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77200000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.44800000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58699999999999997,"15min":0.56100000000000005},"min":{"1min":0.47599999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.92500000000000004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72499999999999998,"5min":1.028,"15min":1.028},"last":0.84799999999999998}]},{"osd":6,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.375,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.627},"min":{"1min":0.40500000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.65800000000000003}]},{"osd":7,"last update":"Wed Jan 31 05:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67300000000000004}]},{"osd":8,"last update":"Wed Jan 31 05:20:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.749},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.86599999999999999}]},{"osd":9,"last update":"Wed Jan 31 05:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.82699999999999996}]},{"osd":10,"last update":"Wed Jan 31 05:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.88100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":913408,"data_stored":432892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":2.766},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":7.835}]},{"osd":2,"last update":"Wed Jan 31 05:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.42899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":1.585},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.72999999999999998,"15min":0.59199999999999997},"min":{"1min":0.39500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.874,"5min":18.466999999999999,"15min":18.466999999999999},"last":3.4529999999999998}]},{"osd":3,"last update":"Wed Jan 31 05:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.252,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.434},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.79100000000000004}]},{"osd":4,"last update":"Wed Jan 31 05:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.54800000000000004,"15min":0.50900000000000001},"min":{"1min":0.47099999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":2.972},{"interface":"front","average":{"1min":0.63,"5min":0.52200000000000002,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":1.835}]},{"osd":5,"last update":"Wed Jan 31 05:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.54800000000000004,"15min":0.52300000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88400000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":7.9130000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.77600000000000002,"15min":0.624},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.089,"5min":18.541,"15min":18.541},"last":3.359}]},{"osd":6,"last update":"Wed Jan 31 05:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46800000000000003,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.308,"15min":0.308},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":3.2749999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.42999999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":8.6560000000000006}]},{"osd":7,"last update":"Wed Jan 31 05:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.56799999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":3.1789999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.63700000000000001,"15min":0.622},"min":{"1min":0.502,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":2.105}]},{"osd":8,"last update":"Wed Jan 31 05:20:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.69999999999999996,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":8.6039999999999992},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90000000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":2.1549999999999998}]},{"osd":9,"last update":"Wed Jan 31 05:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":2.8799999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":8.7050000000000001}]},{"osd":10,"last update":"Wed Jan 31 05:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":3.3290000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":2.2250000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2959999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":444334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.42399999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.77900000000000003,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78700000000000003,"5min":1.617,"15min":1.617},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.42999999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77800000000000002,"5min":1.04,"15min":1.04},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.40699999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Jan 31 05:20:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71199999999999997,"5min":1.661,"15min":1.661},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.40899999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.70899999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.71499999999999997}]},{"osd":4,"last update":"Wed Jan 31 05:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58899999999999997,"15min":0.57699999999999996},"min":{"1min":0.48899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":1.077,"15min":1.077},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84499999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.82699999999999996}]},{"osd":5,"last update":"Wed Jan 31 05:20:31 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60999999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.434,"5min":0.45500000000000002,"15min":0.45000000000000001},"min":{"1min":0.33200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.65100000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Wed Jan 31 05:19:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.503,"5min":0.495,"15min":0.495},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.75},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.378,"5min":0.376,"15min":0.376},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.77000000000000002}]},{"osd":7,"last update":"Wed Jan 31 05:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.746,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.878,"15min":0.878},"last":0.66800000000000004}]},{"osd":8,"last update":"Wed Jan 31 05:20:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57099999999999995,"15min":0.57399999999999995},"min":{"1min":0.36599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73699999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.94799999999999995}]},{"osd":9,"last update":"Wed Jan 31 05:20:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.93100000000000005}]},{"osd":10,"last update":"Wed Jan 31 05:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":1.004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149748,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":441566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.32200000000000001},"min":{"1min":0.28499999999999998,"5min":0.189,"15min":0.189},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.34200000000000003,"15min":0.32400000000000001},"min":{"1min":0.27200000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.496}]},{"osd":1,"last update":"Wed Jan 31 05:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54600000000000004,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Jan 31 05:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55900000000000005,"15min":0.54100000000000004},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.85699999999999998}]},{"osd":4,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.36299999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.45400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed Jan 31 05:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.875}]},{"osd":6,"last update":"Wed Jan 31 05:19:50 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.35599999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.50600000000000001,"15min":0.502},"min":{"1min":0.376,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71599999999999997}]},{"osd":7,"last update":"Wed Jan 31 05:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.46500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.41999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73899999999999999}]},{"osd":8,"last update":"Wed Jan 31 05:20:35 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":0.92300000000000004,"15min":0.77700000000000002},"min":{"1min":0.504,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":17.98,"5min":17.98,"15min":17.98},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.82699999999999996}]},{"osd":9,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.80700000000000005}]},{"osd":10,"last update":"Wed Jan 31 05:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.90400000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40404,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704748,"statfs":{"total":95995035648,"available":95953661952,"internally_reserved":0,"allocated":925696,"data_stored":441394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53400000000000003,"15min":0.50800000000000001},"min":{"1min":0.38400000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78900000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.443},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.49199999999999999},"min":{"1min":0.45200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.753,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.43099999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.378},"min":{"1min":0.254,"5min":0.245,"15min":0.245},"max":{"1min":0.52200000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.38800000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57099999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56200000000000006}]},{"osd":2,"last update":"Wed Jan 31 05:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68000000000000005}]},{"osd":3,"last update":"Wed Jan 31 05:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.41899999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85299999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63100000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.72699999999999998}]},{"osd":6,"last update":"Wed Jan 31 05:19:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64400000000000002,"15min":0.63900000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75800000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86299999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.45000000000000001}]},{"osd":8,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67300000000000004},"min":{"1min":0.54200000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.90200000000000002}]},{"osd":9,"last update":"Wed Jan 31 05:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.78100000000000003}]},{"osd":10,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724587,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1031922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.50800000000000001,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33600000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73399999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.54700000000000004,"15min":0.53600000000000003},"min":{"1min":0.40799999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39900000000000002,"15min":0.39100000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.379},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39300000000000002,"15min":0.38300000000000001},"min":{"1min":0.255,"5min":0.252,"15min":0.252},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.35999999999999999}]},{"osd":3,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.40500000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.45600000000000002}]},{"osd":4,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76200000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.69199999999999995}]},{"osd":6,"last update":"Wed Jan 31 05:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.49199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.503},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.45700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.49199999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.626,"15min":0.625},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.629},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85399999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64700000000000002}]},{"osd":8,"last update":"Wed Jan 31 05:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.77000000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52000000000000002}]},{"osd":9,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59999999999999998}]},{"osd":10,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.70999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":9561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:20:47.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:20:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.356+0000 7f4936509640 1 -- 172.21.15.45:0/254431216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4930101840 msgr2=0x7f49301080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.356+0000 7f4936509640 1 --2- 172.21.15.45:0/254431216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4930101840 0x7f49301080d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4928009ca0 tx=0x7f492802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.356+0000 7f4936509640 1 -- 172.21.15.45:0/254431216 shutdown_connections 2024-01-31T05:20:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.356+0000 7f4936509640 1 --2- 172.21.15.45:0/254431216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4930101840 0x7f49301080d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.356+0000 7f4936509640 1 --2- 172.21.15.45:0/254431216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4930100e80 0x7f4930101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.356+0000 7f4936509640 1 --2- 172.21.15.45:0/254431216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49300ffc90 0x7f49301000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.356+0000 7f4936509640 1 -- 172.21.15.45:0/254431216 >> 172.21.15.45:0/254431216 conn(0x7f49300fb220 msgr2=0x7f49300fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.356+0000 7f4936509640 1 -- 172.21.15.45:0/254431216 shutdown_connections 2024-01-31T05:20:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.356+0000 7f4936509640 1 -- 172.21.15.45:0/254431216 wait complete. 2024-01-31T05:20:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.357+0000 7f4936509640 1 Processor -- start 2024-01-31T05:20:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.357+0000 7f4936509640 1 -- start start 2024-01-31T05:20:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.357+0000 7f4936509640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49300ffc90 0x7f4930069eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.357+0000 7f4936509640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4930100e80 0x7f493019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.357+0000 7f4936509640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4930101840 0x7f493019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.357+0000 7f4936509640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493019e780 con 0x7f4930100e80 2024-01-31T05:20:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.357+0000 7f4936509640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493019e8f0 con 0x7f49300ffc90 2024-01-31T05:20:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.357+0000 7f4936509640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493019eab0 con 0x7f4930101840 2024-01-31T05:20:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.357+0000 7f492f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4930100e80 0x7f493019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.357+0000 7f492f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4930100e80 0x7f493019b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55020/0 (socket says 172.21.15.45:55020) 2024-01-31T05:20:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.357+0000 7f492f7fe640 1 -- 172.21.15.45:0/829145316 learned_addr learned my addr 172.21.15.45:0/829145316 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f492ffff640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49300ffc90 0x7f4930069eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:47.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f492f7fe640 1 -- 172.21.15.45:0/829145316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4930101840 msgr2=0x7f493019dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:47.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f4934aa6640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4930101840 0x7f493019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:47.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f492f7fe640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4930101840 0x7f493019dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f492f7fe640 1 -- 172.21.15.45:0/829145316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49300ffc90 msgr2=0x7f4930069eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:47.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f492f7fe640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49300ffc90 0x7f4930069eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f492f7fe640 1 -- 172.21.15.45:0/829145316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928009900 con 0x7f4930100e80 2024-01-31T05:20:47.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f492f7fe640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4930100e80 0x7f493019b9b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f492400d950 tx=0x7f492400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:47.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f492d7fa640 1 -- 172.21.15.45:0/829145316 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924009a30 con 0x7f4930100e80 2024-01-31T05:20:47.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f4936509640 1 -- 172.21.15.45:0/829145316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493019ed90 con 0x7f4930100e80 2024-01-31T05:20:47.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f492d7fa640 1 -- 172.21.15.45:0/829145316 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4924009b90 con 0x7f4930100e80 2024-01-31T05:20:47.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f492d7fa640 1 -- 172.21.15.45:0/829145316 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49240047a0 con 0x7f4930100e80 2024-01-31T05:20:47.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.358+0000 7f4936509640 1 -- 172.21.15.45:0/829145316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493019f240 con 0x7f4930100e80 2024-01-31T05:20:47.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.359+0000 7f4936509640 1 -- 172.21.15.45:0/829145316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4930107020 con 0x7f4930100e80 2024-01-31T05:20:47.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.360+0000 7f492d7fa640 1 -- 172.21.15.45:0/829145316 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f4924009d00 con 0x7f4930100e80 2024-01-31T05:20:47.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.361+0000 7f492d7fa640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f49180a1370 0x7f49180a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:47.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.361+0000 7f492ffff640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f49180a1370 0x7f49180a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:47.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.361+0000 7f492d7fa640 1 -- 172.21.15.45:0/829145316 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f49240cc750 con 0x7f4930100e80 2024-01-31T05:20:47.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.361+0000 7f492ffff640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f49180a1370 0x7f49180a3830 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4930100ce0 tx=0x7f4920005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:47.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.365+0000 7f492d7fa640 1 -- 172.21.15.45:0/829145316 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f4924095b20 con 0x7f4930100e80 2024-01-31T05:20:47.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.544+0000 7f4936509640 1 -- 172.21.15.45:0/829145316 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f493002cc30 con 0x7f49180a1370 2024-01-31T05:20:47.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.554+0000 7f492d7fa640 1 -- 172.21.15.45:0/829145316 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336163 (secure 0 0 0) 0x7f493002cc30 con 0x7f49180a1370 2024-01-31T05:20:47.557 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:47.559 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-01-31T05:20:47.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.558+0000 7f4936509640 1 -- 172.21.15.45:0/829145316 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f49180a1370 msgr2=0x7f49180a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:47.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.558+0000 7f4936509640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f49180a1370 0x7f49180a3830 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4930100ce0 tx=0x7f4920005f90 comp rx=0 tx=0).stop 2024-01-31T05:20:47.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.558+0000 7f4936509640 1 -- 172.21.15.45:0/829145316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4930100e80 msgr2=0x7f493019b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:47.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.558+0000 7f4936509640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4930100e80 0x7f493019b9b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f492400d950 tx=0x7f492400de10 comp rx=0 tx=0).stop 2024-01-31T05:20:47.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.558+0000 7f4936509640 1 -- 172.21.15.45:0/829145316 shutdown_connections 2024-01-31T05:20:47.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.558+0000 7f4936509640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f49180a1370 0x7f49180a3830 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.558+0000 7f4936509640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4930100e80 0x7f493019b9b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.558+0000 7f4936509640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4930101840 0x7f493019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.558+0000 7f4936509640 1 --2- 172.21.15.45:0/829145316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49300ffc90 0x7f4930069eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:47.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.558+0000 7f4936509640 1 -- 172.21.15.45:0/829145316 >> 172.21.15.45:0/829145316 conn(0x7f49300fb220 msgr2=0x7f49301041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:47.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.558+0000 7f4936509640 1 -- 172.21.15.45:0/829145316 shutdown_connections 2024-01-31T05:20:47.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:47.558+0000 7f4936509640 1 -- 172.21.15.45:0/829145316 wait complete. 2024-01-31T05:20:47.568 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-01-31T05:20:46.591268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613190,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":153,"num_read_kb":152,"num_write":301,"num_write_kb":2995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":268,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":399376,"kb_used_data":12724,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124542448,"statfs":{"total":1151940427776,"available":1151531466752,"internally_reserved":0,"allocated":13029376,"data_stored":7088608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22822,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":17,"num_read_kb":33,"num_write":120,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.896534"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989980+0000","last_change":"2024-01-31T05:20:40.720465+0000","last_active":"2024-01-31T05:20:41.989980+0000","last_peered":"2024-01-31T05:20:41.989980+0000","last_clean":"2024-01-31T05:20:41.989980+0000","last_became_active":"2024-01-31T05:20:40.717615+0000","last_became_peered":"2024-01-31T05:20:40.717615+0000","last_unstale":"2024-01-31T05:20:41.989980+0000","last_undegraded":"2024-01-31T05:20:41.989980+0000","last_fullsized":"2024-01-31T05:20:41.989980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:30:01.564811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.730410+0000","last_change":"2024-01-31T05:20:40.729011+0000","last_active":"2024-01-31T05:20:40.730410+0000","last_peered":"2024-01-31T05:20:40.730410+0000","last_clean":"2024-01-31T05:20:40.730410+0000","last_became_active":"2024-01-31T05:20:40.728021+0000","last_became_peered":"2024-01-31T05:20:40.728021+0000","last_unstale":"2024-01-31T05:20:40.730410+0000","last_undegraded":"2024-01-31T05:20:40.730410+0000","last_fullsized":"2024-01-31T05:20:40.730410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:10:19.013530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.730853+0000","last_change":"2024-01-31T05:20:40.729136+0000","last_active":"2024-01-31T05:20:40.730853+0000","last_peered":"2024-01-31T05:20:40.730853+0000","last_clean":"2024-01-31T05:20:40.730853+0000","last_became_active":"2024-01-31T05:20:40.728273+0000","last_became_peered":"2024-01-31T05:20:40.728273+0000","last_unstale":"2024-01-31T05:20:40.730853+0000","last_undegraded":"2024-01-31T05:20:40.730853+0000","last_fullsized":"2024-01-31T05:20:40.730853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:46:43.462275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.887470+0000","last_change":"2024-01-31T05:20:40.721057+0000","last_active":"2024-01-31T05:20:41.887470+0000","last_peered":"2024-01-31T05:20:41.887470+0000","last_clean":"2024-01-31T05:20:41.887470+0000","last_became_active":"2024-01-31T05:20:40.720595+0000","last_became_peered":"2024-01-31T05:20:40.720595+0000","last_unstale":"2024-01-31T05:20:41.887470+0000","last_undegraded":"2024-01-31T05:20:41.887470+0000","last_fullsized":"2024-01-31T05:20:41.887470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:07:49.519349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990332+0000","last_change":"2024-01-31T05:20:40.724551+0000","last_active":"2024-01-31T05:20:41.990332+0000","last_peered":"2024-01-31T05:20:41.990332+0000","last_clean":"2024-01-31T05:20:41.990332+0000","last_became_active":"2024-01-31T05:20:40.724172+0000","last_became_peered":"2024-01-31T05:20:40.724172+0000","last_unstale":"2024-01-31T05:20:41.990332+0000","last_undegraded":"2024-01-31T05:20:41.990332+0000","last_fullsized":"2024-01-31T05:20:41.990332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:45:28.748481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989682+0000","last_change":"2024-01-31T05:20:40.721982+0000","last_active":"2024-01-31T05:20:41.989682+0000","last_peered":"2024-01-31T05:20:41.989682+0000","last_clean":"2024-01-31T05:20:41.989682+0000","last_became_active":"2024-01-31T05:20:40.721649+0000","last_became_peered":"2024-01-31T05:20:40.721649+0000","last_unstale":"2024-01-31T05:20:41.989682+0000","last_undegraded":"2024-01-31T05:20:41.989682+0000","last_fullsized":"2024-01-31T05:20:41.989682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:47:32.776711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.988984+0000","last_change":"2024-01-31T05:20:40.718340+0000","last_active":"2024-01-31T05:20:41.988984+0000","last_peered":"2024-01-31T05:20:41.988984+0000","last_clean":"2024-01-31T05:20:41.988984+0000","last_became_active":"2024-01-31T05:20:40.718017+0000","last_became_peered":"2024-01-31T05:20:40.718017+0000","last_unstale":"2024-01-31T05:20:41.988984+0000","last_undegraded":"2024-01-31T05:20:41.988984+0000","last_fullsized":"2024-01-31T05:20:41.988984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:17:45.606831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.711507+0000","last_change":"2024-01-31T05:20:40.720271+0000","last_active":"2024-01-31T05:20:41.711507+0000","last_peered":"2024-01-31T05:20:41.711507+0000","last_clean":"2024-01-31T05:20:41.711507+0000","last_became_active":"2024-01-31T05:20:40.719960+0000","last_became_peered":"2024-01-31T05:20:40.719960+0000","last_unstale":"2024-01-31T05:20:41.711507+0000","last_undegraded":"2024-01-31T05:20:41.711507+0000","last_fullsized":"2024-01-31T05:20:41.711507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:59:16.243268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712859+0000","last_change":"2024-01-31T05:20:40.725779+0000","last_active":"2024-01-31T05:20:41.712859+0000","last_peered":"2024-01-31T05:20:41.712859+0000","last_clean":"2024-01-31T05:20:41.712859+0000","last_became_active":"2024-01-31T05:20:40.725423+0000","last_became_peered":"2024-01-31T05:20:40.725423+0000","last_unstale":"2024-01-31T05:20:41.712859+0000","last_undegraded":"2024-01-31T05:20:41.712859+0000","last_fullsized":"2024-01-31T05:20:41.712859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:11:46.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990266+0000","last_change":"2024-01-31T05:20:40.723144+0000","last_active":"2024-01-31T05:20:41.990266+0000","last_peered":"2024-01-31T05:20:41.990266+0000","last_clean":"2024-01-31T05:20:41.990266+0000","last_became_active":"2024-01-31T05:20:40.722806+0000","last_became_peered":"2024-01-31T05:20:40.722806+0000","last_unstale":"2024-01-31T05:20:41.990266+0000","last_undegraded":"2024-01-31T05:20:41.990266+0000","last_fullsized":"2024-01-31T05:20:41.990266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:33.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989730+0000","last_change":"2024-01-31T05:20:40.721083+0000","last_active":"2024-01-31T05:20:41.989730+0000","last_peered":"2024-01-31T05:20:41.989730+0000","last_clean":"2024-01-31T05:20:41.989730+0000","last_became_active":"2024-01-31T05:20:40.720773+0000","last_became_peered":"2024-01-31T05:20:40.720773+0000","last_unstale":"2024-01-31T05:20:41.989730+0000","last_undegraded":"2024-01-31T05:20:41.989730+0000","last_fullsized":"2024-01-31T05:20:41.989730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:39:08.114126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.887435+0000","last_change":"2024-01-31T05:20:40.723630+0000","last_active":"2024-01-31T05:20:41.887435+0000","last_peered":"2024-01-31T05:20:41.887435+0000","last_clean":"2024-01-31T05:20:41.887435+0000","last_became_active":"2024-01-31T05:20:40.723340+0000","last_became_peered":"2024-01-31T05:20:40.723340+0000","last_unstale":"2024-01-31T05:20:41.887435+0000","last_undegraded":"2024-01-31T05:20:41.887435+0000","last_fullsized":"2024-01-31T05:20:41.887435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:53:21.199111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.713122+0000","last_change":"2024-01-31T05:20:40.712602+0000","last_active":"2024-01-31T05:20:40.713122+0000","last_peered":"2024-01-31T05:20:40.713122+0000","last_clean":"2024-01-31T05:20:40.713122+0000","last_became_active":"2024-01-31T05:20:40.712261+0000","last_became_peered":"2024-01-31T05:20:40.712261+0000","last_unstale":"2024-01-31T05:20:40.713122+0000","last_undegraded":"2024-01-31T05:20:40.713122+0000","last_fullsized":"2024-01-31T05:20:40.713122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:40:20.754437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.726917+0000","last_change":"2024-01-31T05:20:40.723759+0000","last_active":"2024-01-31T05:20:40.726917+0000","last_peered":"2024-01-31T05:20:40.726917+0000","last_clean":"2024-01-31T05:20:40.726917+0000","last_became_active":"2024-01-31T05:20:40.723421+0000","last_became_peered":"2024-01-31T05:20:40.723421+0000","last_unstale":"2024-01-31T05:20:40.726917+0000","last_undegraded":"2024-01-31T05:20:40.726917+0000","last_fullsized":"2024-01-31T05:20:40.726917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:35.268012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.727170+0000","last_change":"2024-01-31T05:20:40.726482+0000","last_active":"2024-01-31T05:20:40.727170+0000","last_peered":"2024-01-31T05:20:40.727170+0000","last_clean":"2024-01-31T05:20:40.727170+0000","last_became_active":"2024-01-31T05:20:40.726086+0000","last_became_peered":"2024-01-31T05:20:40.726086+0000","last_unstale":"2024-01-31T05:20:40.727170+0000","last_undegraded":"2024-01-31T05:20:40.727170+0000","last_fullsized":"2024-01-31T05:20:40.727170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:27:53.695166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712234+0000","last_change":"2024-01-31T05:20:40.728363+0000","last_active":"2024-01-31T05:20:41.712234+0000","last_peered":"2024-01-31T05:20:41.712234+0000","last_clean":"2024-01-31T05:20:41.712234+0000","last_became_active":"2024-01-31T05:20:40.727795+0000","last_became_peered":"2024-01-31T05:20:40.727795+0000","last_unstale":"2024-01-31T05:20:41.712234+0000","last_undegraded":"2024-01-31T05:20:41.712234+0000","last_fullsized":"2024-01-31T05:20:41.712234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:43:08.123159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989887+0000","last_change":"2024-01-31T05:20:40.720902+0000","last_active":"2024-01-31T05:20:41.989887+0000","last_peered":"2024-01-31T05:20:41.989887+0000","last_clean":"2024-01-31T05:20:41.989887+0000","last_became_active":"2024-01-31T05:20:40.720554+0000","last_became_peered":"2024-01-31T05:20:40.720554+0000","last_unstale":"2024-01-31T05:20:41.989887+0000","last_undegraded":"2024-01-31T05:20:41.989887+0000","last_fullsized":"2024-01-31T05:20:41.989887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:56:49.441368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990090+0000","last_change":"2024-01-31T05:20:40.710504+0000","last_active":"2024-01-31T05:20:41.990090+0000","last_peered":"2024-01-31T05:20:41.990090+0000","last_clean":"2024-01-31T05:20:41.990090+0000","last_became_active":"2024-01-31T05:20:40.710162+0000","last_became_peered":"2024-01-31T05:20:40.710162+0000","last_unstale":"2024-01-31T05:20:41.990090+0000","last_undegraded":"2024-01-31T05:20:41.990090+0000","last_fullsized":"2024-01-31T05:20:41.990090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:49:49.930573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989921+0000","last_change":"2024-01-31T05:20:40.729023+0000","last_active":"2024-01-31T05:20:41.989921+0000","last_peered":"2024-01-31T05:20:41.989921+0000","last_clean":"2024-01-31T05:20:41.989921+0000","last_became_active":"2024-01-31T05:20:40.728672+0000","last_became_peered":"2024-01-31T05:20:40.728672+0000","last_unstale":"2024-01-31T05:20:41.989921+0000","last_undegraded":"2024-01-31T05:20:41.989921+0000","last_fullsized":"2024-01-31T05:20:41.989921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:56:41.578440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.484132+0000","last_change":"2024-01-31T05:20:39.716771+0000","last_active":"2024-01-31T05:20:43.484132+0000","last_peered":"2024-01-31T05:20:43.484132+0000","last_clean":"2024-01-31T05:20:43.484132+0000","last_became_active":"2024-01-31T05:20:39.716439+0000","last_became_peered":"2024-01-31T05:20:39.716439+0000","last_unstale":"2024-01-31T05:20:43.484132+0000","last_undegraded":"2024-01-31T05:20:43.484132+0000","last_fullsized":"2024-01-31T05:20:43.484132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:14:24.317293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.711931+0000","last_change":"2024-01-31T05:20:40.730695+0000","last_active":"2024-01-31T05:20:41.711931+0000","last_peered":"2024-01-31T05:20:41.711931+0000","last_clean":"2024-01-31T05:20:41.711931+0000","last_became_active":"2024-01-31T05:20:40.730343+0000","last_became_peered":"2024-01-31T05:20:40.730343+0000","last_unstale":"2024-01-31T05:20:41.711931+0000","last_undegraded":"2024-01-31T05:20:41.711931+0000","last_fullsized":"2024-01-31T05:20:41.711931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:32.308239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712063+0000","last_change":"2024-01-31T05:20:39.711082+0000","last_active":"2024-01-31T05:20:41.712063+0000","last_peered":"2024-01-31T05:20:41.712063+0000","last_clean":"2024-01-31T05:20:41.712063+0000","last_became_active":"2024-01-31T05:20:39.710741+0000","last_became_peered":"2024-01-31T05:20:39.710741+0000","last_unstale":"2024-01-31T05:20:41.712063+0000","last_undegraded":"2024-01-31T05:20:41.712063+0000","last_fullsized":"2024-01-31T05:20:41.712063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:20:01.934837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.730946+0000","last_change":"2024-01-31T05:20:40.729269+0000","last_active":"2024-01-31T05:20:40.730946+0000","last_peered":"2024-01-31T05:20:40.730946+0000","last_clean":"2024-01-31T05:20:40.730946+0000","last_became_active":"2024-01-31T05:20:40.728910+0000","last_became_peered":"2024-01-31T05:20:40.728910+0000","last_unstale":"2024-01-31T05:20:40.730946+0000","last_undegraded":"2024-01-31T05:20:40.730946+0000","last_fullsized":"2024-01-31T05:20:40.730946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:28:16.951673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.482200+0000","last_change":"2024-01-31T05:20:39.710336+0000","last_active":"2024-01-31T05:20:43.482200+0000","last_peered":"2024-01-31T05:20:43.482200+0000","last_clean":"2024-01-31T05:20:43.482200+0000","last_became_active":"2024-01-31T05:20:39.710034+0000","last_became_peered":"2024-01-31T05:20:39.710034+0000","last_unstale":"2024-01-31T05:20:43.482200+0000","last_undegraded":"2024-01-31T05:20:43.482200+0000","last_fullsized":"2024-01-31T05:20:43.482200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:27:46.964866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712457+0000","last_change":"2024-01-31T05:20:40.727934+0000","last_active":"2024-01-31T05:20:41.712457+0000","last_peered":"2024-01-31T05:20:41.712457+0000","last_clean":"2024-01-31T05:20:41.712457+0000","last_became_active":"2024-01-31T05:20:40.727517+0000","last_became_peered":"2024-01-31T05:20:40.727517+0000","last_unstale":"2024-01-31T05:20:41.712457+0000","last_undegraded":"2024-01-31T05:20:41.712457+0000","last_fullsized":"2024-01-31T05:20:41.712457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:35:46.329596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990495+0000","last_change":"2024-01-31T05:20:39.709541+0000","last_active":"2024-01-31T05:20:41.990495+0000","last_peered":"2024-01-31T05:20:41.990495+0000","last_clean":"2024-01-31T05:20:41.990495+0000","last_became_active":"2024-01-31T05:20:39.708918+0000","last_became_peered":"2024-01-31T05:20:39.708918+0000","last_unstale":"2024-01-31T05:20:41.990495+0000","last_undegraded":"2024-01-31T05:20:41.990495+0000","last_fullsized":"2024-01-31T05:20:41.990495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:55:51.545144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712901+0000","last_change":"2024-01-31T05:20:40.729677+0000","last_active":"2024-01-31T05:20:41.712901+0000","last_peered":"2024-01-31T05:20:41.712901+0000","last_clean":"2024-01-31T05:20:41.712901+0000","last_became_active":"2024-01-31T05:20:40.727880+0000","last_became_peered":"2024-01-31T05:20:40.727880+0000","last_unstale":"2024-01-31T05:20:41.712901+0000","last_undegraded":"2024-01-31T05:20:41.712901+0000","last_fullsized":"2024-01-31T05:20:41.712901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:49:55.935842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712568+0000","last_change":"2024-01-31T05:20:39.711310+0000","last_active":"2024-01-31T05:20:41.712568+0000","last_peered":"2024-01-31T05:20:41.712568+0000","last_clean":"2024-01-31T05:20:41.712568+0000","last_became_active":"2024-01-31T05:20:39.710108+0000","last_became_peered":"2024-01-31T05:20:39.710108+0000","last_unstale":"2024-01-31T05:20:41.712568+0000","last_undegraded":"2024-01-31T05:20:41.712568+0000","last_fullsized":"2024-01-31T05:20:41.712568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:03:25.550777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990347+0000","last_change":"2024-01-31T05:20:40.722095+0000","last_active":"2024-01-31T05:20:41.990347+0000","last_peered":"2024-01-31T05:20:41.990347+0000","last_clean":"2024-01-31T05:20:41.990347+0000","last_became_active":"2024-01-31T05:20:40.721778+0000","last_became_peered":"2024-01-31T05:20:40.721778+0000","last_unstale":"2024-01-31T05:20:41.990347+0000","last_undegraded":"2024-01-31T05:20:41.990347+0000","last_fullsized":"2024-01-31T05:20:41.990347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:48:04.284745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.483270+0000","last_change":"2024-01-31T05:20:39.705704+0000","last_active":"2024-01-31T05:20:43.483270+0000","last_peered":"2024-01-31T05:20:43.483270+0000","last_clean":"2024-01-31T05:20:43.483270+0000","last_became_active":"2024-01-31T05:20:39.705304+0000","last_became_peered":"2024-01-31T05:20:39.705304+0000","last_unstale":"2024-01-31T05:20:43.483270+0000","last_undegraded":"2024-01-31T05:20:43.483270+0000","last_fullsized":"2024-01-31T05:20:43.483270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:01:40.116085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712765+0000","last_change":"2024-01-31T05:20:40.728028+0000","last_active":"2024-01-31T05:20:41.712765+0000","last_peered":"2024-01-31T05:20:41.712765+0000","last_clean":"2024-01-31T05:20:41.712765+0000","last_became_active":"2024-01-31T05:20:40.727197+0000","last_became_peered":"2024-01-31T05:20:40.727197+0000","last_unstale":"2024-01-31T05:20:41.712765+0000","last_undegraded":"2024-01-31T05:20:41.712765+0000","last_fullsized":"2024-01-31T05:20:41.712765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:12:19.749802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.698962+0000","last_change":"2024-01-31T05:20:39.705157+0000","last_active":"2024-01-31T05:20:40.698962+0000","last_peered":"2024-01-31T05:20:40.698962+0000","last_clean":"2024-01-31T05:20:40.698962+0000","last_became_active":"2024-01-31T05:20:39.704794+0000","last_became_peered":"2024-01-31T05:20:39.704794+0000","last_unstale":"2024-01-31T05:20:40.698962+0000","last_undegraded":"2024-01-31T05:20:40.698962+0000","last_fullsized":"2024-01-31T05:20:40.698962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:21:20.314515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.711868+0000","last_change":"2024-01-31T05:20:40.723906+0000","last_active":"2024-01-31T05:20:41.711868+0000","last_peered":"2024-01-31T05:20:41.711868+0000","last_clean":"2024-01-31T05:20:41.711868+0000","last_became_active":"2024-01-31T05:20:40.723612+0000","last_became_peered":"2024-01-31T05:20:40.723612+0000","last_unstale":"2024-01-31T05:20:41.711868+0000","last_undegraded":"2024-01-31T05:20:41.711868+0000","last_fullsized":"2024-01-31T05:20:41.711868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:42:42.509983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712228+0000","last_change":"2024-01-31T05:20:39.709315+0000","last_active":"2024-01-31T05:20:41.712228+0000","last_peered":"2024-01-31T05:20:41.712228+0000","last_clean":"2024-01-31T05:20:41.712228+0000","last_became_active":"2024-01-31T05:20:39.709033+0000","last_became_peered":"2024-01-31T05:20:39.709033+0000","last_unstale":"2024-01-31T05:20:41.712228+0000","last_undegraded":"2024-01-31T05:20:41.712228+0000","last_fullsized":"2024-01-31T05:20:41.712228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:43:29.540029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989378+0000","last_change":"2024-01-31T05:20:40.717396+0000","last_active":"2024-01-31T05:20:41.989378+0000","last_peered":"2024-01-31T05:20:41.989378+0000","last_clean":"2024-01-31T05:20:41.989378+0000","last_became_active":"2024-01-31T05:20:40.717076+0000","last_became_peered":"2024-01-31T05:20:40.717076+0000","last_unstale":"2024-01-31T05:20:41.989378+0000","last_undegraded":"2024-01-31T05:20:41.989378+0000","last_fullsized":"2024-01-31T05:20:41.989378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:42:51.667070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:44.484240+0000","last_change":"2024-01-31T05:20:39.716940+0000","last_active":"2024-01-31T05:20:44.484240+0000","last_peered":"2024-01-31T05:20:44.484240+0000","last_clean":"2024-01-31T05:20:44.484240+0000","last_became_active":"2024-01-31T05:20:39.716691+0000","last_became_peered":"2024-01-31T05:20:39.716691+0000","last_unstale":"2024-01-31T05:20:44.484240+0000","last_undegraded":"2024-01-31T05:20:44.484240+0000","last_fullsized":"2024-01-31T05:20:44.484240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:38:23.786807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712747+0000","last_change":"2024-01-31T05:20:40.727618+0000","last_active":"2024-01-31T05:20:41.712747+0000","last_peered":"2024-01-31T05:20:41.712747+0000","last_clean":"2024-01-31T05:20:41.712747+0000","last_became_active":"2024-01-31T05:20:40.727282+0000","last_became_peered":"2024-01-31T05:20:40.727282+0000","last_unstale":"2024-01-31T05:20:41.712747+0000","last_undegraded":"2024-01-31T05:20:41.712747+0000","last_fullsized":"2024-01-31T05:20:41.712747+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:07:12.416575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.470336+0000","last_change":"2024-01-31T05:20:39.709584+0000","last_active":"2024-01-31T05:20:42.470336+0000","last_peered":"2024-01-31T05:20:42.470336+0000","last_clean":"2024-01-31T05:20:42.470336+0000","last_became_active":"2024-01-31T05:20:39.708660+0000","last_became_peered":"2024-01-31T05:20:39.708660+0000","last_unstale":"2024-01-31T05:20:42.470336+0000","last_undegraded":"2024-01-31T05:20:42.470336+0000","last_fullsized":"2024-01-31T05:20:42.470336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:21:06.450913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.734563+0000","last_change":"2024-01-31T05:20:40.734040+0000","last_active":"2024-01-31T05:20:40.734563+0000","last_peered":"2024-01-31T05:20:40.734563+0000","last_clean":"2024-01-31T05:20:40.734563+0000","last_became_active":"2024-01-31T05:20:40.733307+0000","last_became_peered":"2024-01-31T05:20:40.733307+0000","last_unstale":"2024-01-31T05:20:40.734563+0000","last_undegraded":"2024-01-31T05:20:40.734563+0000","last_fullsized":"2024-01-31T05:20:40.734563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:32:22.657358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712952+0000","last_change":"2024-01-31T05:20:39.708004+0000","last_active":"2024-01-31T05:20:41.712952+0000","last_peered":"2024-01-31T05:20:41.712952+0000","last_clean":"2024-01-31T05:20:41.712952+0000","last_became_active":"2024-01-31T05:20:39.707681+0000","last_became_peered":"2024-01-31T05:20:39.707681+0000","last_unstale":"2024-01-31T05:20:41.712952+0000","last_undegraded":"2024-01-31T05:20:41.712952+0000","last_fullsized":"2024-01-31T05:20:41.712952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:24:10.003803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712652+0000","last_change":"2024-01-31T05:20:40.714449+0000","last_active":"2024-01-31T05:20:41.712652+0000","last_peered":"2024-01-31T05:20:41.712652+0000","last_clean":"2024-01-31T05:20:41.712652+0000","last_became_active":"2024-01-31T05:20:40.714064+0000","last_became_peered":"2024-01-31T05:20:40.714064+0000","last_unstale":"2024-01-31T05:20:41.712652+0000","last_undegraded":"2024-01-31T05:20:41.712652+0000","last_fullsized":"2024-01-31T05:20:41.712652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:17:04.803852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:45.487844+0000","last_change":"2024-01-31T05:20:39.709050+0000","last_active":"2024-01-31T05:20:45.487844+0000","last_peered":"2024-01-31T05:20:45.487844+0000","last_clean":"2024-01-31T05:20:45.487844+0000","last_became_active":"2024-01-31T05:20:39.708674+0000","last_became_peered":"2024-01-31T05:20:39.708674+0000","last_unstale":"2024-01-31T05:20:45.487844+0000","last_undegraded":"2024-01-31T05:20:45.487844+0000","last_fullsized":"2024-01-31T05:20:45.487844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:08:34.994683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20894,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":23,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990562+0000","last_change":"2024-01-31T05:20:40.727191+0000","last_active":"2024-01-31T05:20:41.990562+0000","last_peered":"2024-01-31T05:20:41.990562+0000","last_clean":"2024-01-31T05:20:41.990562+0000","last_became_active":"2024-01-31T05:20:40.725114+0000","last_became_peered":"2024-01-31T05:20:40.725114+0000","last_unstale":"2024-01-31T05:20:41.990562+0000","last_undegraded":"2024-01-31T05:20:41.990562+0000","last_fullsized":"2024-01-31T05:20:41.990562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:00.758702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.482923+0000","last_change":"2024-01-31T05:20:39.719349+0000","last_active":"2024-01-31T05:20:43.482923+0000","last_peered":"2024-01-31T05:20:43.482923+0000","last_clean":"2024-01-31T05:20:43.482923+0000","last_became_active":"2024-01-31T05:20:39.718918+0000","last_became_peered":"2024-01-31T05:20:39.718918+0000","last_unstale":"2024-01-31T05:20:43.482923+0000","last_undegraded":"2024-01-31T05:20:43.482923+0000","last_fullsized":"2024-01-31T05:20:43.482923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:47:48.521638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.717336+0000","last_change":"2024-01-31T05:20:40.716868+0000","last_active":"2024-01-31T05:20:40.717336+0000","last_peered":"2024-01-31T05:20:40.717336+0000","last_clean":"2024-01-31T05:20:40.717336+0000","last_became_active":"2024-01-31T05:20:40.716519+0000","last_became_peered":"2024-01-31T05:20:40.716519+0000","last_unstale":"2024-01-31T05:20:40.717336+0000","last_undegraded":"2024-01-31T05:20:40.717336+0000","last_fullsized":"2024-01-31T05:20:40.717336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:30:42.910337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712615+0000","last_change":"2024-01-31T05:20:39.716631+0000","last_active":"2024-01-31T05:20:41.712615+0000","last_peered":"2024-01-31T05:20:41.712615+0000","last_clean":"2024-01-31T05:20:41.712615+0000","last_became_active":"2024-01-31T05:20:39.716331+0000","last_became_peered":"2024-01-31T05:20:39.716331+0000","last_unstale":"2024-01-31T05:20:41.712615+0000","last_undegraded":"2024-01-31T05:20:41.712615+0000","last_fullsized":"2024-01-31T05:20:41.712615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:05:59.555000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989669+0000","last_change":"2024-01-31T05:20:40.721548+0000","last_active":"2024-01-31T05:20:41.989669+0000","last_peered":"2024-01-31T05:20:41.989669+0000","last_clean":"2024-01-31T05:20:41.989669+0000","last_became_active":"2024-01-31T05:20:40.720296+0000","last_became_peered":"2024-01-31T05:20:40.720296+0000","last_unstale":"2024-01-31T05:20:41.989669+0000","last_undegraded":"2024-01-31T05:20:41.989669+0000","last_fullsized":"2024-01-31T05:20:41.989669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:04:08.406863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.887065+0000","last_change":"2024-01-31T05:20:39.707644+0000","last_active":"2024-01-31T05:20:41.887065+0000","last_peered":"2024-01-31T05:20:41.887065+0000","last_clean":"2024-01-31T05:20:41.887065+0000","last_became_active":"2024-01-31T05:20:39.707334+0000","last_became_peered":"2024-01-31T05:20:39.707334+0000","last_unstale":"2024-01-31T05:20:41.887065+0000","last_undegraded":"2024-01-31T05:20:41.887065+0000","last_fullsized":"2024-01-31T05:20:41.887065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:13:25.630415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.988922+0000","last_change":"2024-01-31T05:20:40.715835+0000","last_active":"2024-01-31T05:20:41.988922+0000","last_peered":"2024-01-31T05:20:41.988922+0000","last_clean":"2024-01-31T05:20:41.988922+0000","last_became_active":"2024-01-31T05:20:40.715467+0000","last_became_peered":"2024-01-31T05:20:40.715467+0000","last_unstale":"2024-01-31T05:20:41.988922+0000","last_undegraded":"2024-01-31T05:20:41.988922+0000","last_fullsized":"2024-01-31T05:20:41.988922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:40:33.583238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.483883+0000","last_change":"2024-01-31T05:20:39.717929+0000","last_active":"2024-01-31T05:20:43.483883+0000","last_peered":"2024-01-31T05:20:43.483883+0000","last_clean":"2024-01-31T05:20:43.483883+0000","last_became_active":"2024-01-31T05:20:39.717644+0000","last_became_peered":"2024-01-31T05:20:39.717644+0000","last_unstale":"2024-01-31T05:20:43.483883+0000","last_undegraded":"2024-01-31T05:20:43.483883+0000","last_fullsized":"2024-01-31T05:20:43.483883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:24:41.296706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712394+0000","last_change":"2024-01-31T05:20:40.726182+0000","last_active":"2024-01-31T05:20:41.712394+0000","last_peered":"2024-01-31T05:20:41.712394+0000","last_clean":"2024-01-31T05:20:41.712394+0000","last_became_active":"2024-01-31T05:20:40.725858+0000","last_became_peered":"2024-01-31T05:20:40.725858+0000","last_unstale":"2024-01-31T05:20:41.712394+0000","last_undegraded":"2024-01-31T05:20:41.712394+0000","last_fullsized":"2024-01-31T05:20:41.712394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:27.209744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990875+0000","last_change":"2024-01-31T05:20:39.706132+0000","last_active":"2024-01-31T05:20:41.990875+0000","last_peered":"2024-01-31T05:20:41.990875+0000","last_clean":"2024-01-31T05:20:41.990875+0000","last_became_active":"2024-01-31T05:20:39.705791+0000","last_became_peered":"2024-01-31T05:20:39.705791+0000","last_unstale":"2024-01-31T05:20:41.990875+0000","last_undegraded":"2024-01-31T05:20:41.990875+0000","last_fullsized":"2024-01-31T05:20:41.990875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:54:57.857011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989542+0000","last_change":"2024-01-31T05:20:40.720232+0000","last_active":"2024-01-31T05:20:41.989542+0000","last_peered":"2024-01-31T05:20:41.989542+0000","last_clean":"2024-01-31T05:20:41.989542+0000","last_became_active":"2024-01-31T05:20:40.719917+0000","last_became_peered":"2024-01-31T05:20:40.719917+0000","last_unstale":"2024-01-31T05:20:41.989542+0000","last_undegraded":"2024-01-31T05:20:41.989542+0000","last_fullsized":"2024-01-31T05:20:41.989542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:24:25.931378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989744+0000","last_change":"2024-01-31T05:20:39.710195+0000","last_active":"2024-01-31T05:20:41.989744+0000","last_peered":"2024-01-31T05:20:41.989744+0000","last_clean":"2024-01-31T05:20:41.989744+0000","last_became_active":"2024-01-31T05:20:39.709787+0000","last_became_peered":"2024-01-31T05:20:39.709787+0000","last_unstale":"2024-01-31T05:20:41.989744+0000","last_undegraded":"2024-01-31T05:20:41.989744+0000","last_fullsized":"2024-01-31T05:20:41.989744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:28:36.283468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990259+0000","last_change":"2024-01-31T05:20:40.716243+0000","last_active":"2024-01-31T05:20:41.990259+0000","last_peered":"2024-01-31T05:20:41.990259+0000","last_clean":"2024-01-31T05:20:41.990259+0000","last_became_active":"2024-01-31T05:20:40.715927+0000","last_became_peered":"2024-01-31T05:20:40.715927+0000","last_unstale":"2024-01-31T05:20:41.990259+0000","last_undegraded":"2024-01-31T05:20:41.990259+0000","last_fullsized":"2024-01-31T05:20:41.990259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:20:19.289014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989133+0000","last_change":"2024-01-31T05:20:39.712333+0000","last_active":"2024-01-31T05:20:41.989133+0000","last_peered":"2024-01-31T05:20:41.989133+0000","last_clean":"2024-01-31T05:20:41.989133+0000","last_became_active":"2024-01-31T05:20:39.712008+0000","last_became_peered":"2024-01-31T05:20:39.712008+0000","last_unstale":"2024-01-31T05:20:41.989133+0000","last_undegraded":"2024-01-31T05:20:41.989133+0000","last_fullsized":"2024-01-31T05:20:41.989133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:04:52.962130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990046+0000","last_change":"2024-01-31T05:20:40.725131+0000","last_active":"2024-01-31T05:20:41.990046+0000","last_peered":"2024-01-31T05:20:41.990046+0000","last_clean":"2024-01-31T05:20:41.990046+0000","last_became_active":"2024-01-31T05:20:40.724797+0000","last_became_peered":"2024-01-31T05:20:40.724797+0000","last_unstale":"2024-01-31T05:20:41.990046+0000","last_undegraded":"2024-01-31T05:20:41.990046+0000","last_fullsized":"2024-01-31T05:20:41.990046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:23:26.535023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712887+0000","last_change":"2024-01-31T05:20:40.726857+0000","last_active":"2024-01-31T05:20:41.712887+0000","last_peered":"2024-01-31T05:20:41.712887+0000","last_clean":"2024-01-31T05:20:41.712887+0000","last_became_active":"2024-01-31T05:20:40.726545+0000","last_became_peered":"2024-01-31T05:20:40.726545+0000","last_unstale":"2024-01-31T05:20:41.712887+0000","last_undegraded":"2024-01-31T05:20:41.712887+0000","last_fullsized":"2024-01-31T05:20:41.712887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:43:55.852474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.698763+0000","last_change":"2024-01-31T05:20:39.713920+0000","last_active":"2024-01-31T05:20:40.698763+0000","last_peered":"2024-01-31T05:20:40.698763+0000","last_clean":"2024-01-31T05:20:40.698763+0000","last_became_active":"2024-01-31T05:20:39.713582+0000","last_became_peered":"2024-01-31T05:20:39.713582+0000","last_unstale":"2024-01-31T05:20:40.698763+0000","last_undegraded":"2024-01-31T05:20:40.698763+0000","last_fullsized":"2024-01-31T05:20:40.698763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:20:18.932993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712051+0000","last_change":"2024-01-31T05:20:40.720442+0000","last_active":"2024-01-31T05:20:41.712051+0000","last_peered":"2024-01-31T05:20:41.712051+0000","last_clean":"2024-01-31T05:20:41.712051+0000","last_became_active":"2024-01-31T05:20:40.720114+0000","last_became_peered":"2024-01-31T05:20:40.720114+0000","last_unstale":"2024-01-31T05:20:41.712051+0000","last_undegraded":"2024-01-31T05:20:41.712051+0000","last_fullsized":"2024-01-31T05:20:41.712051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:11:01.536820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.466410+0000","last_change":"2024-01-31T05:20:39.706791+0000","last_active":"2024-01-31T05:20:42.466410+0000","last_peered":"2024-01-31T05:20:42.466410+0000","last_clean":"2024-01-31T05:20:42.466410+0000","last_became_active":"2024-01-31T05:20:39.706447+0000","last_became_peered":"2024-01-31T05:20:39.706447+0000","last_unstale":"2024-01-31T05:20:42.466410+0000","last_undegraded":"2024-01-31T05:20:42.466410+0000","last_fullsized":"2024-01-31T05:20:42.466410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:41:05.118138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.988972+0000","last_change":"2024-01-31T05:19:28.446661+0000","last_active":"2024-01-31T05:20:41.988972+0000","last_peered":"2024-01-31T05:20:41.988972+0000","last_clean":"2024-01-31T05:20:41.988972+0000","last_became_active":"2024-01-31T05:19:28.446238+0000","last_became_peered":"2024-01-31T05:19:28.446238+0000","last_unstale":"2024-01-31T05:20:41.988972+0000","last_undegraded":"2024-01-31T05:20:41.988972+0000","last_fullsized":"2024-01-31T05:20:41.988972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:16:11.415995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:16:11.415995+0000","last_clean_scrub_stamp":"2024-01-31T05:16:11.415995+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:56:53.665942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.715768+0000","last_change":"2024-01-31T05:20:40.714343+0000","last_active":"2024-01-31T05:20:40.715768+0000","last_peered":"2024-01-31T05:20:40.715768+0000","last_clean":"2024-01-31T05:20:40.715768+0000","last_became_active":"2024-01-31T05:20:40.713820+0000","last_became_peered":"2024-01-31T05:20:40.713820+0000","last_unstale":"2024-01-31T05:20:40.715768+0000","last_undegraded":"2024-01-31T05:20:40.715768+0000","last_fullsized":"2024-01-31T05:20:40.715768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:46:29.228809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.482100+0000","last_change":"2024-01-31T05:20:39.712068+0000","last_active":"2024-01-31T05:20:43.482100+0000","last_peered":"2024-01-31T05:20:43.482100+0000","last_clean":"2024-01-31T05:20:43.482100+0000","last_became_active":"2024-01-31T05:20:39.711641+0000","last_became_peered":"2024-01-31T05:20:39.711641+0000","last_unstale":"2024-01-31T05:20:43.482100+0000","last_undegraded":"2024-01-31T05:20:43.482100+0000","last_fullsized":"2024-01-31T05:20:43.482100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:27.199351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.471461+0000","last_change":"2024-01-31T05:20:39.716003+0000","last_active":"2024-01-31T05:20:42.471461+0000","last_peered":"2024-01-31T05:20:42.471461+0000","last_clean":"2024-01-31T05:20:42.471461+0000","last_became_active":"2024-01-31T05:20:39.715653+0000","last_became_peered":"2024-01-31T05:20:39.715653+0000","last_unstale":"2024-01-31T05:20:42.471461+0000","last_undegraded":"2024-01-31T05:20:42.471461+0000","last_fullsized":"2024-01-31T05:20:42.471461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:45:56.886077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.713156+0000","last_change":"2024-01-31T05:20:40.712298+0000","last_active":"2024-01-31T05:20:40.713156+0000","last_peered":"2024-01-31T05:20:40.713156+0000","last_clean":"2024-01-31T05:20:40.713156+0000","last_became_active":"2024-01-31T05:20:40.711907+0000","last_became_peered":"2024-01-31T05:20:40.711907+0000","last_unstale":"2024-01-31T05:20:40.713156+0000","last_undegraded":"2024-01-31T05:20:40.713156+0000","last_fullsized":"2024-01-31T05:20:40.713156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:17:52.282149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:45.000870+0000","last_change":"2024-01-31T05:20:39.720639+0000","last_active":"2024-01-31T05:20:45.000870+0000","last_peered":"2024-01-31T05:20:45.000870+0000","last_clean":"2024-01-31T05:20:45.000870+0000","last_became_active":"2024-01-31T05:20:39.720302+0000","last_became_peered":"2024-01-31T05:20:39.720302+0000","last_unstale":"2024-01-31T05:20:45.000870+0000","last_undegraded":"2024-01-31T05:20:45.000870+0000","last_fullsized":"2024-01-31T05:20:45.000870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:11:31.507908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712008+0000","last_change":"2024-01-31T05:20:40.725324+0000","last_active":"2024-01-31T05:20:41.712008+0000","last_peered":"2024-01-31T05:20:41.712008+0000","last_clean":"2024-01-31T05:20:41.712008+0000","last_became_active":"2024-01-31T05:20:40.725027+0000","last_became_peered":"2024-01-31T05:20:40.725027+0000","last_unstale":"2024-01-31T05:20:41.712008+0000","last_undegraded":"2024-01-31T05:20:41.712008+0000","last_fullsized":"2024-01-31T05:20:41.712008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:06:24.129276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.471871+0000","last_change":"2024-01-31T05:20:39.705805+0000","last_active":"2024-01-31T05:20:42.471871+0000","last_peered":"2024-01-31T05:20:42.471871+0000","last_clean":"2024-01-31T05:20:42.471871+0000","last_became_active":"2024-01-31T05:20:39.705447+0000","last_became_peered":"2024-01-31T05:20:39.705447+0000","last_unstale":"2024-01-31T05:20:42.471871+0000","last_undegraded":"2024-01-31T05:20:42.471871+0000","last_fullsized":"2024-01-31T05:20:42.471871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:24:47.856773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.734557+0000","last_change":"2024-01-31T05:20:40.733818+0000","last_active":"2024-01-31T05:20:40.734557+0000","last_peered":"2024-01-31T05:20:40.734557+0000","last_clean":"2024-01-31T05:20:40.734557+0000","last_became_active":"2024-01-31T05:20:40.733272+0000","last_became_peered":"2024-01-31T05:20:40.733272+0000","last_unstale":"2024-01-31T05:20:40.734557+0000","last_undegraded":"2024-01-31T05:20:40.734557+0000","last_fullsized":"2024-01-31T05:20:40.734557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:27:19.767212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.471935+0000","last_change":"2024-01-31T05:20:39.707920+0000","last_active":"2024-01-31T05:20:42.471935+0000","last_peered":"2024-01-31T05:20:42.471935+0000","last_clean":"2024-01-31T05:20:42.471935+0000","last_became_active":"2024-01-31T05:20:39.707612+0000","last_became_peered":"2024-01-31T05:20:39.707612+0000","last_unstale":"2024-01-31T05:20:42.471935+0000","last_undegraded":"2024-01-31T05:20:42.471935+0000","last_fullsized":"2024-01-31T05:20:42.471935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:57.522664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989302+0000","last_change":"2024-01-31T05:20:40.722376+0000","last_active":"2024-01-31T05:20:41.989302+0000","last_peered":"2024-01-31T05:20:41.989302+0000","last_clean":"2024-01-31T05:20:41.989302+0000","last_became_active":"2024-01-31T05:20:40.722055+0000","last_became_peered":"2024-01-31T05:20:40.722055+0000","last_unstale":"2024-01-31T05:20:41.989302+0000","last_undegraded":"2024-01-31T05:20:41.989302+0000","last_fullsized":"2024-01-31T05:20:41.989302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:11:47.198866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.466727+0000","last_change":"2024-01-31T05:20:39.714498+0000","last_active":"2024-01-31T05:20:42.466727+0000","last_peered":"2024-01-31T05:20:42.466727+0000","last_clean":"2024-01-31T05:20:42.466727+0000","last_became_active":"2024-01-31T05:20:39.714179+0000","last_became_peered":"2024-01-31T05:20:39.714179+0000","last_unstale":"2024-01-31T05:20:42.466727+0000","last_undegraded":"2024-01-31T05:20:42.466727+0000","last_fullsized":"2024-01-31T05:20:42.466727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:42:55.913696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989706+0000","last_change":"2024-01-31T05:20:40.716878+0000","last_active":"2024-01-31T05:20:41.989706+0000","last_peered":"2024-01-31T05:20:41.989706+0000","last_clean":"2024-01-31T05:20:41.989706+0000","last_became_active":"2024-01-31T05:20:40.715998+0000","last_became_peered":"2024-01-31T05:20:40.715998+0000","last_unstale":"2024-01-31T05:20:41.989706+0000","last_undegraded":"2024-01-31T05:20:41.989706+0000","last_fullsized":"2024-01-31T05:20:41.989706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:59:31.030147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989446+0000","last_change":"2024-01-31T05:20:39.719267+0000","last_active":"2024-01-31T05:20:41.989446+0000","last_peered":"2024-01-31T05:20:41.989446+0000","last_clean":"2024-01-31T05:20:41.989446+0000","last_became_active":"2024-01-31T05:20:39.718964+0000","last_became_peered":"2024-01-31T05:20:39.718964+0000","last_unstale":"2024-01-31T05:20:41.989446+0000","last_undegraded":"2024-01-31T05:20:41.989446+0000","last_fullsized":"2024-01-31T05:20:41.989446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:31.150796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.711786+0000","last_change":"2024-01-31T05:20:40.727530+0000","last_active":"2024-01-31T05:20:41.711786+0000","last_peered":"2024-01-31T05:20:41.711786+0000","last_clean":"2024-01-31T05:20:41.711786+0000","last_became_active":"2024-01-31T05:20:40.727016+0000","last_became_peered":"2024-01-31T05:20:40.727016+0000","last_unstale":"2024-01-31T05:20:41.711786+0000","last_undegraded":"2024-01-31T05:20:41.711786+0000","last_fullsized":"2024-01-31T05:20:41.711786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:15:25.851713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:42.472142+0000","last_change":"2024-01-31T05:20:39.711083+0000","last_active":"2024-01-31T05:20:42.472142+0000","last_peered":"2024-01-31T05:20:42.472142+0000","last_clean":"2024-01-31T05:20:42.472142+0000","last_became_active":"2024-01-31T05:20:39.710732+0000","last_became_peered":"2024-01-31T05:20:39.710732+0000","last_unstale":"2024-01-31T05:20:42.472142+0000","last_undegraded":"2024-01-31T05:20:42.472142+0000","last_fullsized":"2024-01-31T05:20:42.472142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:43.846910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989238+0000","last_change":"2024-01-31T05:20:40.723667+0000","last_active":"2024-01-31T05:20:41.989238+0000","last_peered":"2024-01-31T05:20:41.989238+0000","last_clean":"2024-01-31T05:20:41.989238+0000","last_became_active":"2024-01-31T05:20:40.723343+0000","last_became_peered":"2024-01-31T05:20:40.723343+0000","last_unstale":"2024-01-31T05:20:41.989238+0000","last_undegraded":"2024-01-31T05:20:41.989238+0000","last_fullsized":"2024-01-31T05:20:41.989238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:12.983137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:43.483898+0000","last_change":"2024-01-31T05:20:39.717263+0000","last_active":"2024-01-31T05:20:43.483898+0000","last_peered":"2024-01-31T05:20:43.483898+0000","last_clean":"2024-01-31T05:20:43.483898+0000","last_became_active":"2024-01-31T05:20:39.716666+0000","last_became_peered":"2024-01-31T05:20:39.716666+0000","last_unstale":"2024-01-31T05:20:43.483898+0000","last_undegraded":"2024-01-31T05:20:43.483898+0000","last_fullsized":"2024-01-31T05:20:43.483898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:10:41.789503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989721+0000","last_change":"2024-01-31T05:20:40.721728+0000","last_active":"2024-01-31T05:20:41.989721+0000","last_peered":"2024-01-31T05:20:41.989721+0000","last_clean":"2024-01-31T05:20:41.989721+0000","last_became_active":"2024-01-31T05:20:40.717082+0000","last_became_peered":"2024-01-31T05:20:40.717082+0000","last_unstale":"2024-01-31T05:20:41.989721+0000","last_undegraded":"2024-01-31T05:20:41.989721+0000","last_fullsized":"2024-01-31T05:20:41.989721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:15:55.823060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712785+0000","last_change":"2024-01-31T05:20:39.715450+0000","last_active":"2024-01-31T05:20:41.712785+0000","last_peered":"2024-01-31T05:20:41.712785+0000","last_clean":"2024-01-31T05:20:41.712785+0000","last_became_active":"2024-01-31T05:20:39.715140+0000","last_became_peered":"2024-01-31T05:20:39.715140+0000","last_unstale":"2024-01-31T05:20:41.712785+0000","last_undegraded":"2024-01-31T05:20:41.712785+0000","last_fullsized":"2024-01-31T05:20:41.712785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:34:01.299228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989819+0000","last_change":"2024-01-31T05:20:40.723855+0000","last_active":"2024-01-31T05:20:41.989819+0000","last_peered":"2024-01-31T05:20:41.989819+0000","last_clean":"2024-01-31T05:20:41.989819+0000","last_became_active":"2024-01-31T05:20:40.723532+0000","last_became_peered":"2024-01-31T05:20:40.723532+0000","last_unstale":"2024-01-31T05:20:41.989819+0000","last_undegraded":"2024-01-31T05:20:41.989819+0000","last_fullsized":"2024-01-31T05:20:41.989819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:48:59.700145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.731877+0000","last_change":"2024-01-31T05:20:39.713941+0000","last_active":"2024-01-31T05:20:40.731877+0000","last_peered":"2024-01-31T05:20:40.731877+0000","last_clean":"2024-01-31T05:20:40.731877+0000","last_became_active":"2024-01-31T05:20:39.713610+0000","last_became_peered":"2024-01-31T05:20:39.713610+0000","last_unstale":"2024-01-31T05:20:40.731877+0000","last_undegraded":"2024-01-31T05:20:40.731877+0000","last_fullsized":"2024-01-31T05:20:40.731877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:48:13.722639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989566+0000","last_change":"2024-01-31T05:20:40.716485+0000","last_active":"2024-01-31T05:20:41.989566+0000","last_peered":"2024-01-31T05:20:41.989566+0000","last_clean":"2024-01-31T05:20:41.989566+0000","last_became_active":"2024-01-31T05:20:40.716177+0000","last_became_peered":"2024-01-31T05:20:40.716177+0000","last_unstale":"2024-01-31T05:20:41.989566+0000","last_undegraded":"2024-01-31T05:20:41.989566+0000","last_fullsized":"2024-01-31T05:20:41.989566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:25:37.359617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989885+0000","last_change":"2024-01-31T05:20:40.710376+0000","last_active":"2024-01-31T05:20:41.989885+0000","last_peered":"2024-01-31T05:20:41.989885+0000","last_clean":"2024-01-31T05:20:41.989885+0000","last_became_active":"2024-01-31T05:20:40.710017+0000","last_became_peered":"2024-01-31T05:20:40.710017+0000","last_unstale":"2024-01-31T05:20:41.989885+0000","last_undegraded":"2024-01-31T05:20:41.989885+0000","last_fullsized":"2024-01-31T05:20:41.989885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:48:37.519561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989391+0000","last_change":"2024-01-31T05:20:40.727375+0000","last_active":"2024-01-31T05:20:41.989391+0000","last_peered":"2024-01-31T05:20:41.989391+0000","last_clean":"2024-01-31T05:20:41.989391+0000","last_became_active":"2024-01-31T05:20:40.725995+0000","last_became_peered":"2024-01-31T05:20:40.725995+0000","last_unstale":"2024-01-31T05:20:41.989391+0000","last_undegraded":"2024-01-31T05:20:41.989391+0000","last_fullsized":"2024-01-31T05:20:41.989391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:13:03.392972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989342+0000","last_change":"2024-01-31T05:20:40.725974+0000","last_active":"2024-01-31T05:20:41.989342+0000","last_peered":"2024-01-31T05:20:41.989342+0000","last_clean":"2024-01-31T05:20:41.989342+0000","last_became_active":"2024-01-31T05:20:40.725586+0000","last_became_peered":"2024-01-31T05:20:40.725586+0000","last_unstale":"2024-01-31T05:20:41.989342+0000","last_undegraded":"2024-01-31T05:20:41.989342+0000","last_fullsized":"2024-01-31T05:20:41.989342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:03:06.413502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.734533+0000","last_change":"2024-01-31T05:20:40.733524+0000","last_active":"2024-01-31T05:20:40.734533+0000","last_peered":"2024-01-31T05:20:40.734533+0000","last_clean":"2024-01-31T05:20:40.734533+0000","last_became_active":"2024-01-31T05:20:40.733162+0000","last_became_peered":"2024-01-31T05:20:40.733162+0000","last_unstale":"2024-01-31T05:20:40.734533+0000","last_undegraded":"2024-01-31T05:20:40.734533+0000","last_fullsized":"2024-01-31T05:20:40.734533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:00:56.321364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.717753+0000","last_change":"2024-01-31T05:20:40.715460+0000","last_active":"2024-01-31T05:20:40.717753+0000","last_peered":"2024-01-31T05:20:40.717753+0000","last_clean":"2024-01-31T05:20:40.717753+0000","last_became_active":"2024-01-31T05:20:40.715103+0000","last_became_peered":"2024-01-31T05:20:40.715103+0000","last_unstale":"2024-01-31T05:20:40.717753+0000","last_undegraded":"2024-01-31T05:20:40.717753+0000","last_fullsized":"2024-01-31T05:20:40.717753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:39:31.084636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989806+0000","last_change":"2024-01-31T05:20:40.714317+0000","last_active":"2024-01-31T05:20:41.989806+0000","last_peered":"2024-01-31T05:20:41.989806+0000","last_clean":"2024-01-31T05:20:41.989806+0000","last_became_active":"2024-01-31T05:20:40.714008+0000","last_became_peered":"2024-01-31T05:20:40.714008+0000","last_unstale":"2024-01-31T05:20:41.989806+0000","last_undegraded":"2024-01-31T05:20:41.989806+0000","last_fullsized":"2024-01-31T05:20:41.989806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:36:39.740278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.990314+0000","last_change":"2024-01-31T05:20:40.710901+0000","last_active":"2024-01-31T05:20:41.990314+0000","last_peered":"2024-01-31T05:20:41.990314+0000","last_clean":"2024-01-31T05:20:41.990314+0000","last_became_active":"2024-01-31T05:20:40.710412+0000","last_became_peered":"2024-01-31T05:20:40.710412+0000","last_unstale":"2024-01-31T05:20:41.990314+0000","last_undegraded":"2024-01-31T05:20:41.990314+0000","last_fullsized":"2024-01-31T05:20:41.990314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:39.966723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.711934+0000","last_change":"2024-01-31T05:20:40.732753+0000","last_active":"2024-01-31T05:20:41.711934+0000","last_peered":"2024-01-31T05:20:41.711934+0000","last_clean":"2024-01-31T05:20:41.711934+0000","last_became_active":"2024-01-31T05:20:40.732330+0000","last_became_peered":"2024-01-31T05:20:40.732330+0000","last_unstale":"2024-01-31T05:20:41.711934+0000","last_undegraded":"2024-01-31T05:20:41.711934+0000","last_fullsized":"2024-01-31T05:20:41.711934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:54:01.933393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712468+0000","last_change":"2024-01-31T05:20:40.723129+0000","last_active":"2024-01-31T05:20:41.712468+0000","last_peered":"2024-01-31T05:20:41.712468+0000","last_clean":"2024-01-31T05:20:41.712468+0000","last_became_active":"2024-01-31T05:20:40.722796+0000","last_became_peered":"2024-01-31T05:20:40.722796+0000","last_unstale":"2024-01-31T05:20:41.712468+0000","last_undegraded":"2024-01-31T05:20:41.712468+0000","last_fullsized":"2024-01-31T05:20:41.712468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:18.120495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.711542+0000","last_change":"2024-01-31T05:20:40.722612+0000","last_active":"2024-01-31T05:20:41.711542+0000","last_peered":"2024-01-31T05:20:41.711542+0000","last_clean":"2024-01-31T05:20:41.711542+0000","last_became_active":"2024-01-31T05:20:40.722289+0000","last_became_peered":"2024-01-31T05:20:40.722289+0000","last_unstale":"2024-01-31T05:20:41.711542+0000","last_undegraded":"2024-01-31T05:20:41.711542+0000","last_fullsized":"2024-01-31T05:20:41.711542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:30:24.130753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.712456+0000","last_change":"2024-01-31T05:20:40.720798+0000","last_active":"2024-01-31T05:20:41.712456+0000","last_peered":"2024-01-31T05:20:41.712456+0000","last_clean":"2024-01-31T05:20:41.712456+0000","last_became_active":"2024-01-31T05:20:40.720393+0000","last_became_peered":"2024-01-31T05:20:40.720393+0000","last_unstale":"2024-01-31T05:20:41.712456+0000","last_undegraded":"2024-01-31T05:20:41.712456+0000","last_fullsized":"2024-01-31T05:20:41.712456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:05:51.442874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-31T05:20:40.716112+0000","last_change":"2024-01-31T05:20:40.715469+0000","last_active":"2024-01-31T05:20:40.716112+0000","last_peered":"2024-01-31T05:20:40.716112+0000","last_clean":"2024-01-31T05:20:40.716112+0000","last_became_active":"2024-01-31T05:20:40.715124+0000","last_became_peered":"2024-01-31T05:20:40.715124+0000","last_unstale":"2024-01-31T05:20:40.716112+0000","last_undegraded":"2024-01-31T05:20:40.716112+0000","last_fullsized":"2024-01-31T05:20:40.716112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:39.759250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-31T05:20:41.989505+0000","last_change":"2024-01-31T05:20:40.719815+0000","last_active":"2024-01-31T05:20:41.989505+0000","last_peered":"2024-01-31T05:20:41.989505+0000","last_clean":"2024-01-31T05:20:41.989505+0000","last_became_active":"2024-01-31T05:20:40.719438+0000","last_became_peered":"2024-01-31T05:20:40.719438+0000","last_unstale":"2024-01-31T05:20:41.989505+0000","last_undegraded":"2024-01-31T05:20:41.989505+0000","last_fullsized":"2024-01-31T05:20:41.989505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:24:44.814291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22588,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":17,"num_read_kb":33,"num_write":118,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":57345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":441954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36880,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708272,"statfs":{"total":95995035648,"available":95957270528,"internally_reserved":0,"allocated":1581056,"data_stored":1054329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.71799999999999997}]},{"osd":3,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.50600000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.751}]},{"osd":6,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.626},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.61399999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.439},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.67200000000000004}]},{"osd":8,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.60099999999999998}]},{"osd":9,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.68999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40420,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704732,"statfs":{"total":95995035648,"available":95953645568,"internally_reserved":0,"allocated":942080,"data_stored":442190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.374}]},{"osd":1,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68100000000000005}]},{"osd":2,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.626}]},{"osd":3,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.496},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.66400000000000003}]},{"osd":6,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.46200000000000002}]},{"osd":7,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.72499999999999998}]},{"osd":8,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.73499999999999999}]},{"osd":10,"last update":"Wed Jan 31 05:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.75800000000000001}]}]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.37,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.474,"5min":0.77900000000000003,"15min":0.66300000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":17.460999999999999,"5min":17.460999999999999,"15min":17.460999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73499999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.41899999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70199999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.29699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95999999999999996,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77900000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.50900000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.622,"15min":0.628},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79600000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.90500000000000003}]},{"osd":5,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.76100000000000001,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.59999999999999998,"15min":0.61399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.72599999999999998}]},{"osd":6,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":0.83899999999999997,"15min":0.72699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":17.439,"5min":17.439,"15min":17.439},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53300000000000003}]},{"osd":7,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85499999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.548,"5min":0.91700000000000004,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":17.420000000000002,"5min":17.420000000000002,"15min":17.420000000000002},"last":0.86299999999999999}]},{"osd":9,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5,"15min":1.5},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":17.396999999999998,"5min":17.396999999999998,"15min":17.396999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":17.489999999999998,"5min":17.489999999999998,"15min":17.489999999999998},"last":0.76900000000000002}]},{"osd":10,"last update":"Wed Jan 31 05:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.83799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1033532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.49299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.59199999999999997}]},{"osd":1,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40500000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.432,"15min":0.433},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.62,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.82699999999999996}]},{"osd":3,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.036,"5min":0.69699999999999995,"15min":0.64100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":7.883,"5min":7.883,"15min":7.883},"last":0.75900000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58199999999999996}]},{"osd":5,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64800000000000002,"15min":0.64400000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.877}]},{"osd":6,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":0.75700000000000001,"15min":0.70099999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":7.9299999999999997,"5min":7.9299999999999997,"15min":7.9299999999999997},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.89500000000000002}]},{"osd":8,"last update":"Wed Jan 31 05:20:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69299999999999995,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.92900000000000005}]},{"osd":9,"last update":"Wed Jan 31 05:20:02 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":7.9020000000000001,"5min":7.9020000000000001,"15min":7.9020000000000001},"last":0.98599999999999999}]},{"osd":10,"last update":"Wed Jan 31 05:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.0369999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":6,"up_from":47,"seq":201863462949,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40392,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704760,"statfs":{"total":95995035648,"available":95953674240,"internally_reserved":0,"allocated":909312,"data_stored":432802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40100000000000002,"15min":0.39700000000000002},"min":{"1min":0.25,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.34799999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.41199999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.48599999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.496}]},{"osd":3,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.5,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75900000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46200000000000002}]},{"osd":4,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.629,"15min":0.628},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67600000000000005,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.60199999999999998}]},{"osd":5,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.64300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.44500000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.67500000000000004}]},{"osd":8,"last update":"Wed Jan 31 05:20:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93300000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.55000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.622}]},{"osd":9,"last update":"Wed Jan 31 05:20:02 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58799999999999997}]},{"osd":10,"last update":"Wed Jan 31 05:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.78800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":449583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:19:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.50800000000000001,"15min":0.48999999999999999},"min":{"1min":0.29999999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.501,"15min":0.495},"min":{"1min":0.33800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.748,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed Jan 31 05:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53000000000000003,"15min":0.50600000000000001},"min":{"1min":0.44700000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83599999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51100000000000001,"15min":0.499},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68100000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.62}]},{"osd":3,"last update":"Wed Jan 31 05:20:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69899999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.497},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.54800000000000004},"min":{"1min":0.38200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.84899999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.56200000000000006}]},{"osd":4,"last update":"Wed Jan 31 05:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.41899999999999998,"15min":0.39100000000000001},"min":{"1min":0.29799999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.42299999999999999,"15min":0.39700000000000002},"min":{"1min":0.33500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77200000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.44800000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58699999999999997,"15min":0.56100000000000005},"min":{"1min":0.47599999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.92500000000000004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72499999999999998,"5min":1.028,"15min":1.028},"last":0.84799999999999998}]},{"osd":6,"last update":"Wed Jan 31 05:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.375,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.627},"min":{"1min":0.40500000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.65800000000000003}]},{"osd":7,"last update":"Wed Jan 31 05:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67300000000000004}]},{"osd":8,"last update":"Wed Jan 31 05:20:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.749},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.86599999999999999}]},{"osd":9,"last update":"Wed Jan 31 05:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.82699999999999996}]},{"osd":10,"last update":"Wed Jan 31 05:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.88100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":913408,"data_stored":432892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":2.766},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":7.835}]},{"osd":2,"last update":"Wed Jan 31 05:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.42899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83299999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":1.585},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.72999999999999998,"15min":0.59199999999999997},"min":{"1min":0.39500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.874,"5min":18.466999999999999,"15min":18.466999999999999},"last":3.4529999999999998}]},{"osd":3,"last update":"Wed Jan 31 05:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.252,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.434},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.79100000000000004}]},{"osd":4,"last update":"Wed Jan 31 05:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.54800000000000004,"15min":0.50900000000000001},"min":{"1min":0.47099999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":2.972},{"interface":"front","average":{"1min":0.63,"5min":0.52200000000000002,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":1.835}]},{"osd":5,"last update":"Wed Jan 31 05:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.54800000000000004,"15min":0.52300000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88400000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":7.9130000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.77600000000000002,"15min":0.624},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.089,"5min":18.541,"15min":18.541},"last":3.359}]},{"osd":6,"last update":"Wed Jan 31 05:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46800000000000003,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.308,"15min":0.308},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":3.2749999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.51700000000000002,"15min":0.502},"min":{"1min":0.42999999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":8.6560000000000006}]},{"osd":7,"last update":"Wed Jan 31 05:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.56799999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":3.1789999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.63700000000000001,"15min":0.622},"min":{"1min":0.502,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":2.105}]},{"osd":8,"last update":"Wed Jan 31 05:20:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.69999999999999996,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":8.6039999999999992},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90000000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":2.1549999999999998}]},{"osd":9,"last update":"Wed Jan 31 05:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":2.8799999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":8.7050000000000001}]},{"osd":10,"last update":"Wed Jan 31 05:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":3.3290000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":2.2250000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2959999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":444334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.42399999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.77900000000000003,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78700000000000003,"5min":1.617,"15min":1.617},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.42999999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77800000000000002,"5min":1.04,"15min":1.04},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.40699999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Jan 31 05:20:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71199999999999997,"5min":1.661,"15min":1.661},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.40899999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.70899999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.71499999999999997}]},{"osd":4,"last update":"Wed Jan 31 05:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58899999999999997,"15min":0.57699999999999996},"min":{"1min":0.48899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":1.077,"15min":1.077},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84499999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.82699999999999996}]},{"osd":5,"last update":"Wed Jan 31 05:20:31 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60999999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.434,"5min":0.45500000000000002,"15min":0.45000000000000001},"min":{"1min":0.33200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.65100000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Wed Jan 31 05:19:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.503,"5min":0.495,"15min":0.495},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.75},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.378,"5min":0.376,"15min":0.376},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.77000000000000002}]},{"osd":7,"last update":"Wed Jan 31 05:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.746,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.878,"15min":0.878},"last":0.66800000000000004}]},{"osd":8,"last update":"Wed Jan 31 05:20:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57099999999999995,"15min":0.57399999999999995},"min":{"1min":0.36599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73699999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.94799999999999995}]},{"osd":9,"last update":"Wed Jan 31 05:20:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.93100000000000005}]},{"osd":10,"last update":"Wed Jan 31 05:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":1.004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149748,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":441566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.32200000000000001},"min":{"1min":0.28499999999999998,"5min":0.189,"15min":0.189},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.34200000000000003,"15min":0.32400000000000001},"min":{"1min":0.27200000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.496}]},{"osd":1,"last update":"Wed Jan 31 05:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54600000000000004,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Jan 31 05:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55900000000000005,"15min":0.54100000000000004},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.85699999999999998}]},{"osd":4,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.36299999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.45400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed Jan 31 05:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.875}]},{"osd":6,"last update":"Wed Jan 31 05:19:50 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.35599999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.50600000000000001,"15min":0.502},"min":{"1min":0.376,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71599999999999997}]},{"osd":7,"last update":"Wed Jan 31 05:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.46500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.41999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73899999999999999}]},{"osd":8,"last update":"Wed Jan 31 05:20:35 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":0.92300000000000004,"15min":0.77700000000000002},"min":{"1min":0.504,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":17.98,"5min":17.98,"15min":17.98},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.82699999999999996}]},{"osd":9,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.80700000000000005}]},{"osd":10,"last update":"Wed Jan 31 05:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.90400000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40404,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704748,"statfs":{"total":95995035648,"available":95953661952,"internally_reserved":0,"allocated":925696,"data_stored":441394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53400000000000003,"15min":0.50800000000000001},"min":{"1min":0.38400000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78900000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.443},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.49199999999999999},"min":{"1min":0.45200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.753,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.43099999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.378},"min":{"1min":0.254,"5min":0.245,"15min":0.245},"max":{"1min":0.52200000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.38800000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57099999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56200000000000006}]},{"osd":2,"last update":"Wed Jan 31 05:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68000000000000005}]},{"osd":3,"last update":"Wed Jan 31 05:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.41899999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85299999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63100000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.72699999999999998}]},{"osd":6,"last update":"Wed Jan 31 05:19:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64400000000000002,"15min":0.63900000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75800000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86299999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.45000000000000001}]},{"osd":8,"last update":"Wed Jan 31 05:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67300000000000004},"min":{"1min":0.54200000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.90200000000000002}]},{"osd":9,"last update":"Wed Jan 31 05:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.78100000000000003}]},{"osd":10,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724587,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1031922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.50800000000000001,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33600000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73399999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.54700000000000004,"15min":0.53600000000000003},"min":{"1min":0.40799999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39900000000000002,"15min":0.39100000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.379},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39300000000000002,"15min":0.38300000000000001},"min":{"1min":0.255,"5min":0.252,"15min":0.252},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.35999999999999999}]},{"osd":3,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.40500000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.45600000000000002}]},{"osd":4,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76200000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.69199999999999995}]},{"osd":6,"last update":"Wed Jan 31 05:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.49199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.503},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.45700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.49199999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.626,"15min":0.625},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.629},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85399999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64700000000000002}]},{"osd":8,"last update":"Wed Jan 31 05:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.77000000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52000000000000002}]},{"osd":9,"last update":"Wed Jan 31 05:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59999999999999998}]},{"osd":10,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.70999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":9561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:20:47.574 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-31T05:20:47.575 INFO:tasks.cephadm:Setting up client nodes... 2024-01-31T05:20:47.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-01-31T05:20:47.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:47 smithi186 ceph-mon[35659]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:47.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:47 smithi186 ceph-mon[35659]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:47.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:47 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/4159742351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:47.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:47 smithi186 ceph-mon[35659]: pgmap v286: 97 pgs: 97 active+clean; 599 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8.3 KiB/s wr, 19 op/s 2024-01-31T05:20:47.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:47 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2861565696' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:47.848 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:47.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:47 smithi045 ceph-mon[31585]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:47.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:47 smithi045 ceph-mon[31585]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:47.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:47 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/4159742351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:47.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:47 smithi045 ceph-mon[31585]: pgmap v286: 97 pgs: 97 active+clean; 599 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8.3 KiB/s wr, 19 op/s 2024-01-31T05:20:47.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:47 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2861565696' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:47 smithi156 ceph-mon[35720]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:47 smithi156 ceph-mon[35720]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:20:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:47 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/4159742351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:20:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:47 smithi156 ceph-mon[35720]: pgmap v286: 97 pgs: 97 active+clean; 599 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8.3 KiB/s wr, 19 op/s 2024-01-31T05:20:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:47 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2861565696' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:48.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:48 smithi045 ceph-mon[31585]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:20:48.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:48 smithi045 ceph-mon[31585]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:20:48.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:48 smithi186 ceph-mon[35659]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:20:48.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:48 smithi186 ceph-mon[35659]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:20:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:48 smithi156 ceph-mon[35720]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:20:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:48 smithi156 ceph-mon[35720]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:20:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.468+0000 7fd566de4700 1 -- 172.21.15.45:0/392131011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd560102360 msgr2=0x7fd560104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.468+0000 7fd566de4700 1 --2- 172.21.15.45:0/392131011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd560102360 0x7fd560104750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd550009990 tx=0x7fd550009ca0 comp rx=0 tx=0).stop 2024-01-31T05:20:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.468+0000 7fd566de4700 1 -- 172.21.15.45:0/392131011 shutdown_connections 2024-01-31T05:20:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.468+0000 7fd566de4700 1 --2- 172.21.15.45:0/392131011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5601075c0 0x7fd5601099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.468+0000 7fd566de4700 1 --2- 172.21.15.45:0/392131011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd560102360 0x7fd560104750 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.468+0000 7fd566de4700 1 --2- 172.21.15.45:0/392131011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd560104c90 0x7fd560107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.468+0000 7fd566de4700 1 -- 172.21.15.45:0/392131011 >> 172.21.15.45:0/392131011 conn(0x7fd5600fbcd0 msgr2=0x7fd5600fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.468+0000 7fd566de4700 1 -- 172.21.15.45:0/392131011 shutdown_connections 2024-01-31T05:20:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.468+0000 7fd566de4700 1 -- 172.21.15.45:0/392131011 wait complete. 2024-01-31T05:20:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.468+0000 7fd566de4700 1 Processor -- start 2024-01-31T05:20:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd566de4700 1 -- start start 2024-01-31T05:20:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd566de4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd560102360 0x7fd560101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd566de4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd560104c90 0x7fd560100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd566de4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5601075c0 0x7fd560100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd566de4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56019b480 con 0x7fd560104c90 2024-01-31T05:20:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd566de4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd560100d90 con 0x7fd5601075c0 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd566de4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd560100f50 con 0x7fd560102360 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd560104c90 0x7fd560100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd560104c90 0x7fd560100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45796/0 (socket says 172.21.15.45:45796) 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd55ffff700 1 -- 172.21.15.45:0/510043520 learned_addr learned my addr 172.21.15.45:0/510043520 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd564b80700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd560102360 0x7fd560101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd565381700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5601075c0 0x7fd560100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd55ffff700 1 -- 172.21.15.45:0/510043520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd560102360 msgr2=0x7fd560101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd55ffff700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd560102360 0x7fd560101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd55ffff700 1 -- 172.21.15.45:0/510043520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5601075c0 msgr2=0x7fd560100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd55ffff700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5601075c0 0x7fd560100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd55ffff700 1 -- 172.21.15.45:0/510043520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5500095f0 con 0x7fd560104c90 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd55ffff700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd560104c90 0x7fd560100310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd55400d980 tx=0x7fd55400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd55dffb700 1 -- 172.21.15.45:0/510043520 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554004220 con 0x7fd560104c90 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.469+0000 7fd566de4700 1 -- 172.21.15.45:0/510043520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5601a5e50 con 0x7fd560104c90 2024-01-31T05:20:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.470+0000 7fd55dffb700 1 -- 172.21.15.45:0/510043520 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd554007640 con 0x7fd560104c90 2024-01-31T05:20:49.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.470+0000 7fd55dffb700 1 -- 172.21.15.45:0/510043520 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554005610 con 0x7fd560104c90 2024-01-31T05:20:49.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.470+0000 7fd566de4700 1 -- 172.21.15.45:0/510043520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5601a6260 con 0x7fd560104c90 2024-01-31T05:20:49.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.471+0000 7fd566de4700 1 -- 172.21.15.45:0/510043520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5600fd850 con 0x7fd560104c90 2024-01-31T05:20:49.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.471+0000 7fd55dffb700 1 -- 172.21.15.45:0/510043520 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fd554004750 con 0x7fd560104c90 2024-01-31T05:20:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.472+0000 7fd55dffb700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd5480a14b0 0x7fd5480a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.472+0000 7fd564b80700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd5480a14b0 0x7fd5480a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.472+0000 7fd55dffb700 1 -- 172.21.15.45:0/510043520 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd5540cc9c0 con 0x7fd560104c90 2024-01-31T05:20:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.472+0000 7fd564b80700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd5480a14b0 0x7fd5480a3970 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd55000b5c0 tx=0x7fd550003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:49.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.477+0000 7fd55dffb700 1 -- 172.21.15.45:0/510043520 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd554095e90 con 0x7fd560104c90 2024-01-31T05:20:49.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.759+0000 7fd566de4700 1 -- 172.21.15.45:0/510043520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fd56010d0c0 con 0x7fd560104c90 2024-01-31T05:20:49.759 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:49 smithi045 ceph-mon[31585]: pgmap v287: 97 pgs: 97 active+clean; 601 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.9 KiB/s wr, 21 op/s 2024-01-31T05:20:49.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.761+0000 7fd55dffb700 1 -- 172.21.15.45:0/510043520 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fd5540160f0 con 0x7fd560104c90 2024-01-31T05:20:49.761 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-01-31T05:20:49.761 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCx2Lllx8BcLRAA0zoBH8+Zlyx4C0LoZnjMfQ== 2024-01-31T05:20:49.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.764+0000 7fd566de4700 1 -- 172.21.15.45:0/510043520 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd5480a14b0 msgr2=0x7fd5480a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:49.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.764+0000 7fd566de4700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd5480a14b0 0x7fd5480a3970 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd55000b5c0 tx=0x7fd550003790 comp rx=0 tx=0).stop 2024-01-31T05:20:49.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.764+0000 7fd566de4700 1 -- 172.21.15.45:0/510043520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd560104c90 msgr2=0x7fd560100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:49.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.764+0000 7fd566de4700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd560104c90 0x7fd560100310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd55400d980 tx=0x7fd55400dc90 comp rx=0 tx=0).stop 2024-01-31T05:20:49.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.764+0000 7fd566de4700 1 -- 172.21.15.45:0/510043520 shutdown_connections 2024-01-31T05:20:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.764+0000 7fd566de4700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5601075c0 0x7fd560100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.764+0000 7fd566de4700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd560102360 0x7fd560101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.764+0000 7fd566de4700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd5480a14b0 0x7fd5480a3970 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.764+0000 7fd566de4700 1 --2- 172.21.15.45:0/510043520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd560104c90 0x7fd560100310 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.764+0000 7fd566de4700 1 -- 172.21.15.45:0/510043520 >> 172.21.15.45:0/510043520 conn(0x7fd5600fbcd0 msgr2=0x7fd560102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.764+0000 7fd566de4700 1 -- 172.21.15.45:0/510043520 shutdown_connections 2024-01-31T05:20:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:49.764+0000 7fd566de4700 1 -- 172.21.15.45:0/510043520 wait complete. 2024-01-31T05:20:49.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:49 smithi186 ceph-mon[35659]: pgmap v287: 97 pgs: 97 active+clean; 601 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.9 KiB/s wr, 21 op/s 2024-01-31T05:20:49.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:49 smithi156 ceph-mon[35720]: pgmap v287: 97 pgs: 97 active+clean; 601 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.9 KiB/s wr, 21 op/s 2024-01-31T05:20:50.458 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:20:50.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-01-31T05:20:50.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-31T05:20:50.502 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-31T05:20:50.502 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-01-31T05:20:50.502 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph mgr dump --format=json 2024-01-31T05:20:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:50.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:50 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/510043520' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-01-31T05:20:50.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:50 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/510043520' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-01-31T05:20:50.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:50 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/510043520' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-01-31T05:20:50.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:50 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/510043520' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-01-31T05:20:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:50 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/510043520' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-01-31T05:20:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:50 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/510043520' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-01-31T05:20:51.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:51 smithi186 ceph-mon[35659]: pgmap v288: 97 pgs: 97 active+clean; 601 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-01-31T05:20:51.845 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:51 smithi045 ceph-mon[31585]: pgmap v288: 97 pgs: 97 active+clean; 601 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-01-31T05:20:51.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:51 smithi156 ceph-mon[35720]: pgmap v288: 97 pgs: 97 active+clean; 601 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-01-31T05:20:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.309+0000 7f2cd0536700 1 -- 172.21.15.45:0/1043624089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc8101110 msgr2=0x7f2cc810d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.309+0000 7f2cd0536700 1 --2- 172.21.15.45:0/1043624089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc8101110 0x7f2cc810d620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009990 tx=0x7f2cc0009ca0 comp rx=0 tx=0).stop 2024-01-31T05:20:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.309+0000 7f2cd0536700 1 -- 172.21.15.45:0/1043624089 shutdown_connections 2024-01-31T05:20:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.309+0000 7f2cd0536700 1 --2- 172.21.15.45:0/1043624089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc8101110 0x7f2cc810d620 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.309+0000 7f2cd0536700 1 --2- 172.21.15.45:0/1043624089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc8106750 0x7f2cc8106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.309+0000 7f2cd0536700 1 --2- 172.21.15.45:0/1043624089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc8100750 0x7f2cc8100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.309+0000 7f2cd0536700 1 -- 172.21.15.45:0/1043624089 >> 172.21.15.45:0/1043624089 conn(0x7f2cc8077810 msgr2=0x7f2cc8077c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.310+0000 7f2cd0536700 1 -- 172.21.15.45:0/1043624089 shutdown_connections 2024-01-31T05:20:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.310+0000 7f2cd0536700 1 -- 172.21.15.45:0/1043624089 wait complete. 2024-01-31T05:20:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.310+0000 7f2cd0536700 1 Processor -- start 2024-01-31T05:20:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.310+0000 7f2cd0536700 1 -- start start 2024-01-31T05:20:52.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.310+0000 7f2cd0536700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc8100750 0x7f2cc819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:52.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.310+0000 7f2cd0536700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc8101110 0x7f2cc819dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:52.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.310+0000 7f2cd0536700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc8106750 0x7f2cc81979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:52.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.310+0000 7f2cd0536700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8198030 con 0x7f2cc8100750 2024-01-31T05:20:52.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.310+0000 7f2cd0536700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8198170 con 0x7f2cc8106750 2024-01-31T05:20:52.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.310+0000 7f2cd0536700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8198330 con 0x7f2cc8101110 2024-01-31T05:20:52.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.310+0000 7f2cce2d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc8100750 0x7f2cc819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:52.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cce2d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc8100750 0x7f2cc819d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45814/0 (socket says 172.21.15.45:45814) 2024-01-31T05:20:52.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cce2d2700 1 -- 172.21.15.45:0/1721880869 learned_addr learned my addr 172.21.15.45:0/1721880869 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:52.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2ccdad1700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc8101110 0x7f2cc819dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:52.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2ccead3700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc8106750 0x7f2cc81979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:52.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cce2d2700 1 -- 172.21.15.45:0/1721880869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc8101110 msgr2=0x7f2cc819dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:52.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cce2d2700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc8101110 0x7f2cc819dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:52.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cce2d2700 1 -- 172.21.15.45:0/1721880869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc8106750 msgr2=0x7f2cc81979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:52.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cce2d2700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc8106750 0x7f2cc81979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:52.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cce2d2700 1 -- 172.21.15.45:0/1721880869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc00095f0 con 0x7f2cc8100750 2024-01-31T05:20:52.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cce2d2700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc8100750 0x7f2cc819d810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4007e20 tx=0x7f2cc400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:52.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cbf7fe700 1 -- 172.21.15.45:0/1721880869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400ac90 con 0x7f2cc8100750 2024-01-31T05:20:52.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cbf7fe700 1 -- 172.21.15.45:0/1721880869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cc400adf0 con 0x7f2cc8100750 2024-01-31T05:20:52.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cd0536700 1 -- 172.21.15.45:0/1721880869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc8198610 con 0x7f2cc8100750 2024-01-31T05:20:52.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cbf7fe700 1 -- 172.21.15.45:0/1721880869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4011690 con 0x7f2cc8100750 2024-01-31T05:20:52.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.311+0000 7f2cd0536700 1 -- 172.21.15.45:0/1721880869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc81a48b0 con 0x7f2cc8100750 2024-01-31T05:20:52.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.312+0000 7f2cd0536700 1 -- 172.21.15.45:0/1721880869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc8109d60 con 0x7f2cc8100750 2024-01-31T05:20:52.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.313+0000 7f2cbf7fe700 1 -- 172.21.15.45:0/1721880869 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f2cc4012420 con 0x7f2cc8100750 2024-01-31T05:20:52.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.314+0000 7f2cbf7fe700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2cb40a1400 0x7f2cb40a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:52.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.314+0000 7f2ccdad1700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2cb40a1400 0x7f2cb40a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:52.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.314+0000 7f2cbf7fe700 1 -- 172.21.15.45:0/1721880869 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2cc40cc920 con 0x7f2cc8100750 2024-01-31T05:20:52.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.314+0000 7f2ccdad1700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2cb40a1400 0x7f2cb40a38c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8077e10 tx=0x7f2cb800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:52.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.318+0000 7f2cbf7fe700 1 -- 172.21.15.45:0/1721880869 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f2cc4095f00 con 0x7f2cc8100750 2024-01-31T05:20:52.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.579+0000 7f2cd0536700 1 -- 172.21.15.45:0/1721880869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f2cc8198f00 con 0x7f2cc8100750 2024-01-31T05:20:52.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.583+0000 7f2cbf7fe700 1 -- 172.21.15.45:0/1721880869 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+259959 (secure 0 0 0) 0x7f2cc4099520 con 0x7f2cc8100750 2024-01-31T05:20:52.586 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:52.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.586+0000 7f2cd0536700 1 -- 172.21.15.45:0/1721880869 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2cb40a1400 msgr2=0x7f2cb40a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:52.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.586+0000 7f2cd0536700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2cb40a1400 0x7f2cb40a38c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8077e10 tx=0x7f2cb800a380 comp rx=0 tx=0).stop 2024-01-31T05:20:52.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.586+0000 7f2cd0536700 1 -- 172.21.15.45:0/1721880869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc8100750 msgr2=0x7f2cc819d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:52.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.586+0000 7f2cd0536700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc8100750 0x7f2cc819d810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4007e20 tx=0x7f2cc400a340 comp rx=0 tx=0).stop 2024-01-31T05:20:52.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.586+0000 7f2cd0536700 1 -- 172.21.15.45:0/1721880869 shutdown_connections 2024-01-31T05:20:52.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.586+0000 7f2cd0536700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc8106750 0x7f2cc81979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:52.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.586+0000 7f2cd0536700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2cb40a1400 0x7f2cb40a38c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:52.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.586+0000 7f2cd0536700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc8100750 0x7f2cc819d810 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:52.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.586+0000 7f2cd0536700 1 --2- 172.21.15.45:0/1721880869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc8101110 0x7f2cc819dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:52.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.586+0000 7f2cd0536700 1 -- 172.21.15.45:0/1721880869 >> 172.21.15.45:0/1721880869 conn(0x7f2cc8077810 msgr2=0x7f2cc806a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:52.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.586+0000 7f2cd0536700 1 -- 172.21.15.45:0/1721880869 shutdown_connections 2024-01-31T05:20:52.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:52.586+0000 7f2cd0536700 1 -- 172.21.15.45:0/1721880869 wait complete. 2024-01-31T05:20:53.301 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":2788245525},{"type":"v1","addr":"172.21.15.45:6801","nonce":2788245525}]},"active_addr":"172.21.15.45:6801/2788245525","active_change":"2024-01-31T05:13:46.510815+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.45:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2742663751}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1829415949}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":782292734}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":948304328}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1579155175}]}]} 2024-01-31T05:20:53.306 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-01-31T05:20:53.306 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-01-31T05:20:53.306 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd dump --format=json 2024-01-31T05:20:53.556 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:53.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:53 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1721880869' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-01-31T05:20:53.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:53 smithi045 ceph-mon[31585]: pgmap v289: 97 pgs: 97 active+clean; 624 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-01-31T05:20:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:53 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1721880869' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-01-31T05:20:53.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:53 smithi156 ceph-mon[35720]: pgmap v289: 97 pgs: 97 active+clean; 624 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-01-31T05:20:53.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:53 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1721880869' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-01-31T05:20:53.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:53 smithi186 ceph-mon[35659]: pgmap v289: 97 pgs: 97 active+clean; 624 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-01-31T05:20:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.197+0000 7f508255f700 1 -- 172.21.15.45:0/1368407929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507c106680 msgr2=0x7f507c106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.197+0000 7f508255f700 1 --2- 172.21.15.45:0/1368407929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507c106680 0x7f507c106a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f506c009990 tx=0x7f506c009ca0 comp rx=0 tx=0).stop 2024-01-31T05:20:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.197+0000 7f508255f700 1 -- 172.21.15.45:0/1368407929 shutdown_connections 2024-01-31T05:20:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.197+0000 7f508255f700 1 --2- 172.21.15.45:0/1368407929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f507c1010d0 0x7f507c10d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.197+0000 7f508255f700 1 --2- 172.21.15.45:0/1368407929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507c106680 0x7f507c106a60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.197+0000 7f508255f700 1 --2- 172.21.15.45:0/1368407929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f507c100710 0x7f507c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.197+0000 7f508255f700 1 -- 172.21.15.45:0/1368407929 >> 172.21.15.45:0/1368407929 conn(0x7f507c076730 msgr2=0x7f507c076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.197+0000 7f508255f700 1 -- 172.21.15.45:0/1368407929 shutdown_connections 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.197+0000 7f508255f700 1 -- 172.21.15.45:0/1368407929 wait complete. 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.198+0000 7f508255f700 1 Processor -- start 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.198+0000 7f508255f700 1 -- start start 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.198+0000 7f508255f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507c100710 0x7f507c102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.198+0000 7f508255f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f507c1010d0 0x7f507c102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.198+0000 7f508255f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f507c106680 0x7f507c102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.198+0000 7f508255f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c1a6180 con 0x7f507c100710 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.198+0000 7f508255f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c1a62f0 con 0x7f507c106680 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.198+0000 7f508255f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c1a64b0 con 0x7f507c1010d0 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.198+0000 7f507bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507c100710 0x7f507c102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.198+0000 7f507bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507c100710 0x7f507c102250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45822/0 (socket says 172.21.15.45:45822) 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.198+0000 7f507bfff700 1 -- 172.21.15.45:0/3561742139 learned_addr learned my addr 172.21.15.45:0/3561742139 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f5080afc700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f507c106680 0x7f507c102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f507b7fe700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f507c1010d0 0x7f507c102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f507bfff700 1 -- 172.21.15.45:0/3561742139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f507c1010d0 msgr2=0x7f507c102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f507bfff700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f507c1010d0 0x7f507c102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f507bfff700 1 -- 172.21.15.45:0/3561742139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f507c106680 msgr2=0x7f507c102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f507bfff700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f507c106680 0x7f507c102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f507bfff700 1 -- 172.21.15.45:0/3561742139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c0095f0 con 0x7f507c100710 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f507b7fe700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f507c1010d0 0x7f507c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f507bfff700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507c100710 0x7f507c102250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f506c000c00 tx=0x7f506c00f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f50797fa700 1 -- 172.21.15.45:0/3561742139 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c01d070 con 0x7f507c100710 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f508255f700 1 -- 172.21.15.45:0/3561742139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c1a6730 con 0x7f507c100710 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f50797fa700 1 -- 172.21.15.45:0/3561742139 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f506c00fde0 con 0x7f507c100710 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f50797fa700 1 -- 172.21.15.45:0/3561742139 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c017770 con 0x7f507c100710 2024-01-31T05:20:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.199+0000 7f508255f700 1 -- 172.21.15.45:0/3561742139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c1a6b50 con 0x7f507c100710 2024-01-31T05:20:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.200+0000 7f508255f700 1 -- 172.21.15.45:0/3561742139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507c109c40 con 0x7f507c100710 2024-01-31T05:20:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.201+0000 7f50797fa700 1 -- 172.21.15.45:0/3561742139 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f506c022aa0 con 0x7f507c100710 2024-01-31T05:20:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.202+0000 7f50797fa700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f50680a1400 0x7f50680a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.202+0000 7f507b7fe700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f50680a1400 0x7f50680a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.202+0000 7f50797fa700 1 -- 172.21.15.45:0/3561742139 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f506c0ce0e0 con 0x7f507c100710 2024-01-31T05:20:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.202+0000 7f507b7fe700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f50680a1400 0x7f50680a38c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f507c076f50 tx=0x7f5064008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.206+0000 7f50797fa700 1 -- 172.21.15.45:0/3561742139 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f506c097580 con 0x7f507c100710 2024-01-31T05:20:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.407+0000 7f508255f700 1 -- 172.21.15.45:0/3561742139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f507c103730 con 0x7f507c100710 2024-01-31T05:20:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.409+0000 7f50797fa700 1 -- 172.21.15.45:0/3561742139 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22133 (secure 0 0 0) 0x7f506c09aba0 con 0x7f507c100710 2024-01-31T05:20:55.408 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:55.408 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":86,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:20:41.704316+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:20:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.411+0000 7f508255f700 1 -- 172.21.15.45:0/3561742139 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f50680a1400 msgr2=0x7f50680a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.411+0000 7f508255f700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f50680a1400 0x7f50680a38c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f507c076f50 tx=0x7f5064008040 comp rx=0 tx=0).stop 2024-01-31T05:20:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.411+0000 7f508255f700 1 -- 172.21.15.45:0/3561742139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507c100710 msgr2=0x7f507c102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.411+0000 7f508255f700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507c100710 0x7f507c102250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f506c000c00 tx=0x7f506c00f880 comp rx=0 tx=0).stop 2024-01-31T05:20:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.411+0000 7f508255f700 1 -- 172.21.15.45:0/3561742139 shutdown_connections 2024-01-31T05:20:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.411+0000 7f508255f700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f507c106680 0x7f507c102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.411+0000 7f508255f700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f50680a1400 0x7f50680a38c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.411+0000 7f508255f700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507c100710 0x7f507c102250 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.411+0000 7f508255f700 1 --2- 172.21.15.45:0/3561742139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f507c1010d0 0x7f507c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.411+0000 7f508255f700 1 -- 172.21.15.45:0/3561742139 >> 172.21.15.45:0/3561742139 conn(0x7f507c076730 msgr2=0x7f507c06a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.411+0000 7f508255f700 1 -- 172.21.15.45:0/3561742139 shutdown_connections 2024-01-31T05:20:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:55.411+0000 7f508255f700 1 -- 172.21.15.45:0/3561742139 wait complete. 2024-01-31T05:20:55.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:55 smithi045 ceph-mon[31585]: pgmap v290: 97 pgs: 97 active+clean; 624 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-01-31T05:20:55.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:55 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3561742139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:55 smithi156 ceph-mon[35720]: pgmap v290: 97 pgs: 97 active+clean; 624 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-01-31T05:20:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:55 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3561742139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:56.047 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-01-31T05:20:56.047 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd dump --format=json 2024-01-31T05:20:56.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:55 smithi186 ceph-mon[35659]: pgmap v290: 97 pgs: 97 active+clean; 624 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-01-31T05:20:56.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:55 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3561742139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:56.290 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.883+0000 7faaa6d51700 1 -- 172.21.15.45:0/3665876588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaa0106650 msgr2=0x7faaa0106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.883+0000 7faaa6d51700 1 --2- 172.21.15.45:0/3665876588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaa0106650 0x7faaa0106a30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa940099c0 tx=0x7faa94009cd0 comp rx=0 tx=0).stop 2024-01-31T05:20:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 -- 172.21.15.45:0/3665876588 shutdown_connections 2024-01-31T05:20:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 --2- 172.21.15.45:0/3665876588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa01021f0 0x7faaa010e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 --2- 172.21.15.45:0/3665876588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaa0106650 0x7faaa0106a30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 --2- 172.21.15.45:0/3665876588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faaa0101830 0x7faaa0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 -- 172.21.15.45:0/3665876588 >> 172.21.15.45:0/3665876588 conn(0x7faaa0078910 msgr2=0x7faaa0078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 -- 172.21.15.45:0/3665876588 shutdown_connections 2024-01-31T05:20:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 -- 172.21.15.45:0/3665876588 wait complete. 2024-01-31T05:20:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 Processor -- start 2024-01-31T05:20:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 -- start start 2024-01-31T05:20:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa0101830 0x7faaa019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faaa01021f0 0x7faaa019de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaa0106650 0x7faaa0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa0198110 con 0x7faaa0106650 2024-01-31T05:20:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa0198250 con 0x7faaa0101830 2024-01-31T05:20:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.884+0000 7faaa6d51700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa0198410 con 0x7faaa01021f0 2024-01-31T05:20:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa52ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaa0106650 0x7faaa0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa4aed700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa0101830 0x7faaa019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa52ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaa0106650 0x7faaa0197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33242/0 (socket says 172.21.15.45:33242) 2024-01-31T05:20:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faa9ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faaa01021f0 0x7faaa019de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa4aed700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa0101830 0x7faaa019d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:34806/0 (socket says 172.21.15.45:34806) 2024-01-31T05:20:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa52ee700 1 -- 172.21.15.45:0/147938527 learned_addr learned my addr 172.21.15.45:0/147938527 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:20:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa52ee700 1 -- 172.21.15.45:0/147938527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faaa01021f0 msgr2=0x7faaa019de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa52ee700 1 --2- 172.21.15.45:0/147938527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faaa01021f0 0x7faaa019de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa52ee700 1 -- 172.21.15.45:0/147938527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa0101830 msgr2=0x7faaa019d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa52ee700 1 --2- 172.21.15.45:0/147938527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa0101830 0x7faaa019d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa52ee700 1 -- 172.21.15.45:0/147938527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa940095f0 con 0x7faaa0106650 2024-01-31T05:20:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa52ee700 1 --2- 172.21.15.45:0/147938527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaa0106650 0x7faaa0197ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faa98007ee0 tx=0x7faa9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faa9dffb700 1 -- 172.21.15.45:0/147938527 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98017070 con 0x7faaa0106650 2024-01-31T05:20:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faa9dffb700 1 -- 172.21.15.45:0/147938527 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa9800f040 con 0x7faaa0106650 2024-01-31T05:20:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa6d51700 1 -- 172.21.15.45:0/147938527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa01986f0 con 0x7faaa0106650 2024-01-31T05:20:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faa9dffb700 1 -- 172.21.15.45:0/147938527 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98005580 con 0x7faaa0106650 2024-01-31T05:20:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.885+0000 7faaa6d51700 1 -- 172.21.15.45:0/147938527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa01a4990 con 0x7faaa0106650 2024-01-31T05:20:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.886+0000 7faaa6d51700 1 -- 172.21.15.45:0/147938527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa0109d10 con 0x7faaa0106650 2024-01-31T05:20:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.887+0000 7faa9dffb700 1 -- 172.21.15.45:0/147938527 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7faa98005760 con 0x7faaa0106650 2024-01-31T05:20:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.888+0000 7faa9dffb700 1 --2- 172.21.15.45:0/147938527 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa900a1400 0x7faa900a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:20:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.888+0000 7faaa4aed700 1 --2- 172.21.15.45:0/147938527 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa900a1400 0x7faa900a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:20:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.888+0000 7faa9dffb700 1 -- 172.21.15.45:0/147938527 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faa980cc210 con 0x7faaa0106650 2024-01-31T05:20:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.888+0000 7faaa4aed700 1 --2- 172.21.15.45:0/147938527 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa900a1400 0x7faa900a38c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faaa0198bf0 tx=0x7faa94003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:20:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:57.892+0000 7faa9dffb700 1 -- 172.21.15.45:0/147938527 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7faa980956e0 con 0x7faaa0106650 2024-01-31T05:20:57.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:57 smithi156 ceph-mon[35720]: pgmap v291: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-01-31T05:20:58.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:57 smithi186 ceph-mon[35659]: pgmap v291: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-01-31T05:20:58.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.094+0000 7faaa6d51700 1 -- 172.21.15.45:0/147938527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faaa0198fe0 con 0x7faaa0106650 2024-01-31T05:20:58.095 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:57 smithi045 ceph-mon[31585]: pgmap v291: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-01-31T05:20:58.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.096+0000 7faa9dffb700 1 -- 172.21.15.45:0/147938527 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22133 (secure 0 0 0) 0x7faa98098d00 con 0x7faaa0106650 2024-01-31T05:20:58.096 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:20:58.097 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":86,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:20:41.704316+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:20:58.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.098+0000 7faaa6d51700 1 -- 172.21.15.45:0/147938527 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa900a1400 msgr2=0x7faa900a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:58.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.098+0000 7faaa6d51700 1 --2- 172.21.15.45:0/147938527 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa900a1400 0x7faa900a38c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faaa0198bf0 tx=0x7faa94003680 comp rx=0 tx=0).stop 2024-01-31T05:20:58.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.098+0000 7faaa6d51700 1 -- 172.21.15.45:0/147938527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaa0106650 msgr2=0x7faaa0197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:20:58.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.098+0000 7faaa6d51700 1 --2- 172.21.15.45:0/147938527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaa0106650 0x7faaa0197ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faa98007ee0 tx=0x7faa9800d3b0 comp rx=0 tx=0).stop 2024-01-31T05:20:58.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.099+0000 7faaa6d51700 1 -- 172.21.15.45:0/147938527 shutdown_connections 2024-01-31T05:20:58.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.099+0000 7faaa6d51700 1 --2- 172.21.15.45:0/147938527 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa900a1400 0x7faa900a38c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:58.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.099+0000 7faaa6d51700 1 --2- 172.21.15.45:0/147938527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaa0106650 0x7faaa0197ab0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:58.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.099+0000 7faaa6d51700 1 --2- 172.21.15.45:0/147938527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa0101830 0x7faaa019d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:58.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.099+0000 7faaa6d51700 1 --2- 172.21.15.45:0/147938527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faaa01021f0 0x7faaa019de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:20:58.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.099+0000 7faaa6d51700 1 -- 172.21.15.45:0/147938527 >> 172.21.15.45:0/147938527 conn(0x7faaa0078910 msgr2=0x7faaa00ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:20:58.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.099+0000 7faaa6d51700 1 -- 172.21.15.45:0/147938527 shutdown_connections 2024-01-31T05:20:58.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:20:58.099+0000 7faaa6d51700 1 -- 172.21.15.45:0/147938527 wait complete. 2024-01-31T05:20:58.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-01-31T05:20:58.671 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-01-31T05:20:58.671 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-01-31T05:20:58.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-01-31T05:20:58.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-01-31T05:20:58.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-01-31T05:20:58.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-01-31T05:20:58.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-01-31T05:20:58.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-01-31T05:20:58.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-01-31T05:20:58.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-01-31T05:20:58.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-01-31T05:20:58.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:58 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/147938527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:59.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:58 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/147938527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:59.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:58 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/147938527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:20:59.396 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:59.462 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:59.462 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:59.464 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:59.464 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:59.464 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:20:59.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:20:59 smithi045 ceph-mon[31585]: pgmap v292: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-01-31T05:20:59.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:20:59 smithi156 ceph-mon[35720]: pgmap v292: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-01-31T05:21:00.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:20:59 smithi186 ceph-mon[35659]: pgmap v292: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-01-31T05:21:01.949 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:01 smithi045 ceph-mon[31585]: pgmap v293: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-01-31T05:21:01.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:01 smithi156 ceph-mon[35720]: pgmap v293: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-01-31T05:21:02.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:01 smithi186 ceph-mon[35659]: pgmap v293: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-01-31T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:02.903 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:03.623 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:03 smithi045 ceph-mon[31585]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-01-31T05:21:03.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:03 smithi156 ceph-mon[35720]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-01-31T05:21:03.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:03 smithi186 ceph-mon[35659]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-01-31T05:21:04.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.414+0000 7f3c2b713700 1 -- 172.21.15.45:0/3198733768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c2406a6e0 msgr2=0x7f3c2410e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:04.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.414+0000 7f3c2b713700 1 --2- 172.21.15.45:0/3198733768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c2406a6e0 0x7f3c2410e640 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00b0d0 tx=0x7f3c1c00b3e0 comp rx=0 tx=0).stop 2024-01-31T05:21:04.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.414+0000 7f3c2b713700 1 -- 172.21.15.45:0/3198733768 shutdown_connections 2024-01-31T05:21:04.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.414+0000 7f3c2b713700 1 --2- 172.21.15.45:0/3198733768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c2406a6e0 0x7f3c2410e640 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:04.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.414+0000 7f3c2b713700 1 --2- 172.21.15.45:0/3198733768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c24107790 0x7f3c24107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:04.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.414+0000 7f3c2b713700 1 --2- 172.21.15.45:0/3198733768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c24069d20 0x7f3c2406a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:04.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.414+0000 7f3c2b713700 1 -- 172.21.15.45:0/3198733768 >> 172.21.15.45:0/3198733768 conn(0x7f3c24076730 msgr2=0x7f3c24076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:04.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.414+0000 7f3c2b713700 1 -- 172.21.15.45:0/3198733768 shutdown_connections 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.414+0000 7f3c2b713700 1 -- 172.21.15.45:0/3198733768 wait complete. 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.415+0000 7f3c2b713700 1 Processor -- start 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.415+0000 7f3c2b713700 1 -- start start 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.415+0000 7f3c2b713700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c24069d20 0x7f3c2419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.415+0000 7f3c2b713700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c2406a6e0 0x7f3c2419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.415+0000 7f3c2b713700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c24107790 0x7f3c24197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.415+0000 7f3c2b713700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c24198140 con 0x7f3c24069d20 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.415+0000 7f3c2b713700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c24198280 con 0x7f3c24107790 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.415+0000 7f3c2b713700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c24198440 con 0x7f3c2406a6e0 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c294af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c24069d20 0x7f3c2419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c294af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c24069d20 0x7f3c2419d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33254/0 (socket says 172.21.15.45:33254) 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c294af700 1 -- 172.21.15.45:0/207308220 learned_addr learned my addr 172.21.15.45:0/207308220 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c28cae700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c2406a6e0 0x7f3c2419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c29cb0700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c24107790 0x7f3c24197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c294af700 1 -- 172.21.15.45:0/207308220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c2406a6e0 msgr2=0x7f3c2419de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c294af700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c2406a6e0 0x7f3c2419de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c294af700 1 -- 172.21.15.45:0/207308220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c24107790 msgr2=0x7f3c24197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c294af700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c24107790 0x7f3c24197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c294af700 1 -- 172.21.15.45:0/207308220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c009d00 con 0x7f3c24069d20 2024-01-31T05:21:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c29cb0700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c24107790 0x7f3c24197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:21:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c294af700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c24069d20 0x7f3c2419d8d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c9b0 tx=0x7f3c2000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c1a7fc700 1 -- 172.21.15.45:0/207308220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20007b00 con 0x7f3c24069d20 2024-01-31T05:21:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c2b713700 1 -- 172.21.15.45:0/207308220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c24198720 con 0x7f3c24069d20 2024-01-31T05:21:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c1a7fc700 1 -- 172.21.15.45:0/207308220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c20007c60 con 0x7f3c24069d20 2024-01-31T05:21:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c1a7fc700 1 -- 172.21.15.45:0/207308220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20018660 con 0x7f3c24069d20 2024-01-31T05:21:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.416+0000 7f3c2b713700 1 -- 172.21.15.45:0/207308220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c241a4970 con 0x7f3c24069d20 2024-01-31T05:21:04.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.417+0000 7f3c2b713700 1 -- 172.21.15.45:0/207308220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3c24102f10 con 0x7f3c24069d20 2024-01-31T05:21:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.418+0000 7f3c1a7fc700 1 -- 172.21.15.45:0/207308220 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f3c2000f450 con 0x7f3c24069d20 2024-01-31T05:21:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.419+0000 7f3c1a7fc700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3c100a1400 0x7f3c100a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.419+0000 7f3c28cae700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3c100a1400 0x7f3c100a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.419+0000 7f3c1a7fc700 1 -- 172.21.15.45:0/207308220 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3c200ccee0 con 0x7f3c24069d20 2024-01-31T05:21:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.419+0000 7f3c1a7fc700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645] conn(0x7f3c100aabe0 0x7f3c100ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.419+0000 7f3c1a7fc700 1 -- 172.21.15.45:0/207308220 --> [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c100ad6b0 con 0x7f3c100aabe0 2024-01-31T05:21:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.419+0000 7f3c1a7fc700 1 -- 172.21.15.45:0/207308220 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c200cd380 con 0x7f3c24069d20 2024-01-31T05:21:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.419+0000 7f3c29cb0700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645] conn(0x7f3c100aabe0 0x7f3c100ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.419+0000 7f3c28cae700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3c100a1400 0x7f3c100a38c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3c24076d10 tx=0x7f3c1400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.420+0000 7f3c29cb0700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645] conn(0x7f3c100aabe0 0x7f3c100ad000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.420+0000 7f3c1a7fc700 1 -- 172.21.15.45:0/207308220 <== osd.3 v2:172.21.15.45:6810/3653915645 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3c100ad6b0 con 0x7f3c100aabe0 2024-01-31T05:21:04.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.437+0000 7f3c2b713700 1 -- 172.21.15.45:0/207308220 --> [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3c241119f0 con 0x7f3c100aabe0 2024-01-31T05:21:04.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.437+0000 7f3c1a7fc700 1 -- 172.21.15.45:0/207308220 <== osd.3 v2:172.21.15.45:6810/3653915645 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3c241119f0 con 0x7f3c100aabe0 2024-01-31T05:21:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.437+0000 7f3c2b713700 1 -- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645] conn(0x7f3c100aabe0 msgr2=0x7f3c100ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.437+0000 7f3c2b713700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645] conn(0x7f3c100aabe0 0x7f3c100ad000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.437+0000 7f3c2b713700 1 -- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3c100a1400 msgr2=0x7f3c100a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.437+0000 7f3c2b713700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3c100a1400 0x7f3c100a38c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3c24076d10 tx=0x7f3c1400a300 comp rx=0 tx=0).stop 2024-01-31T05:21:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.437+0000 7f3c2b713700 1 -- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c24069d20 msgr2=0x7f3c2419d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.437+0000 7f3c2b713700 1 --2- 172.21.15.45:0/207308220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c24069d20 0x7f3c2419d8d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c9b0 tx=0x7f3c2000ccc0 comp rx=0 tx=0).stop 2024-01-31T05:21:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.438+0000 7f3c29cb0700 1 -- 172.21.15.45:0/207308220 reap_dead start 2024-01-31T05:21:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.438+0000 7f3c2b713700 1 -- 172.21.15.45:0/207308220 shutdown_connections 2024-01-31T05:21:04.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.438+0000 7f3c2b713700 1 -- 172.21.15.45:0/207308220 >> 172.21.15.45:0/207308220 conn(0x7f3c24076730 msgr2=0x7f3c240ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:04.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.438+0000 7f3c2b713700 1 -- 172.21.15.45:0/207308220 shutdown_connections 2024-01-31T05:21:04.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:04.438+0000 7f3c2b713700 1 -- 172.21.15.45:0/207308220 wait complete. 2024-01-31T05:21:05.106 INFO:teuthology.orchestra.run.smithi045.stdout:111669149753 2024-01-31T05:21:05.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-01-31T05:21:05.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:05 smithi045 ceph-mon[31585]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-01-31T05:21:05.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:05 smithi156 ceph-mon[35720]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-01-31T05:21:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.980+0000 7f920ad6e700 1 -- 172.21.15.45:0/3221270224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920406a900 msgr2=0x7f920410e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.980+0000 7f920ad6e700 1 --2- 172.21.15.45:0/3221270224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920406a900 0x7f920410e640 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0099c0 tx=0x7f91fc009cd0 comp rx=0 tx=0).stop 2024-01-31T05:21:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.981+0000 7f920ad6e700 1 -- 172.21.15.45:0/3221270224 shutdown_connections 2024-01-31T05:21:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.981+0000 7f920ad6e700 1 --2- 172.21.15.45:0/3221270224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920406a900 0x7f920410e640 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.981+0000 7f920ad6e700 1 --2- 172.21.15.45:0/3221270224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9204107790 0x7f9204107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.981+0000 7f920ad6e700 1 --2- 172.21.15.45:0/3221270224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9204069f40 0x7f920406a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.981+0000 7f920ad6e700 1 -- 172.21.15.45:0/3221270224 >> 172.21.15.45:0/3221270224 conn(0x7f9204076720 msgr2=0x7f9204076b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.981+0000 7f920ad6e700 1 -- 172.21.15.45:0/3221270224 shutdown_connections 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.981+0000 7f920ad6e700 1 -- 172.21.15.45:0/3221270224 wait complete. 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.981+0000 7f920ad6e700 1 Processor -- start 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.981+0000 7f920ad6e700 1 -- start start 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.981+0000 7f920ad6e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9204069f40 0x7f920419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920ad6e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f920406a900 0x7f920419df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920ad6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9204107790 0x7f9204197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920ad6e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041982c0 con 0x7f9204107790 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920ad6e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9204198430 con 0x7f9204069f40 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920ad6e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041985f0 con 0x7f920406a900 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920930b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9204107790 0x7f9204197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f9208b0a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9204069f40 0x7f920419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920930b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9204107790 0x7f9204197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33274/0 (socket says 172.21.15.45:33274) 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920930b700 1 -- 172.21.15.45:0/1588747491 learned_addr learned my addr 172.21.15.45:0/1588747491 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f9203fff700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f920406a900 0x7f920419df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920930b700 1 -- 172.21.15.45:0/1588747491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f920406a900 msgr2=0x7f920419df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920930b700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f920406a900 0x7f920419df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920930b700 1 -- 172.21.15.45:0/1588747491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9204069f40 msgr2=0x7f920419da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920930b700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9204069f40 0x7f920419da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920930b700 1 -- 172.21.15.45:0/1588747491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc0095f0 con 0x7f9204107790 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920930b700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9204107790 0x7f9204197b10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f91fc004760 tx=0x7f91fc004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f9201ffb700 1 -- 172.21.15.45:0/1588747491 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc01d070 con 0x7f9204107790 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f9201ffb700 1 -- 172.21.15.45:0/1588747491 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91fc004cf0 con 0x7f9204107790 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f9201ffb700 1 -- 172.21.15.45:0/1588747491 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc00f950 con 0x7f9204107790 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.982+0000 7f920ad6e700 1 -- 172.21.15.45:0/1588747491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9204198870 con 0x7f9204107790 2024-01-31T05:21:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.983+0000 7f920ad6e700 1 -- 172.21.15.45:0/1588747491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92041a49c0 con 0x7f9204107790 2024-01-31T05:21:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.983+0000 7f920ad6e700 1 -- 172.21.15.45:0/1588747491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9204111a60 con 0x7f9204107790 2024-01-31T05:21:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.985+0000 7f9201ffb700 1 -- 172.21.15.45:0/1588747491 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f91fc005120 con 0x7f9204107790 2024-01-31T05:21:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.986+0000 7f9201ffb700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f91f00a1500 0x7f91f00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.986+0000 7f9208b0a700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f91f00a1500 0x7f91f00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.986+0000 7f9201ffb700 1 -- 172.21.15.45:0/1588747491 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f91fc0ce1a0 con 0x7f9204107790 2024-01-31T05:21:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.987+0000 7f9201ffb700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681] conn(0x7f91f00aace0 0x7f91f00ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.987+0000 7f9201ffb700 1 -- 172.21.15.45:0/1588747491 --> [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91f00ad7b0 con 0x7f91f00aace0 2024-01-31T05:21:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.987+0000 7f9201ffb700 1 -- 172.21.15.45:0/1588747491 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91fc0ce550 con 0x7f9204107790 2024-01-31T05:21:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.987+0000 7f9203fff700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681] conn(0x7f91f00aace0 0x7f91f00ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.987+0000 7f9208b0a700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f91f00a1500 0x7f91f00a39c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f91f40060b0 tx=0x7f91f400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.987+0000 7f9203fff700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681] conn(0x7f91f00aace0 0x7f91f00ad100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:05.988+0000 7f9201ffb700 1 -- 172.21.15.45:0/1588747491 <== osd.6 v2:172.21.15.45:6818/2753791681 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f91f00ad7b0 con 0x7f91f00aace0 2024-01-31T05:21:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.021+0000 7f920ad6e700 1 -- 172.21.15.45:0/1588747491 --> [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9204061e70 con 0x7f91f00aace0 2024-01-31T05:21:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.021+0000 7f9201ffb700 1 -- 172.21.15.45:0/1588747491 <== osd.6 v2:172.21.15.45:6818/2753791681 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9204061e70 con 0x7f91f00aace0 2024-01-31T05:21:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.021+0000 7f920ad6e700 1 -- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681] conn(0x7f91f00aace0 msgr2=0x7f91f00ad100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.021+0000 7f920ad6e700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681] conn(0x7f91f00aace0 0x7f91f00ad100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.022+0000 7f920ad6e700 1 -- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f91f00a1500 msgr2=0x7f91f00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.022+0000 7f920ad6e700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f91f00a1500 0x7f91f00a39c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f91f40060b0 tx=0x7f91f400c040 comp rx=0 tx=0).stop 2024-01-31T05:21:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.022+0000 7f920ad6e700 1 -- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9204107790 msgr2=0x7f9204197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.022+0000 7f920ad6e700 1 --2- 172.21.15.45:0/1588747491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9204107790 0x7f9204197b10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f91fc004760 tx=0x7f91fc004790 comp rx=0 tx=0).stop 2024-01-31T05:21:06.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.022+0000 7f920930b700 1 -- 172.21.15.45:0/1588747491 reap_dead start 2024-01-31T05:21:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.022+0000 7f920ad6e700 1 -- 172.21.15.45:0/1588747491 shutdown_connections 2024-01-31T05:21:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.022+0000 7f920ad6e700 1 -- 172.21.15.45:0/1588747491 >> 172.21.15.45:0/1588747491 conn(0x7f9204076720 msgr2=0x7f92040ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.022+0000 7f920ad6e700 1 -- 172.21.15.45:0/1588747491 shutdown_connections 2024-01-31T05:21:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:06.022+0000 7f920ad6e700 1 -- 172.21.15.45:0/1588747491 wait complete. 2024-01-31T05:21:06.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:05 smithi186 ceph-mon[35659]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-01-31T05:21:07.765 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:07 smithi045 ceph-mon[31585]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-01-31T05:21:07.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:07 smithi156 ceph-mon[35720]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-01-31T05:21:08.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:07 smithi186 ceph-mon[35659]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-01-31T05:21:09.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:09 smithi156 ceph-mon[35720]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-01-31T05:21:10.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:09 smithi045 ceph-mon[31585]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-01-31T05:21:10.079 INFO:teuthology.orchestra.run.smithi045.stdout:201863462954 2024-01-31T05:21:10.079 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-01-31T05:21:10.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:09 smithi186 ceph-mon[35659]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-01-31T05:21:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.730+0000 7fb9063c2700 1 -- 172.21.15.45:0/1346850374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900100710 msgr2=0x7fb900100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.730+0000 7fb9063c2700 1 --2- 172.21.15.45:0/1346850374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900100710 0x7fb900100b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009990 tx=0x7fb8f4009ca0 comp rx=0 tx=0).stop 2024-01-31T05:21:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.730+0000 7fb9063c2700 1 -- 172.21.15.45:0/1346850374 shutdown_connections 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.730+0000 7fb9063c2700 1 --2- 172.21.15.45:0/1346850374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9001010d0 0x7fb90010d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.730+0000 7fb9063c2700 1 --2- 172.21.15.45:0/1346850374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9001066c0 0x7fb900106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.730+0000 7fb9063c2700 1 --2- 172.21.15.45:0/1346850374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900100710 0x7fb900100b90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.730+0000 7fb9063c2700 1 -- 172.21.15.45:0/1346850374 >> 172.21.15.45:0/1346850374 conn(0x7fb900076720 msgr2=0x7fb900076b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.730+0000 7fb9063c2700 1 -- 172.21.15.45:0/1346850374 shutdown_connections 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.730+0000 7fb9063c2700 1 -- 172.21.15.45:0/1346850374 wait complete. 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.731+0000 7fb9063c2700 1 Processor -- start 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.731+0000 7fb9063c2700 1 -- start start 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.731+0000 7fb9063c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900100710 0x7fb900102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.731+0000 7fb9063c2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9001010d0 0x7fb9001027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.731+0000 7fb9063c2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9001066c0 0x7fb900102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.731+0000 7fb9063c2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9001a6110 con 0x7fb900100710 2024-01-31T05:21:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.731+0000 7fb9063c2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9001a6280 con 0x7fb9001010d0 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.731+0000 7fb9063c2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9001a6440 con 0x7fb9001066c0 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.731+0000 7fb8fffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900100710 0x7fb900102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.731+0000 7fb8fffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900100710 0x7fb900102280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58464/0 (socket says 172.21.15.45:58464) 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.731+0000 7fb8fffff700 1 -- 172.21.15.45:0/2931798791 learned_addr learned my addr 172.21.15.45:0/2931798791 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.731+0000 7fb8ff7fe700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9001010d0 0x7fb9001027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.732+0000 7fb90495f700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9001066c0 0x7fb900102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.732+0000 7fb8fffff700 1 -- 172.21.15.45:0/2931798791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9001066c0 msgr2=0x7fb900102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.732+0000 7fb8fffff700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9001066c0 0x7fb900102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.732+0000 7fb8fffff700 1 -- 172.21.15.45:0/2931798791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9001010d0 msgr2=0x7fb9001027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.732+0000 7fb8fffff700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9001010d0 0x7fb9001027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.732+0000 7fb8fffff700 1 -- 172.21.15.45:0/2931798791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f40095f0 con 0x7fb900100710 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.732+0000 7fb8fffff700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900100710 0x7fb900102280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000b530 tx=0x7fb8f000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.732+0000 7fb8fd7fa700 1 -- 172.21.15.45:0/2931798791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0011830 con 0x7fb900100710 2024-01-31T05:21:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.732+0000 7fb8fd7fa700 1 -- 172.21.15.45:0/2931798791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8f0011e50 con 0x7fb900100710 2024-01-31T05:21:11.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.732+0000 7fb9063c2700 1 -- 172.21.15.45:0/2931798791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9001a6720 con 0x7fb900100710 2024-01-31T05:21:11.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.732+0000 7fb8fd7fa700 1 -- 172.21.15.45:0/2931798791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f000f2e0 con 0x7fb900100710 2024-01-31T05:21:11.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.732+0000 7fb9063c2700 1 -- 172.21.15.45:0/2931798791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9001a6bf0 con 0x7fb900100710 2024-01-31T05:21:11.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.733+0000 7fb9063c2700 1 -- 172.21.15.45:0/2931798791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb900110960 con 0x7fb900100710 2024-01-31T05:21:11.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.734+0000 7fb8fd7fa700 1 -- 172.21.15.45:0/2931798791 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fb8f0010380 con 0x7fb900100710 2024-01-31T05:21:11.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.735+0000 7fb8fd7fa700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8e80a1400 0x7fb8e80a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:11.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.735+0000 7fb8ff7fe700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8e80a1400 0x7fb8e80a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:11.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.735+0000 7fb8fd7fa700 1 -- 172.21.15.45:0/2931798791 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb8f00cccc0 con 0x7fb900100710 2024-01-31T05:21:11.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.735+0000 7fb8fd7fa700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396] conn(0x7fb8e80aabe0 0x7fb8e80ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:11.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.735+0000 7fb8fd7fa700 1 -- 172.21.15.45:0/2931798791 --> [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8e80ad6b0 con 0x7fb8e80aabe0 2024-01-31T05:21:11.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.735+0000 7fb8fd7fa700 1 -- 172.21.15.45:0/2931798791 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8f00cd030 con 0x7fb900100710 2024-01-31T05:21:11.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.735+0000 7fb8ff7fe700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8e80a1400 0x7fb8e80a38c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b5c0 tx=0x7fb8f4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:11.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.735+0000 7fb90495f700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396] conn(0x7fb8e80aabe0 0x7fb8e80ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:11.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.736+0000 7fb90495f700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396] conn(0x7fb8e80aabe0 0x7fb8e80ad000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:11.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.737+0000 7fb8fd7fa700 1 -- 172.21.15.45:0/2931798791 <== osd.2 v2:172.21.15.186:6800/811586396 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb8e80ad6b0 con 0x7fb8e80aabe0 2024-01-31T05:21:11.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.753+0000 7fb9063c2700 1 -- 172.21.15.45:0/2931798791 --> [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9001056a0 con 0x7fb8e80aabe0 2024-01-31T05:21:11.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.753+0000 7fb8fd7fa700 1 -- 172.21.15.45:0/2931798791 <== osd.2 v2:172.21.15.186:6800/811586396 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb9001056a0 con 0x7fb8e80aabe0 2024-01-31T05:21:11.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.754+0000 7fb9063c2700 1 -- 172.21.15.45:0/2931798791 >> [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396] conn(0x7fb8e80aabe0 msgr2=0x7fb8e80ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:11.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.754+0000 7fb9063c2700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396] conn(0x7fb8e80aabe0 0x7fb8e80ad000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:11.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.754+0000 7fb9063c2700 1 -- 172.21.15.45:0/2931798791 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8e80a1400 msgr2=0x7fb8e80a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:11.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.754+0000 7fb9063c2700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8e80a1400 0x7fb8e80a38c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b5c0 tx=0x7fb8f4005960 comp rx=0 tx=0).stop 2024-01-31T05:21:11.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.754+0000 7fb9063c2700 1 -- 172.21.15.45:0/2931798791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900100710 msgr2=0x7fb900102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.754+0000 7fb9063c2700 1 --2- 172.21.15.45:0/2931798791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb900100710 0x7fb900102280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000b530 tx=0x7fb8f000b840 comp rx=0 tx=0).stop 2024-01-31T05:21:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.754+0000 7fb90495f700 1 -- 172.21.15.45:0/2931798791 reap_dead start 2024-01-31T05:21:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.754+0000 7fb9063c2700 1 -- 172.21.15.45:0/2931798791 shutdown_connections 2024-01-31T05:21:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.754+0000 7fb9063c2700 1 -- 172.21.15.45:0/2931798791 >> 172.21.15.45:0/2931798791 conn(0x7fb900076720 msgr2=0x7fb90006a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.754+0000 7fb9063c2700 1 -- 172.21.15.45:0/2931798791 shutdown_connections 2024-01-31T05:21:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:11.754+0000 7fb9063c2700 1 -- 172.21.15.45:0/2931798791 wait complete. 2024-01-31T05:21:11.763 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:11 smithi045 ceph-mon[31585]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-01-31T05:21:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:11 smithi156 ceph-mon[35720]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-01-31T05:21:12.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:11 smithi186 ceph-mon[35659]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-01-31T05:21:13.079 INFO:teuthology.orchestra.run.smithi045.stdout:77309411392 2024-01-31T05:21:13.079 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-01-31T05:21:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.494+0000 7fea214a5700 1 -- 172.21.15.45:0/3829831270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea1c107790 msgr2=0x7fea1c107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.494+0000 7fea214a5700 1 --2- 172.21.15.45:0/3829831270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea1c107790 0x7fea1c107b70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0099c0 tx=0x7fea0c009cd0 comp rx=0 tx=0).stop 2024-01-31T05:21:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.494+0000 7fea214a5700 1 -- 172.21.15.45:0/3829831270 shutdown_connections 2024-01-31T05:21:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.494+0000 7fea214a5700 1 --2- 172.21.15.45:0/3829831270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c06a900 0x7fea1c10e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.494+0000 7fea214a5700 1 --2- 172.21.15.45:0/3829831270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea1c107790 0x7fea1c107b70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.494+0000 7fea214a5700 1 --2- 172.21.15.45:0/3829831270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea1c069f40 0x7fea1c06a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.494+0000 7fea214a5700 1 -- 172.21.15.45:0/3829831270 >> 172.21.15.45:0/3829831270 conn(0x7fea1c0766f0 msgr2=0x7fea1c076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.494+0000 7fea214a5700 1 -- 172.21.15.45:0/3829831270 shutdown_connections 2024-01-31T05:21:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.494+0000 7fea214a5700 1 -- 172.21.15.45:0/3829831270 wait complete. 2024-01-31T05:21:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.495+0000 7fea214a5700 1 Processor -- start 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.495+0000 7fea214a5700 1 -- start start 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.495+0000 7fea214a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea1c069f40 0x7fea1c19d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.495+0000 7fea214a5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea1c06a900 0x7fea1c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.495+0000 7fea214a5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c107790 0x7fea1c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.495+0000 7fea214a5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c1980c0 con 0x7fea1c069f40 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.495+0000 7fea214a5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c198200 con 0x7fea1c107790 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.495+0000 7fea214a5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c1983c0 con 0x7fea1c06a900 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.495+0000 7fea1affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea1c069f40 0x7fea1c19d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.495+0000 7fea1affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea1c069f40 0x7fea1c19d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58474/0 (socket says 172.21.15.45:58474) 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.495+0000 7fea1affd700 1 -- 172.21.15.45:0/2978309559 learned_addr learned my addr 172.21.15.45:0/2978309559 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.495+0000 7fea1b7fe700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c107790 0x7fea1c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.496+0000 7fea1affd700 1 -- 172.21.15.45:0/2978309559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea1c06a900 msgr2=0x7fea1c19dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.496+0000 7fea1affd700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea1c06a900 0x7fea1c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.496+0000 7fea1affd700 1 -- 172.21.15.45:0/2978309559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c107790 msgr2=0x7fea1c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.496+0000 7fea1affd700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c107790 0x7fea1c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:13.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.496+0000 7fea1affd700 1 -- 172.21.15.45:0/2978309559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c0095f0 con 0x7fea1c069f40 2024-01-31T05:21:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.496+0000 7fea1affd700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea1c069f40 0x7fea1c19d8a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00b580 tx=0x7fea0c003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.496+0000 7fea03fff700 1 -- 172.21.15.45:0/2978309559 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c01d070 con 0x7fea1c069f40 2024-01-31T05:21:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.496+0000 7fea214a5700 1 -- 172.21.15.45:0/2978309559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1c198640 con 0x7fea1c069f40 2024-01-31T05:21:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.496+0000 7fea03fff700 1 -- 172.21.15.45:0/2978309559 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea0c00fda0 con 0x7fea1c069f40 2024-01-31T05:21:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.496+0000 7fea03fff700 1 -- 172.21.15.45:0/2978309559 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0037c0 con 0x7fea1c069f40 2024-01-31T05:21:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.496+0000 7fea214a5700 1 -- 172.21.15.45:0/2978309559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c1a4940 con 0x7fea1c069f40 2024-01-31T05:21:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.497+0000 7fea214a5700 1 -- 172.21.15.45:0/2978309559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea1c102f10 con 0x7fea1c069f40 2024-01-31T05:21:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.498+0000 7fea03fff700 1 -- 172.21.15.45:0/2978309559 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fea0c022700 con 0x7fea1c069f40 2024-01-31T05:21:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.498+0000 7fea03fff700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fea040a1400 0x7fea040a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.499+0000 7fea1a7fc700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fea040a1400 0x7fea040a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.499+0000 7fea03fff700 1 -- 172.21.15.45:0/2978309559 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fea0c0ce260 con 0x7fea1c069f40 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.499+0000 7fea03fff700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307] conn(0x7fea040aabe0 0x7fea040ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.499+0000 7fea03fff700 1 -- 172.21.15.45:0/2978309559 --> [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea040ad6b0 con 0x7fea040aabe0 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.499+0000 7fea03fff700 1 -- 172.21.15.45:0/2978309559 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea0c0ce700 con 0x7fea1c069f40 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.499+0000 7fea1a7fc700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fea040a1400 0x7fea040a38c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fea1c076dd0 tx=0x7fea10005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.499+0000 7fea1b7fe700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307] conn(0x7fea040aabe0 0x7fea040ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.499+0000 7fea1b7fe700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307] conn(0x7fea040aabe0 0x7fea040ad000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.500+0000 7fea03fff700 1 -- 172.21.15.45:0/2978309559 <== osd.5 v2:172.21.15.186:6808/2142758307 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fea040ad6b0 con 0x7fea040aabe0 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.517+0000 7fea214a5700 1 -- 172.21.15.45:0/2978309559 --> [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea1c111a70 con 0x7fea040aabe0 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.518+0000 7fea03fff700 1 -- 172.21.15.45:0/2978309559 <== osd.5 v2:172.21.15.186:6808/2142758307 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fea1c111a70 con 0x7fea040aabe0 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.518+0000 7fea214a5700 1 -- 172.21.15.45:0/2978309559 >> [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307] conn(0x7fea040aabe0 msgr2=0x7fea040ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.518+0000 7fea214a5700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307] conn(0x7fea040aabe0 0x7fea040ad000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.518+0000 7fea214a5700 1 -- 172.21.15.45:0/2978309559 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fea040a1400 msgr2=0x7fea040a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.518+0000 7fea214a5700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fea040a1400 0x7fea040a38c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fea1c076dd0 tx=0x7fea10005c80 comp rx=0 tx=0).stop 2024-01-31T05:21:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.518+0000 7fea214a5700 1 -- 172.21.15.45:0/2978309559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea1c069f40 msgr2=0x7fea1c19d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.518+0000 7fea214a5700 1 --2- 172.21.15.45:0/2978309559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea1c069f40 0x7fea1c19d8a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00b580 tx=0x7fea0c003710 comp rx=0 tx=0).stop 2024-01-31T05:21:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.518+0000 7fea1b7fe700 1 -- 172.21.15.45:0/2978309559 reap_dead start 2024-01-31T05:21:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.518+0000 7fea214a5700 1 -- 172.21.15.45:0/2978309559 shutdown_connections 2024-01-31T05:21:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.518+0000 7fea214a5700 1 -- 172.21.15.45:0/2978309559 >> 172.21.15.45:0/2978309559 conn(0x7fea1c0766f0 msgr2=0x7fea1c0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.518+0000 7fea214a5700 1 -- 172.21.15.45:0/2978309559 shutdown_connections 2024-01-31T05:21:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:12.518+0000 7fea214a5700 1 -- 172.21.15.45:0/2978309559 wait complete. 2024-01-31T05:21:13.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:13 smithi045 ceph-mon[31585]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-01-31T05:21:13.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:13 smithi156 ceph-mon[35720]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-01-31T05:21:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:13 smithi186 ceph-mon[35659]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-01-31T05:21:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.100+0000 7fd2c7c54700 1 -- 172.21.15.45:0/2805291460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c0100710 msgr2=0x7fd2c0100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.100+0000 7fd2c7c54700 1 --2- 172.21.15.45:0/2805291460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c0100710 0x7fd2c0100b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009990 tx=0x7fd2b0009ca0 comp rx=0 tx=0).stop 2024-01-31T05:21:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.100+0000 7fd2c7c54700 1 -- 172.21.15.45:0/2805291460 shutdown_connections 2024-01-31T05:21:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.100+0000 7fd2c7c54700 1 --2- 172.21.15.45:0/2805291460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c01010d0 0x7fd2c010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.100+0000 7fd2c7c54700 1 --2- 172.21.15.45:0/2805291460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c0106680 0x7fd2c0106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.100+0000 7fd2c7c54700 1 --2- 172.21.15.45:0/2805291460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c0100710 0x7fd2c0100b90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.100+0000 7fd2c7c54700 1 -- 172.21.15.45:0/2805291460 >> 172.21.15.45:0/2805291460 conn(0x7fd2c0076730 msgr2=0x7fd2c0076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.101+0000 7fd2c7c54700 1 -- 172.21.15.45:0/2805291460 shutdown_connections 2024-01-31T05:21:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.101+0000 7fd2c7c54700 1 -- 172.21.15.45:0/2805291460 wait complete. 2024-01-31T05:21:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.101+0000 7fd2c7c54700 1 Processor -- start 2024-01-31T05:21:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.101+0000 7fd2c7c54700 1 -- start start 2024-01-31T05:21:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.101+0000 7fd2c7c54700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c0100710 0x7fd2c0102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.101+0000 7fd2c7c54700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c01010d0 0x7fd2c0102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.101+0000 7fd2c7c54700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c0106680 0x7fd2c0102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.101+0000 7fd2c7c54700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c01a60a0 con 0x7fd2c01010d0 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.101+0000 7fd2c7c54700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c01a6210 con 0x7fd2c0106680 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.101+0000 7fd2c7c54700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c01a63d0 con 0x7fd2c0100710 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2c51ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c01010d0 0x7fd2c0102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2c51ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c01010d0 0x7fd2c0102790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58492/0 (socket says 172.21.15.45:58492) 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2c51ef700 1 -- 172.21.15.45:0/3139307313 learned_addr learned my addr 172.21.15.45:0/3139307313 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2c59f0700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c0100710 0x7fd2c0102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2c61f1700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c0106680 0x7fd2c0102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2c51ef700 1 -- 172.21.15.45:0/3139307313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c0100710 msgr2=0x7fd2c0102250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2c51ef700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c0100710 0x7fd2c0102250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2c51ef700 1 -- 172.21.15.45:0/3139307313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c0106680 msgr2=0x7fd2c0102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2c51ef700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c0106680 0x7fd2c0102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2c51ef700 1 -- 172.21.15.45:0/3139307313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00095f0 con 0x7fd2c01010d0 2024-01-31T05:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2c51ef700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c01010d0 0x7fd2c0102790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00098e0 tx=0x7fd2b000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2b6ffd700 1 -- 172.21.15.45:0/3139307313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b001d070 con 0x7fd2c01010d0 2024-01-31T05:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2b6ffd700 1 -- 172.21.15.45:0/3139307313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b0022470 con 0x7fd2c01010d0 2024-01-31T05:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2c7c54700 1 -- 172.21.15.45:0/3139307313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c01a6650 con 0x7fd2c01010d0 2024-01-31T05:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.102+0000 7fd2b6ffd700 1 -- 172.21.15.45:0/3139307313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0017650 con 0x7fd2c01010d0 2024-01-31T05:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.103+0000 7fd2c7c54700 1 -- 172.21.15.45:0/3139307313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c01a6a60 con 0x7fd2c01010d0 2024-01-31T05:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.103+0000 7fd2c7c54700 1 -- 172.21.15.45:0/3139307313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd2c0110940 con 0x7fd2c01010d0 2024-01-31T05:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.105+0000 7fd2b6ffd700 1 -- 172.21.15.45:0/3139307313 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fd2b00225e0 con 0x7fd2c01010d0 2024-01-31T05:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.105+0000 7fd2b6ffd700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd2ac0a1400 0x7fd2ac0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.105+0000 7fd2b6ffd700 1 -- 172.21.15.45:0/3139307313 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd2b00ce1a0 con 0x7fd2c01010d0 2024-01-31T05:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.105+0000 7fd2c59f0700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd2ac0a1400 0x7fd2ac0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.106+0000 7fd2b6ffd700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514] conn(0x7fd2ac0aabe0 0x7fd2ac0ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.106+0000 7fd2b6ffd700 1 -- 172.21.15.45:0/3139307313 --> [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd2ac0ad6b0 con 0x7fd2ac0aabe0 2024-01-31T05:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.106+0000 7fd2b6ffd700 1 -- 172.21.15.45:0/3139307313 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2b00ce640 con 0x7fd2c01010d0 2024-01-31T05:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.106+0000 7fd2c59f0700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd2ac0a1400 0x7fd2ac0a38c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc005950 tx=0x7fd2bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.106+0000 7fd2c61f1700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514] conn(0x7fd2ac0aabe0 0x7fd2ac0ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.106+0000 7fd2c61f1700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514] conn(0x7fd2ac0aabe0 0x7fd2ac0ad000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.107+0000 7fd2b6ffd700 1 -- 172.21.15.45:0/3139307313 <== osd.1 v2:172.21.15.156:6800/2623092514 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd2ac0ad6b0 con 0x7fd2ac0aabe0 2024-01-31T05:21:14.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.125+0000 7fd2c7c54700 1 -- 172.21.15.45:0/3139307313 --> [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd2c0069f70 con 0x7fd2ac0aabe0 2024-01-31T05:21:14.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.125+0000 7fd2b6ffd700 1 -- 172.21.15.45:0/3139307313 <== osd.1 v2:172.21.15.156:6800/2623092514 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd2c0069f70 con 0x7fd2ac0aabe0 2024-01-31T05:21:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.126+0000 7fd2c7c54700 1 -- 172.21.15.45:0/3139307313 >> [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514] conn(0x7fd2ac0aabe0 msgr2=0x7fd2ac0ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.126+0000 7fd2c7c54700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514] conn(0x7fd2ac0aabe0 0x7fd2ac0ad000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.126+0000 7fd2c7c54700 1 -- 172.21.15.45:0/3139307313 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd2ac0a1400 msgr2=0x7fd2ac0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.126+0000 7fd2c7c54700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd2ac0a1400 0x7fd2ac0a38c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc005950 tx=0x7fd2bc00b410 comp rx=0 tx=0).stop 2024-01-31T05:21:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.126+0000 7fd2c7c54700 1 -- 172.21.15.45:0/3139307313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c01010d0 msgr2=0x7fd2c0102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.126+0000 7fd2c7c54700 1 --2- 172.21.15.45:0/3139307313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2c01010d0 0x7fd2c0102790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00098e0 tx=0x7fd2b000f670 comp rx=0 tx=0).stop 2024-01-31T05:21:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.126+0000 7fd2c61f1700 1 -- 172.21.15.45:0/3139307313 reap_dead start 2024-01-31T05:21:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.126+0000 7fd2c7c54700 1 -- 172.21.15.45:0/3139307313 shutdown_connections 2024-01-31T05:21:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.126+0000 7fd2c7c54700 1 -- 172.21.15.45:0/3139307313 >> 172.21.15.45:0/3139307313 conn(0x7fd2c0076730 msgr2=0x7fd2c01045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.126+0000 7fd2c7c54700 1 -- 172.21.15.45:0/3139307313 shutdown_connections 2024-01-31T05:21:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:14.126+0000 7fd2c7c54700 1 -- 172.21.15.45:0/3139307313 wait complete. 2024-01-31T05:21:15.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:15 smithi045 ceph-mon[31585]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-01-31T05:21:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:15 smithi156 ceph-mon[35720]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-01-31T05:21:16.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:15 smithi186 ceph-mon[35659]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-01-31T05:21:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.250+0000 7f75789d3700 1 -- 172.21.15.45:0/2139328296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7574107800 msgr2=0x7f7574107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.250+0000 7f75789d3700 1 --2- 172.21.15.45:0/2139328296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7574107800 0x7f7574107be0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7564009990 tx=0x7f7564009ca0 comp rx=0 tx=0).stop 2024-01-31T05:21:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.251+0000 7f75789d3700 1 -- 172.21.15.45:0/2139328296 shutdown_connections 2024-01-31T05:21:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.251+0000 7f75789d3700 1 --2- 172.21.15.45:0/2139328296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f757406a900 0x7f757410e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.251+0000 7f75789d3700 1 --2- 172.21.15.45:0/2139328296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7574107800 0x7f7574107be0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.251+0000 7f75789d3700 1 --2- 172.21.15.45:0/2139328296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7574069f40 0x7f757406a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.251+0000 7f75789d3700 1 -- 172.21.15.45:0/2139328296 >> 172.21.15.45:0/2139328296 conn(0x7f75740777e0 msgr2=0x7f7574077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.251+0000 7f75789d3700 1 -- 172.21.15.45:0/2139328296 shutdown_connections 2024-01-31T05:21:16.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.251+0000 7f75789d3700 1 -- 172.21.15.45:0/2139328296 wait complete. 2024-01-31T05:21:16.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.251+0000 7f75789d3700 1 Processor -- start 2024-01-31T05:21:16.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.251+0000 7f75789d3700 1 -- start start 2024-01-31T05:21:16.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f75789d3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7574069f40 0x7f7574199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:16.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f75789d3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f757406a900 0x7f757419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:16.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f75789d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7574107800 0x7f757419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f75789d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757419ed40 con 0x7f7574107800 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f75789d3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757419eeb0 con 0x7f7574069f40 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f75789d3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757419f070 con 0x7f757406a900 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f7572d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7574107800 0x7f757419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f7572d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7574107800 0x7f757419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58496/0 (socket says 172.21.15.45:58496) 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f7572d9d700 1 -- 172.21.15.45:0/1817629053 learned_addr learned my addr 172.21.15.45:0/1817629053 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f757259c700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7574069f40 0x7f7574199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f7571d9b700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f757406a900 0x7f757419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f7572d9d700 1 -- 172.21.15.45:0/1817629053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f757406a900 msgr2=0x7f757419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f7572d9d700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f757406a900 0x7f757419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f7572d9d700 1 -- 172.21.15.45:0/1817629053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7574069f40 msgr2=0x7f7574199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f7572d9d700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7574069f40 0x7f7574199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f7572d9d700 1 -- 172.21.15.45:0/1817629053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75640095f0 con 0x7f7574107800 2024-01-31T05:21:16.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f7572d9d700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7574107800 0x7f757419e730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f756c007d70 tx=0x7f756c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:16.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f75637fe700 1 -- 172.21.15.45:0/1817629053 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c00ad40 con 0x7f7574107800 2024-01-31T05:21:16.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f75637fe700 1 -- 172.21.15.45:0/1817629053 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f756c0128e0 con 0x7f7574107800 2024-01-31T05:21:16.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.252+0000 7f75789d3700 1 -- 172.21.15.45:0/1817629053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757419f350 con 0x7f7574107800 2024-01-31T05:21:16.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.253+0000 7f75637fe700 1 -- 172.21.15.45:0/1817629053 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0116b0 con 0x7f7574107800 2024-01-31T05:21:16.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.253+0000 7f75789d3700 1 -- 172.21.15.45:0/1817629053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75741a6bb0 con 0x7f7574107800 2024-01-31T05:21:16.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.253+0000 7f75789d3700 1 -- 172.21.15.45:0/1817629053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7554001030 con 0x7f7574107800 2024-01-31T05:21:16.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.254+0000 7f75637fe700 1 -- 172.21.15.45:0/1817629053 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f756c012420 con 0x7f7574107800 2024-01-31T05:21:16.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.255+0000 7f75637fe700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f755c0a1400 0x7f755c0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:16.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.255+0000 7f757259c700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f755c0a1400 0x7f755c0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:16.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.255+0000 7f75637fe700 1 -- 172.21.15.45:0/1817629053 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f756c0cd150 con 0x7f7574107800 2024-01-31T05:21:16.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.255+0000 7f75637fe700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526] conn(0x7f755c0aabe0 0x7f755c0ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:16.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.255+0000 7f75637fe700 1 -- 172.21.15.45:0/1817629053 --> [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f755c0ad6b0 con 0x7f755c0aabe0 2024-01-31T05:21:16.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.255+0000 7f75637fe700 1 -- 172.21.15.45:0/1817629053 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f756c0cd5f0 con 0x7f7574107800 2024-01-31T05:21:16.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.256+0000 7f757259c700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f755c0a1400 0x7f755c0a38c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f75640098e0 tx=0x7f7564005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:16.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.256+0000 7f7571d9b700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526] conn(0x7f755c0aabe0 0x7f755c0ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:16.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.256+0000 7f7571d9b700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526] conn(0x7f755c0aabe0 0x7f755c0ad000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:16.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.257+0000 7f75637fe700 1 -- 172.21.15.45:0/1817629053 <== osd.10 v2:172.21.15.156:6824/2036478526 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f755c0ad6b0 con 0x7f755c0aabe0 2024-01-31T05:21:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.273+0000 7f75789d3700 1 -- 172.21.15.45:0/1817629053 --> [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7554002dc0 con 0x7f755c0aabe0 2024-01-31T05:21:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.274+0000 7f75637fe700 1 -- 172.21.15.45:0/1817629053 <== osd.10 v2:172.21.15.156:6824/2036478526 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7554002dc0 con 0x7f755c0aabe0 2024-01-31T05:21:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.274+0000 7f75789d3700 1 -- 172.21.15.45:0/1817629053 >> [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526] conn(0x7f755c0aabe0 msgr2=0x7f755c0ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.274+0000 7f75789d3700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526] conn(0x7f755c0aabe0 0x7f755c0ad000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.274+0000 7f75789d3700 1 -- 172.21.15.45:0/1817629053 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f755c0a1400 msgr2=0x7f755c0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.274+0000 7f75789d3700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f755c0a1400 0x7f755c0a38c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f75640098e0 tx=0x7f7564005960 comp rx=0 tx=0).stop 2024-01-31T05:21:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.274+0000 7f75789d3700 1 -- 172.21.15.45:0/1817629053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7574107800 msgr2=0x7f757419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.274+0000 7f75789d3700 1 --2- 172.21.15.45:0/1817629053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7574107800 0x7f757419e730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f756c007d70 tx=0x7f756c00a340 comp rx=0 tx=0).stop 2024-01-31T05:21:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.274+0000 7f7572d9d700 1 -- 172.21.15.45:0/1817629053 reap_dead start 2024-01-31T05:21:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.274+0000 7f75789d3700 1 -- 172.21.15.45:0/1817629053 shutdown_connections 2024-01-31T05:21:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.274+0000 7f75789d3700 1 -- 172.21.15.45:0/1817629053 >> 172.21.15.45:0/1817629053 conn(0x7f75740777e0 msgr2=0x7f757410cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.274+0000 7f75789d3700 1 -- 172.21.15.45:0/1817629053 shutdown_connections 2024-01-31T05:21:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:16.274+0000 7f75789d3700 1 -- 172.21.15.45:0/1817629053 wait complete. 2024-01-31T05:21:17.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.189+0000 7f6c1c398700 1 -- 172.21.15.45:0/2055077762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c1406a6e0 msgr2=0x7f6c1410e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:17.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.189+0000 7f6c1c398700 1 --2- 172.21.15.45:0/2055077762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c1406a6e0 0x7f6c1410e640 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00b0d0 tx=0x7f6c0c00b3e0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.189+0000 7f6c1c398700 1 -- 172.21.15.45:0/2055077762 shutdown_connections 2024-01-31T05:21:17.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.189+0000 7f6c1c398700 1 --2- 172.21.15.45:0/2055077762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c1406a6e0 0x7f6c1410e640 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.189+0000 7f6c1c398700 1 --2- 172.21.15.45:0/2055077762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c14107790 0x7f6c14107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.189+0000 7f6c1c398700 1 --2- 172.21.15.45:0/2055077762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c14069d20 0x7f6c1406a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.189+0000 7f6c1c398700 1 -- 172.21.15.45:0/2055077762 >> 172.21.15.45:0/2055077762 conn(0x7f6c14076730 msgr2=0x7f6c14076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.189+0000 7f6c1c398700 1 -- 172.21.15.45:0/2055077762 shutdown_connections 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.189+0000 7f6c1c398700 1 -- 172.21.15.45:0/2055077762 wait complete. 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1c398700 1 Processor -- start 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1c398700 1 -- start start 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1c398700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c14069d20 0x7f6c14197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1c398700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c1406a6e0 0x7f6c14198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1c398700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c14107790 0x7f6c1419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1c398700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1419ccf0 con 0x7f6c14069d20 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1c398700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1419ce60 con 0x7f6c1406a6e0 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1c398700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1419d020 con 0x7f6c14107790 2024-01-31T05:21:17.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1a134700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c14069d20 0x7f6c14197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1a134700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c14069d20 0x7f6c14197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58520/0 (socket says 172.21.15.45:58520) 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1a134700 1 -- 172.21.15.45:0/707857184 learned_addr learned my addr 172.21.15.45:0/707857184 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1a134700 1 -- 172.21.15.45:0/707857184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c14107790 msgr2=0x7f6c1419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1a134700 1 --2- 172.21.15.45:0/707857184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c14107790 0x7f6c1419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1a134700 1 -- 172.21.15.45:0/707857184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c1406a6e0 msgr2=0x7f6c14198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1a134700 1 --2- 172.21.15.45:0/707857184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c1406a6e0 0x7f6c14198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.190+0000 7f6c1a134700 1 -- 172.21.15.45:0/707857184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c009d00 con 0x7f6c14069d20 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.191+0000 7f6c1a134700 1 --2- 172.21.15.45:0/707857184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c14069d20 0x7f6c14197e10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000c920 tx=0x7f6c1000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.191+0000 7f6c0b7fe700 1 -- 172.21.15.45:0/707857184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10007b00 con 0x7f6c14069d20 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.191+0000 7f6c0b7fe700 1 -- 172.21.15.45:0/707857184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c10004cf0 con 0x7f6c14069d20 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.191+0000 7f6c1c398700 1 -- 172.21.15.45:0/707857184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1419d300 con 0x7f6c14069d20 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.191+0000 7f6c0b7fe700 1 -- 172.21.15.45:0/707857184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10005610 con 0x7f6c14069d20 2024-01-31T05:21:17.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.191+0000 7f6c1c398700 1 -- 172.21.15.45:0/707857184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c141a4b60 con 0x7f6c14069d20 2024-01-31T05:21:17.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.192+0000 7f6c1c398700 1 -- 172.21.15.45:0/707857184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6c14111a10 con 0x7f6c14069d20 2024-01-31T05:21:17.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.193+0000 7f6c0b7fe700 1 -- 172.21.15.45:0/707857184 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f6c10004750 con 0x7f6c14069d20 2024-01-31T05:21:17.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.193+0000 7f6c0b7fe700 1 --2- 172.21.15.45:0/707857184 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f6c000a1480 0x7f6c000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:17.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.194+0000 7f6c19933700 1 --2- 172.21.15.45:0/707857184 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f6c000a1480 0x7f6c000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:17.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.194+0000 7f6c0b7fe700 1 -- 172.21.15.45:0/707857184 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6c100ccdf0 con 0x7f6c14069d20 2024-01-31T05:21:17.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.194+0000 7f6c0b7fe700 1 --2- 172.21.15.45:0/707857184 >> [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752] conn(0x7f6c000aac60 0x7f6c000ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:17.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.194+0000 7f6c0b7fe700 1 -- 172.21.15.45:0/707857184 --> [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c000ad730 con 0x7f6c000aac60 2024-01-31T05:21:17.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.194+0000 7f6c0b7fe700 1 -- 172.21.15.45:0/707857184 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c100cd160 con 0x7f6c14069d20 2024-01-31T05:21:17.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.194+0000 7f6c19933700 1 --2- 172.21.15.45:0/707857184 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f6c000a1480 0x7f6c000a3940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c14076df0 tx=0x7f6c04006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:17.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.194+0000 7f6c1a935700 1 --2- 172.21.15.45:0/707857184 >> [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752] conn(0x7f6c000aac60 0x7f6c000ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:17.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.195+0000 7f6c1a935700 1 --2- 172.21.15.45:0/707857184 >> [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752] conn(0x7f6c000aac60 0x7f6c000ad080 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:17.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.196+0000 7f6c0b7fe700 1 -- 172.21.15.45:0/707857184 <== osd.4 v2:172.21.15.156:6808/2291851752 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6c000ad730 con 0x7f6c000aac60 2024-01-31T05:21:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.212+0000 7f6c1c398700 1 -- 172.21.15.45:0/707857184 --> [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6c14061e70 con 0x7f6c000aac60 2024-01-31T05:21:17.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.213+0000 7f6c0b7fe700 1 -- 172.21.15.45:0/707857184 <== osd.4 v2:172.21.15.156:6808/2291851752 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6c14061e70 con 0x7f6c000aac60 2024-01-31T05:21:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.213+0000 7f6c1c398700 1 -- 172.21.15.45:0/707857184 >> [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752] conn(0x7f6c000aac60 msgr2=0x7f6c000ad080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:17.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.213+0000 7f6c1c398700 1 --2- 172.21.15.45:0/707857184 >> [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752] conn(0x7f6c000aac60 0x7f6c000ad080 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.213+0000 7f6c1c398700 1 -- 172.21.15.45:0/707857184 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f6c000a1480 msgr2=0x7f6c000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.213+0000 7f6c1c398700 1 --2- 172.21.15.45:0/707857184 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f6c000a1480 0x7f6c000a3940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c14076df0 tx=0x7f6c04006d20 comp rx=0 tx=0).stop 2024-01-31T05:21:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.213+0000 7f6c1c398700 1 -- 172.21.15.45:0/707857184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c14069d20 msgr2=0x7f6c14197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.213+0000 7f6c1c398700 1 --2- 172.21.15.45:0/707857184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c14069d20 0x7f6c14197e10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000c920 tx=0x7f6c1000cce0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.213+0000 7f6c1a935700 1 -- 172.21.15.45:0/707857184 reap_dead start 2024-01-31T05:21:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.213+0000 7f6c1c398700 1 -- 172.21.15.45:0/707857184 shutdown_connections 2024-01-31T05:21:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.213+0000 7f6c1c398700 1 -- 172.21.15.45:0/707857184 >> 172.21.15.45:0/707857184 conn(0x7f6c14076730 msgr2=0x7f6c140ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.213+0000 7f6c1c398700 1 -- 172.21.15.45:0/707857184 shutdown_connections 2024-01-31T05:21:17.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.213+0000 7f6c1c398700 1 -- 172.21.15.45:0/707857184 wait complete. 2024-01-31T05:21:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.069+0000 7f8b688fb700 1 -- 172.21.15.45:0/3040577366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b641077a0 msgr2=0x7f8b64107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.069+0000 7f8b688fb700 1 --2- 172.21.15.45:0/3040577366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b641077a0 0x7f8b64107b80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00b0d0 tx=0x7f8b5c00b3e0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.069+0000 7f8b688fb700 1 -- 172.21.15.45:0/3040577366 shutdown_connections 2024-01-31T05:21:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.069+0000 7f8b688fb700 1 --2- 172.21.15.45:0/3040577366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b64069e30 0x7f8b6410e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.069+0000 7f8b688fb700 1 --2- 172.21.15.45:0/3040577366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b641077a0 0x7f8b64107b80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.069+0000 7f8b688fb700 1 --2- 172.21.15.45:0/3040577366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b64069490 0x7f8b640698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.069+0000 7f8b688fb700 1 -- 172.21.15.45:0/3040577366 >> 172.21.15.45:0/3040577366 conn(0x7f8b64077610 msgr2=0x7f8b64077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.069+0000 7f8b688fb700 1 -- 172.21.15.45:0/3040577366 shutdown_connections 2024-01-31T05:21:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.069+0000 7f8b688fb700 1 -- 172.21.15.45:0/3040577366 wait complete. 2024-01-31T05:21:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b688fb700 1 Processor -- start 2024-01-31T05:21:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b688fb700 1 -- start start 2024-01-31T05:21:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b688fb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b64069490 0x7f8b641007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b688fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b64069e30 0x7f8b64100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b688fb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b641077a0 0x7f8b64101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b688fb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b641a6160 con 0x7f8b64069e30 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b688fb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b641a62d0 con 0x7f8b641077a0 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b688fb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b641a6490 con 0x7f8b64069490 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b61d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b64069e30 0x7f8b64100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b61d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b64069e30 0x7f8b64100cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58508/0 (socket says 172.21.15.45:58508) 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b61d9b700 1 -- 172.21.15.45:0/1539490146 learned_addr learned my addr 172.21.15.45:0/1539490146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b62d9d700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b641077a0 0x7f8b64101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b6259c700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b64069490 0x7f8b641007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b61d9b700 1 -- 172.21.15.45:0/1539490146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b64069490 msgr2=0x7f8b641007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b61d9b700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b64069490 0x7f8b641007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b61d9b700 1 -- 172.21.15.45:0/1539490146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b641077a0 msgr2=0x7f8b64101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b61d9b700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b641077a0 0x7f8b64101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.070+0000 7f8b61d9b700 1 -- 172.21.15.45:0/1539490146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c009d00 con 0x7f8b64069e30 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.071+0000 7f8b61d9b700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b64069e30 0x7f8b64100cf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00c920 tx=0x7f8b4c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.071+0000 7f8b5b7fe700 1 -- 172.21.15.45:0/1539490146 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c007b00 con 0x7f8b64069e30 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.071+0000 7f8b5b7fe700 1 -- 172.21.15.45:0/1539490146 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b4c00ce70 con 0x7f8b64069e30 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.071+0000 7f8b688fb700 1 -- 172.21.15.45:0/1539490146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b641a6770 con 0x7f8b64069e30 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.071+0000 7f8b5b7fe700 1 -- 172.21.15.45:0/1539490146 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c0054e0 con 0x7f8b64069e30 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.071+0000 7f8b688fb700 1 -- 172.21.15.45:0/1539490146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b641a6c70 con 0x7f8b64069e30 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.071+0000 7f8b688fb700 1 -- 172.21.15.45:0/1539490146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b64111a00 con 0x7f8b64069e30 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.073+0000 7f8b5b7fe700 1 -- 172.21.15.45:0/1539490146 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f8b4c004750 con 0x7f8b64069e30 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.073+0000 7f8b5b7fe700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8b500a1400 0x7f8b500a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.074+0000 7f8b6259c700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8b500a1400 0x7f8b500a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.074+0000 7f8b5b7fe700 1 -- 172.21.15.45:0/1539490146 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8b4c0ccaa0 con 0x7f8b64069e30 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.074+0000 7f8b5b7fe700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784] conn(0x7f8b500aabe0 0x7f8b500ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.074+0000 7f8b5b7fe700 1 -- 172.21.15.45:0/1539490146 --> [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b500ad6b0 con 0x7f8b500aabe0 2024-01-31T05:21:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.074+0000 7f8b5b7fe700 1 -- 172.21.15.45:0/1539490146 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b4c0ccf40 con 0x7f8b64069e30 2024-01-31T05:21:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.074+0000 7f8b62d9d700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784] conn(0x7f8b500aabe0 0x7f8b500ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.074+0000 7f8b6259c700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8b500a1400 0x7f8b500a38c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009fd0 tx=0x7f8b5c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.074+0000 7f8b62d9d700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784] conn(0x7f8b500aabe0 0x7f8b500ad000 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.075+0000 7f8b5b7fe700 1 -- 172.21.15.45:0/1539490146 <== osd.0 v2:172.21.15.45:6802/4026202784 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8b500ad6b0 con 0x7f8b500aabe0 2024-01-31T05:21:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b688fb700 1 -- 172.21.15.45:0/1539490146 --> [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8b64061ef0 con 0x7f8b500aabe0 2024-01-31T05:21:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b5b7fe700 1 -- 172.21.15.45:0/1539490146 <== osd.0 v2:172.21.15.45:6802/4026202784 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8b64061ef0 con 0x7f8b500aabe0 2024-01-31T05:21:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b688fb700 1 -- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784] conn(0x7f8b500aabe0 msgr2=0x7f8b500ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b688fb700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784] conn(0x7f8b500aabe0 0x7f8b500ad000 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b688fb700 1 -- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8b500a1400 msgr2=0x7f8b500a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b688fb700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8b500a1400 0x7f8b500a38c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009fd0 tx=0x7f8b5c00b040 comp rx=0 tx=0).stop 2024-01-31T05:21:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b688fb700 1 -- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b64069e30 msgr2=0x7f8b64100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b688fb700 1 --2- 172.21.15.45:0/1539490146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b64069e30 0x7f8b64100cf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00c920 tx=0x7f8b4c00cc30 comp rx=0 tx=0).stop 2024-01-31T05:21:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b62d9d700 1 -- 172.21.15.45:0/1539490146 reap_dead start 2024-01-31T05:21:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b688fb700 1 -- 172.21.15.45:0/1539490146 shutdown_connections 2024-01-31T05:21:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b688fb700 1 -- 172.21.15.45:0/1539490146 >> 172.21.15.45:0/1539490146 conn(0x7f8b64077610 msgr2=0x7f8b640fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b688fb700 1 -- 172.21.15.45:0/1539490146 shutdown_connections 2024-01-31T05:21:17.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:17.092+0000 7f8b688fb700 1 -- 172.21.15.45:0/1539490146 wait complete. 2024-01-31T05:21:17.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:17 smithi045 ceph-mon[31585]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-01-31T05:21:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:17 smithi156 ceph-mon[35720]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-01-31T05:21:18.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:17 smithi186 ceph-mon[35659]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-01-31T05:21:18.553 INFO:teuthology.orchestra.run.smithi045.stdout:167503724594 2024-01-31T05:21:18.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-01-31T05:21:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.727+0000 7f3193dac700 1 -- 172.21.15.45:0/282359931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c101830 msgr2=0x7f318c101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.727+0000 7f3193dac700 1 --2- 172.21.15.45:0/282359931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c101830 0x7f318c101cb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f317c0099c0 tx=0x7f317c009cd0 comp rx=0 tx=0).stop 2024-01-31T05:21:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.727+0000 7f3193dac700 1 -- 172.21.15.45:0/282359931 shutdown_connections 2024-01-31T05:21:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.727+0000 7f3193dac700 1 --2- 172.21.15.45:0/282359931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f318c1021f0 0x7f318c10e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.727+0000 7f3193dac700 1 --2- 172.21.15.45:0/282359931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f318c106650 0x7f318c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.727+0000 7f3193dac700 1 --2- 172.21.15.45:0/282359931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c101830 0x7f318c101cb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.727+0000 7f3193dac700 1 -- 172.21.15.45:0/282359931 >> 172.21.15.45:0/282359931 conn(0x7f318c078910 msgr2=0x7f318c078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.727+0000 7f3193dac700 1 -- 172.21.15.45:0/282359931 shutdown_connections 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.727+0000 7f3193dac700 1 -- 172.21.15.45:0/282359931 wait complete. 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3193dac700 1 Processor -- start 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3193dac700 1 -- start start 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3193dac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f318c101830 0x7f318c19d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3193dac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c1021f0 0x7f318c19ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3193dac700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f318c106650 0x7f318c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3193dac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c1980b0 con 0x7f318c1021f0 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3193dac700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c1981f0 con 0x7f318c106650 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3193dac700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c1983b0 con 0x7f318c101830 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3191347700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c1021f0 0x7f318c19ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3191347700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c1021f0 0x7f318c19ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55862/0 (socket says 172.21.15.45:55862) 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3191347700 1 -- 172.21.15.45:0/1569648495 learned_addr learned my addr 172.21.15.45:0/1569648495 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3191b48700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f318c101830 0x7f318c19d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3192349700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f318c106650 0x7f318c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3191347700 1 -- 172.21.15.45:0/1569648495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f318c101830 msgr2=0x7f318c19d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3191347700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f318c101830 0x7f318c19d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3191347700 1 -- 172.21.15.45:0/1569648495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f318c106650 msgr2=0x7f318c197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3191347700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f318c106650 0x7f318c197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.728+0000 7f3191347700 1 -- 172.21.15.45:0/1569648495 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c0095f0 con 0x7f318c1021f0 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.729+0000 7f3191347700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c1021f0 0x7f318c19ddd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f317c00b580 tx=0x7f317c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.729+0000 7f3182ffd700 1 -- 172.21.15.45:0/1569648495 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c01d070 con 0x7f318c1021f0 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.729+0000 7f3193dac700 1 -- 172.21.15.45:0/1569648495 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c198630 con 0x7f318c1021f0 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.729+0000 7f3182ffd700 1 -- 172.21.15.45:0/1569648495 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f317c00fb30 con 0x7f318c1021f0 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.729+0000 7f3182ffd700 1 -- 172.21.15.45:0/1569648495 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c022c60 con 0x7f318c1021f0 2024-01-31T05:21:18.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.729+0000 7f3193dac700 1 -- 172.21.15.45:0/1569648495 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c1a4930 con 0x7f318c1021f0 2024-01-31T05:21:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.730+0000 7f3193dac700 1 -- 172.21.15.45:0/1569648495 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f318c102ec0 con 0x7f318c1021f0 2024-01-31T05:21:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.731+0000 7f3182ffd700 1 -- 172.21.15.45:0/1569648495 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f317c00fca0 con 0x7f318c1021f0 2024-01-31T05:21:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.732+0000 7f3182ffd700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f31780a1480 0x7f31780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.732+0000 7f3182ffd700 1 -- 172.21.15.45:0/1569648495 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f317c0ce5d0 con 0x7f318c1021f0 2024-01-31T05:21:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.732+0000 7f3191b48700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f31780a1480 0x7f31780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.732+0000 7f3182ffd700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759] conn(0x7f31780aac60 0x7f31780ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.732+0000 7f3182ffd700 1 -- 172.21.15.45:0/1569648495 --> [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31780ad730 con 0x7f31780aac60 2024-01-31T05:21:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.732+0000 7f3182ffd700 1 -- 172.21.15.45:0/1569648495 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f317c0cea70 con 0x7f318c1021f0 2024-01-31T05:21:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.733+0000 7f3191b48700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f31780a1480 0x7f31780a3940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f318c198b90 tx=0x7f318800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.733+0000 7f3192349700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759] conn(0x7f31780aac60 0x7f31780ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.733+0000 7f3192349700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759] conn(0x7f31780aac60 0x7f31780ad080 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.734+0000 7f3182ffd700 1 -- 172.21.15.45:0/1569648495 <== osd.8 v2:172.21.15.186:6816/2054139759 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f31780ad730 con 0x7f31780aac60 2024-01-31T05:21:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.751+0000 7f3193dac700 1 -- 172.21.15.45:0/1569648495 --> [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f318c1119e0 con 0x7f31780aac60 2024-01-31T05:21:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.751+0000 7f3182ffd700 1 -- 172.21.15.45:0/1569648495 <== osd.8 v2:172.21.15.186:6816/2054139759 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f318c1119e0 con 0x7f31780aac60 2024-01-31T05:21:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.751+0000 7f3193dac700 1 -- 172.21.15.45:0/1569648495 >> [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759] conn(0x7f31780aac60 msgr2=0x7f31780ad080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.751+0000 7f3193dac700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759] conn(0x7f31780aac60 0x7f31780ad080 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.752+0000 7f3193dac700 1 -- 172.21.15.45:0/1569648495 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f31780a1480 msgr2=0x7f31780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.752+0000 7f3193dac700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f31780a1480 0x7f31780a3940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f318c198b90 tx=0x7f318800b410 comp rx=0 tx=0).stop 2024-01-31T05:21:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.752+0000 7f3193dac700 1 -- 172.21.15.45:0/1569648495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c1021f0 msgr2=0x7f318c19ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.752+0000 7f3193dac700 1 --2- 172.21.15.45:0/1569648495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c1021f0 0x7f318c19ddd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f317c00b580 tx=0x7f317c004730 comp rx=0 tx=0).stop 2024-01-31T05:21:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.752+0000 7f3192349700 1 -- 172.21.15.45:0/1569648495 reap_dead start 2024-01-31T05:21:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.752+0000 7f3193dac700 1 -- 172.21.15.45:0/1569648495 shutdown_connections 2024-01-31T05:21:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.752+0000 7f3193dac700 1 -- 172.21.15.45:0/1569648495 >> 172.21.15.45:0/1569648495 conn(0x7f318c078910 msgr2=0x7f318c0ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.752+0000 7f3193dac700 1 -- 172.21.15.45:0/1569648495 shutdown_connections 2024-01-31T05:21:18.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:18.752+0000 7f3193dac700 1 -- 172.21.15.45:0/1569648495 wait complete. 2024-01-31T05:21:19.479 INFO:teuthology.orchestra.run.smithi045.stdout:55834574917 2024-01-31T05:21:19.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-01-31T05:21:19.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:19 smithi045 ceph-mon[31585]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-01-31T05:21:19.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:19 smithi156 ceph-mon[35720]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-01-31T05:21:20.060 INFO:teuthology.orchestra.run.smithi045.stdout:146028888119 2024-01-31T05:21:20.060 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-01-31T05:21:20.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:19 smithi186 ceph-mon[35659]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-01-31T05:21:20.561 INFO:teuthology.orchestra.run.smithi045.stdout:313532612633 2024-01-31T05:21:20.561 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-01-31T05:21:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.238+0000 7f31fe7db700 1 -- 172.21.15.45:0/3098093876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f8101080 msgr2=0x7f31f810d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.238+0000 7f31fe7db700 1 --2- 172.21.15.45:0/3098093876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f8101080 0x7f31f810d560 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f31f000b0a0 tx=0x7f31f000b3b0 comp rx=0 tx=0).stop 2024-01-31T05:21:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.238+0000 7f31fe7db700 1 -- 172.21.15.45:0/3098093876 shutdown_connections 2024-01-31T05:21:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.238+0000 7f31fe7db700 1 --2- 172.21.15.45:0/3098093876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f8101080 0x7f31f810d560 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.238+0000 7f31fe7db700 1 --2- 172.21.15.45:0/3098093876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31f8106690 0x7f31f8106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.238+0000 7f31fe7db700 1 --2- 172.21.15.45:0/3098093876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31f81006e0 0x7f31f8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.238+0000 7f31fe7db700 1 -- 172.21.15.45:0/3098093876 >> 172.21.15.45:0/3098093876 conn(0x7f31f8077610 msgr2=0x7f31f8077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.238+0000 7f31fe7db700 1 -- 172.21.15.45:0/3098093876 shutdown_connections 2024-01-31T05:21:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.238+0000 7f31fe7db700 1 -- 172.21.15.45:0/3098093876 wait complete. 2024-01-31T05:21:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31fe7db700 1 Processor -- start 2024-01-31T05:21:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31fe7db700 1 -- start start 2024-01-31T05:21:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31fe7db700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31f81006e0 0x7f31f8199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31fe7db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f8101080 0x7f31f819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31fe7db700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31f8106690 0x7f31f819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31fe7db700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f819ee60 con 0x7f31f8101080 2024-01-31T05:21:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31fe7db700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f819efd0 con 0x7f31f81006e0 2024-01-31T05:21:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31fe7db700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f819f190 con 0x7f31f8106690 2024-01-31T05:21:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31f77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f8101080 0x7f31f819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31f77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f8101080 0x7f31f819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55884/0 (socket says 172.21.15.45:55884) 2024-01-31T05:21:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31f77fe700 1 -- 172.21.15.45:0/688476264 learned_addr learned my addr 172.21.15.45:0/688476264 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31f77fe700 1 -- 172.21.15.45:0/688476264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31f8106690 msgr2=0x7f31f819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31f77fe700 1 --2- 172.21.15.45:0/688476264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31f8106690 0x7f31f819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31f77fe700 1 -- 172.21.15.45:0/688476264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31f81006e0 msgr2=0x7f31f8199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31f77fe700 1 --2- 172.21.15.45:0/688476264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31f81006e0 0x7f31f8199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.239+0000 7f31f77fe700 1 -- 172.21.15.45:0/688476264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f0009d00 con 0x7f31f8101080 2024-01-31T05:21:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.240+0000 7f31f77fe700 1 --2- 172.21.15.45:0/688476264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f8101080 0x7f31f819a4c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00b530 tx=0x7f31ec00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.240+0000 7f31f57fa700 1 -- 172.21.15.45:0/688476264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec011830 con 0x7f31f8101080 2024-01-31T05:21:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.240+0000 7f31f57fa700 1 -- 172.21.15.45:0/688476264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31ec011e50 con 0x7f31f8101080 2024-01-31T05:21:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.240+0000 7f31fe7db700 1 -- 172.21.15.45:0/688476264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f819f470 con 0x7f31f8101080 2024-01-31T05:21:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.240+0000 7f31f57fa700 1 -- 172.21.15.45:0/688476264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00f2e0 con 0x7f31f8101080 2024-01-31T05:21:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.240+0000 7f31fe7db700 1 -- 172.21.15.45:0/688476264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f81a6cd0 con 0x7f31f8101080 2024-01-31T05:21:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.241+0000 7f31fe7db700 1 -- 172.21.15.45:0/688476264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31f8101dd0 con 0x7f31f8101080 2024-01-31T05:21:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.242+0000 7f31f57fa700 1 -- 172.21.15.45:0/688476264 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f31ec010580 con 0x7f31f8101080 2024-01-31T05:21:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.242+0000 7f31f57fa700 1 --2- 172.21.15.45:0/688476264 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f31e00a1400 0x7f31e00a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.242+0000 7f31f7fff700 1 --2- 172.21.15.45:0/688476264 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f31e00a1400 0x7f31e00a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.242+0000 7f31f57fa700 1 -- 172.21.15.45:0/688476264 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f31ec0ccb80 con 0x7f31f8101080 2024-01-31T05:21:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.243+0000 7f31f57fa700 1 --2- 172.21.15.45:0/688476264 >> [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910] conn(0x7f31e00aabe0 0x7f31e00ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.243+0000 7f31f57fa700 1 -- 172.21.15.45:0/688476264 --> [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31e00ad6b0 con 0x7f31e00aabe0 2024-01-31T05:21:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.243+0000 7f31f57fa700 1 -- 172.21.15.45:0/688476264 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31ec0cd020 con 0x7f31f8101080 2024-01-31T05:21:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.243+0000 7f31f7fff700 1 --2- 172.21.15.45:0/688476264 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f31e00a1400 0x7f31e00a38c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f31e800a990 tx=0x7f31e8005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.243+0000 7f31fcd78700 1 --2- 172.21.15.45:0/688476264 >> [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910] conn(0x7f31e00aabe0 0x7f31e00ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.243+0000 7f31fcd78700 1 --2- 172.21.15.45:0/688476264 >> [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910] conn(0x7f31e00aabe0 0x7f31e00ad000 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:21.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.244+0000 7f31f57fa700 1 -- 172.21.15.45:0/688476264 <== osd.11 v2:172.21.15.186:6824/3877143910 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f31e00ad6b0 con 0x7f31e00aabe0 2024-01-31T05:21:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.260+0000 7f31fe7db700 1 -- 172.21.15.45:0/688476264 --> [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f31f8110990 con 0x7f31e00aabe0 2024-01-31T05:21:21.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.261+0000 7f31f57fa700 1 -- 172.21.15.45:0/688476264 <== osd.11 v2:172.21.15.186:6824/3877143910 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f31f8110990 con 0x7f31e00aabe0 2024-01-31T05:21:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.261+0000 7f31fe7db700 1 -- 172.21.15.45:0/688476264 >> [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910] conn(0x7f31e00aabe0 msgr2=0x7f31e00ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.261+0000 7f31fe7db700 1 --2- 172.21.15.45:0/688476264 >> [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910] conn(0x7f31e00aabe0 0x7f31e00ad000 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.261+0000 7f31fe7db700 1 -- 172.21.15.45:0/688476264 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f31e00a1400 msgr2=0x7f31e00a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.261+0000 7f31fe7db700 1 --2- 172.21.15.45:0/688476264 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f31e00a1400 0x7f31e00a38c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f31e800a990 tx=0x7f31e8005c80 comp rx=0 tx=0).stop 2024-01-31T05:21:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.261+0000 7f31fe7db700 1 -- 172.21.15.45:0/688476264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f8101080 msgr2=0x7f31f819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.261+0000 7f31fe7db700 1 --2- 172.21.15.45:0/688476264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f8101080 0x7f31f819a4c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00b530 tx=0x7f31ec00b8f0 comp rx=0 tx=0).stop 2024-01-31T05:21:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.261+0000 7f31fcd78700 1 -- 172.21.15.45:0/688476264 reap_dead start 2024-01-31T05:21:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.261+0000 7f31fe7db700 1 -- 172.21.15.45:0/688476264 shutdown_connections 2024-01-31T05:21:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.261+0000 7f31fe7db700 1 -- 172.21.15.45:0/688476264 >> 172.21.15.45:0/688476264 conn(0x7f31f8077610 msgr2=0x7f31f8069630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.261+0000 7f31fe7db700 1 -- 172.21.15.45:0/688476264 shutdown_connections 2024-01-31T05:21:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:21.262+0000 7f31fe7db700 1 -- 172.21.15.45:0/688476264 wait complete. 2024-01-31T05:21:21.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:21 smithi156 ceph-mon[35720]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:21:22.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:21 smithi186 ceph-mon[35659]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:21:22.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:21 smithi045 ceph-mon[31585]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:21:22.222 INFO:teuthology.orchestra.run.smithi045.stdout:34359738442 2024-01-31T05:21:22.222 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-01-31T05:21:22.848 INFO:teuthology.orchestra.run.smithi045.stdout:257698037796 2024-01-31T05:21:22.848 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-01-31T05:21:23.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.408+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4026607345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b81010d0 msgr2=0x7fe6b810d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:23.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.408+0000 7fe6bfb67700 1 --2- 172.21.15.45:0/4026607345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b81010d0 0x7fe6b810d550 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000b0a0 tx=0x7fe6b000b3b0 comp rx=0 tx=0).stop 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.408+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4026607345 shutdown_connections 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.408+0000 7fe6bfb67700 1 --2- 172.21.15.45:0/4026607345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b81010d0 0x7fe6b810d550 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.408+0000 7fe6bfb67700 1 --2- 172.21.15.45:0/4026607345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6b8106680 0x7fe6b8106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.408+0000 7fe6bfb67700 1 --2- 172.21.15.45:0/4026607345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6b8100710 0x7fe6b8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.408+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4026607345 >> 172.21.15.45:0/4026607345 conn(0x7fe6b8076730 msgr2=0x7fe6b8076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4026607345 shutdown_connections 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4026607345 wait complete. 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6bfb67700 1 Processor -- start 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6bfb67700 1 -- start start 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6bfb67700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6b8100710 0x7fe6b8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6bfb67700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6b81010d0 0x7fe6b8199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6bfb67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b8106680 0x7fe6b81a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6bfb67700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b8199870 con 0x7fe6b8106680 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6bfb67700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b81a40a0 con 0x7fe6b8100710 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6bfb67700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b81a41e0 con 0x7fe6b81010d0 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6be104700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b8106680 0x7fe6b81a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6be104700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b8106680 0x7fe6b81a3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55920/0 (socket says 172.21.15.45:55920) 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.409+0000 7fe6be104700 1 -- 172.21.15.45:0/4160819329 learned_addr learned my addr 172.21.15.45:0/4160819329 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6bd102700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6b81010d0 0x7fe6b8199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6bd903700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6b8100710 0x7fe6b8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6be104700 1 -- 172.21.15.45:0/4160819329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6b81010d0 msgr2=0x7fe6b8199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6be104700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6b81010d0 0x7fe6b8199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6be104700 1 -- 172.21.15.45:0/4160819329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6b8100710 msgr2=0x7fe6b8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6be104700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6b8100710 0x7fe6b8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6be104700 1 -- 172.21.15.45:0/4160819329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b0009d00 con 0x7fe6b8106680 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6be104700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b8106680 0x7fe6b81a3b60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0007930 tx=0x7fe6b0007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6aeffd700 1 -- 172.21.15.45:0/4160819329 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b000e040 con 0x7fe6b8106680 2024-01-31T05:21:23.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6aeffd700 1 -- 172.21.15.45:0/4160819329 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6b0004c50 con 0x7fe6b8106680 2024-01-31T05:21:23.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4160819329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b81a43e0 con 0x7fe6b8106680 2024-01-31T05:21:23.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6aeffd700 1 -- 172.21.15.45:0/4160819329 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00052a0 con 0x7fe6b8106680 2024-01-31T05:21:23.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.410+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4160819329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b81a4850 con 0x7fe6b8106680 2024-01-31T05:21:23.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.411+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4160819329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe6b81108f0 con 0x7fe6b8106680 2024-01-31T05:21:23.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.412+0000 7fe6aeffd700 1 -- 172.21.15.45:0/4160819329 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fe6b001d470 con 0x7fe6b8106680 2024-01-31T05:21:23.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.413+0000 7fe6aeffd700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6a40a1400 0x7fe6a40a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:23.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.413+0000 7fe6aeffd700 1 -- 172.21.15.45:0/4160819329 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe6b00ce8d0 con 0x7fe6b8106680 2024-01-31T05:21:23.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.413+0000 7fe6bd903700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6a40a1400 0x7fe6a40a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:23.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.413+0000 7fe6aeffd700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879] conn(0x7fe6a40aabe0 0x7fe6a40ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:23.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.413+0000 7fe6aeffd700 1 -- 172.21.15.45:0/4160819329 --> [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe6a40ad6b0 con 0x7fe6a40aabe0 2024-01-31T05:21:23.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.413+0000 7fe6aeffd700 1 -- 172.21.15.45:0/4160819329 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6b00cec40 con 0x7fe6b8106680 2024-01-31T05:21:23.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.414+0000 7fe6bd102700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879] conn(0x7fe6a40aabe0 0x7fe6a40ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:23.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.414+0000 7fe6bd903700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6a40a1400 0x7fe6a40a38c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4005fd0 tx=0x7fe6b400af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:23.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.414+0000 7fe6bd102700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879] conn(0x7fe6a40aabe0 0x7fe6a40ad000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:23.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.415+0000 7fe6aeffd700 1 -- 172.21.15.45:0/4160819329 <== osd.7 v2:172.21.15.156:6816/2686361879 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe6a40ad6b0 con 0x7fe6a40aabe0 2024-01-31T05:21:23.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.431+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4160819329 --> [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe6b8061ef0 con 0x7fe6a40aabe0 2024-01-31T05:21:23.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.432+0000 7fe6aeffd700 1 -- 172.21.15.45:0/4160819329 <== osd.7 v2:172.21.15.156:6816/2686361879 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe6b8061ef0 con 0x7fe6a40aabe0 2024-01-31T05:21:23.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.432+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4160819329 >> [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879] conn(0x7fe6a40aabe0 msgr2=0x7fe6a40ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:23.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.432+0000 7fe6bfb67700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879] conn(0x7fe6a40aabe0 0x7fe6a40ad000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:23.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.432+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4160819329 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6a40a1400 msgr2=0x7fe6a40a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:23.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.432+0000 7fe6bfb67700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe6a40a1400 0x7fe6a40a38c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4005fd0 tx=0x7fe6b400af20 comp rx=0 tx=0).stop 2024-01-31T05:21:23.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.432+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4160819329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b8106680 msgr2=0x7fe6b81a3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:23.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.432+0000 7fe6bfb67700 1 --2- 172.21.15.45:0/4160819329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b8106680 0x7fe6b81a3b60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0007930 tx=0x7fe6b0007a10 comp rx=0 tx=0).stop 2024-01-31T05:21:23.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.432+0000 7fe6be104700 1 -- 172.21.15.45:0/4160819329 reap_dead start 2024-01-31T05:21:23.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.432+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4160819329 shutdown_connections 2024-01-31T05:21:23.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.432+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4160819329 >> 172.21.15.45:0/4160819329 conn(0x7fe6b8076730 msgr2=0x7fe6b80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:23.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.433+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4160819329 shutdown_connections 2024-01-31T05:21:23.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.433+0000 7fe6bfb67700 1 -- 172.21.15.45:0/4160819329 wait complete. 2024-01-31T05:21:23.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:23 smithi045 ceph-mon[31585]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:21:23.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:23 smithi156 ceph-mon[35720]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:21:23.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:23 smithi186 ceph-mon[35659]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:21:24.168 INFO:teuthology.orchestra.run.smithi045.stdout:347892350998 2024-01-31T05:21:24.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-01-31T05:21:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.320+0000 7fc22d593700 1 -- 172.21.15.45:0/3352776383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc22806a6e0 msgr2=0x7fc22810e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:24.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.320+0000 7fc22d593700 1 --2- 172.21.15.45:0/3352776383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc22806a6e0 0x7fc22810e640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc22000b0d0 tx=0x7fc22000b3e0 comp rx=0 tx=0).stop 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.320+0000 7fc22d593700 1 -- 172.21.15.45:0/3352776383 shutdown_connections 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.320+0000 7fc22d593700 1 --2- 172.21.15.45:0/3352776383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc22806a6e0 0x7fc22810e640 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.320+0000 7fc22d593700 1 --2- 172.21.15.45:0/3352776383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc228107790 0x7fc228107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.320+0000 7fc22d593700 1 --2- 172.21.15.45:0/3352776383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc228069d20 0x7fc22806a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.320+0000 7fc22d593700 1 -- 172.21.15.45:0/3352776383 >> 172.21.15.45:0/3352776383 conn(0x7fc228076730 msgr2=0x7fc228076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.320+0000 7fc22d593700 1 -- 172.21.15.45:0/3352776383 shutdown_connections 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.320+0000 7fc22d593700 1 -- 172.21.15.45:0/3352776383 wait complete. 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc22d593700 1 Processor -- start 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc22d593700 1 -- start start 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc22d593700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc228069d20 0x7fc228199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc22d593700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc22806a6e0 0x7fc22819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc22d593700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc228107790 0x7fc22819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc22d593700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22819ede0 con 0x7fc228069d20 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc22d593700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22819ef50 con 0x7fc228107790 2024-01-31T05:21:24.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc22d593700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22819f110 con 0x7fc22806a6e0 2024-01-31T05:21:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc226ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc228069d20 0x7fc228199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc226ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc228069d20 0x7fc228199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55904/0 (socket says 172.21.15.45:55904) 2024-01-31T05:21:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc226ffd700 1 -- 172.21.15.45:0/3989589018 learned_addr learned my addr 172.21.15.45:0/3989589018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc2267fc700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc22806a6e0 0x7fc22819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc226ffd700 1 -- 172.21.15.45:0/3989589018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc22806a6e0 msgr2=0x7fc22819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc226ffd700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc22806a6e0 0x7fc22819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc226ffd700 1 -- 172.21.15.45:0/3989589018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc228107790 msgr2=0x7fc22819e7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-31T05:21:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc226ffd700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc228107790 0x7fc22819e7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.321+0000 7fc226ffd700 1 -- 172.21.15.45:0/3989589018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc220009d00 con 0x7fc228069d20 2024-01-31T05:21:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.322+0000 7fc226ffd700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc228069d20 0x7fc228199fa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00b550 tx=0x7fc21c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.322+0000 7fc20ffff700 1 -- 172.21.15.45:0/3989589018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c011830 con 0x7fc228069d20 2024-01-31T05:21:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.322+0000 7fc20ffff700 1 -- 172.21.15.45:0/3989589018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc21c011e50 con 0x7fc228069d20 2024-01-31T05:21:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.322+0000 7fc20ffff700 1 -- 172.21.15.45:0/3989589018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00f2e0 con 0x7fc228069d20 2024-01-31T05:21:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.322+0000 7fc22d593700 1 -- 172.21.15.45:0/3989589018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22819f3f0 con 0x7fc228069d20 2024-01-31T05:21:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.322+0000 7fc22d593700 1 -- 172.21.15.45:0/3989589018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2281a6c50 con 0x7fc228069d20 2024-01-31T05:21:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.323+0000 7fc22d593700 1 -- 172.21.15.45:0/3989589018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc228102f10 con 0x7fc228069d20 2024-01-31T05:21:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.324+0000 7fc20ffff700 1 -- 172.21.15.45:0/3989589018 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fc21c010580 con 0x7fc228069d20 2024-01-31T05:21:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.324+0000 7fc20ffff700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc2100a1400 0x7fc2100a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.325+0000 7fc2267fc700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc2100a1400 0x7fc2100a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.325+0000 7fc20ffff700 1 -- 172.21.15.45:0/3989589018 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc21c0ccb40 con 0x7fc228069d20 2024-01-31T05:21:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.325+0000 7fc20ffff700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931] conn(0x7fc2100aabe0 0x7fc2100ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.325+0000 7fc20ffff700 1 -- 172.21.15.45:0/3989589018 --> [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc2100ad6b0 con 0x7fc2100aabe0 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.325+0000 7fc20ffff700 1 -- 172.21.15.45:0/3989589018 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc21c0ccfe0 con 0x7fc228069d20 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.325+0000 7fc2277fe700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931] conn(0x7fc2100aabe0 0x7fc2100ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.325+0000 7fc2267fc700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc2100a1400 0x7fc2100a38c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc218006fd0 tx=0x7fc218008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.325+0000 7fc2277fe700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931] conn(0x7fc2100aabe0 0x7fc2100ad000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.326+0000 7fc20ffff700 1 -- 172.21.15.45:0/3989589018 <== osd.9 v2:172.21.15.45:6826/717478931 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc2100ad6b0 con 0x7fc2100aabe0 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.343+0000 7fc22d593700 1 -- 172.21.15.45:0/3989589018 --> [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc228111b10 con 0x7fc2100aabe0 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.343+0000 7fc20ffff700 1 -- 172.21.15.45:0/3989589018 <== osd.9 v2:172.21.15.45:6826/717478931 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc228111b10 con 0x7fc2100aabe0 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.344+0000 7fc22d593700 1 -- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931] conn(0x7fc2100aabe0 msgr2=0x7fc2100ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.344+0000 7fc22d593700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931] conn(0x7fc2100aabe0 0x7fc2100ad000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.344+0000 7fc22d593700 1 -- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc2100a1400 msgr2=0x7fc2100a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.344+0000 7fc22d593700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc2100a1400 0x7fc2100a38c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc218006fd0 tx=0x7fc218008040 comp rx=0 tx=0).stop 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.344+0000 7fc22d593700 1 -- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc228069d20 msgr2=0x7fc228199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.344+0000 7fc22d593700 1 --2- 172.21.15.45:0/3989589018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc228069d20 0x7fc228199fa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00b550 tx=0x7fc21c00b910 comp rx=0 tx=0).stop 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.344+0000 7fc2277fe700 1 -- 172.21.15.45:0/3989589018 reap_dead start 2024-01-31T05:21:24.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.344+0000 7fc22d593700 1 -- 172.21.15.45:0/3989589018 shutdown_connections 2024-01-31T05:21:24.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.344+0000 7fc22d593700 1 -- 172.21.15.45:0/3989589018 >> 172.21.15.45:0/3989589018 conn(0x7fc228076730 msgr2=0x7fc22810c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:24.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.344+0000 7fc22d593700 1 -- 172.21.15.45:0/3989589018 shutdown_connections 2024-01-31T05:21:24.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:23.344+0000 7fc22d593700 1 -- 172.21.15.45:0/3989589018 wait complete. 2024-01-31T05:21:24.816 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:24.816 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:24.817 INFO:teuthology.orchestra.run.smithi045.stdout:292057776160 2024-01-31T05:21:24.817 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-01-31T05:21:25.343 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:25.381 INFO:teuthology.orchestra.run.smithi045.stdout:223338299434 2024-01-31T05:21:25.382 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-01-31T05:21:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:25.471 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:25.475 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:25.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:25 smithi045 ceph-mon[31585]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:21:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:25 smithi156 ceph-mon[35720]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:21:26.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:25 smithi186 ceph-mon[35659]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:21:27.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:27 smithi045 ceph-mon[31585]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:21:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:27 smithi156 ceph-mon[35720]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:21:28.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:27 smithi186 ceph-mon[35659]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:21:29.851 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:29.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:29 smithi045 ceph-mon[31585]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:21:29.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:29 smithi156 ceph-mon[35720]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:21:30.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:29 smithi186 ceph-mon[35659]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:21:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.423+0000 7f88d877d700 1 -- 172.21.15.45:0/2408409592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88d01054b0 msgr2=0x7f88d0105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.423+0000 7f88d877d700 1 --2- 172.21.15.45:0/2408409592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88d01054b0 0x7f88d0105930 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009990 tx=0x7f88c0009ca0 comp rx=0 tx=0).stop 2024-01-31T05:21:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.423+0000 7f88d877d700 1 -- 172.21.15.45:0/2408409592 shutdown_connections 2024-01-31T05:21:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.423+0000 7f88d877d700 1 --2- 172.21.15.45:0/2408409592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88d0105e70 0x7f88d0110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.423+0000 7f88d877d700 1 --2- 172.21.15.45:0/2408409592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88d0103d60 0x7f88d0104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.423+0000 7f88d877d700 1 --2- 172.21.15.45:0/2408409592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88d01054b0 0x7f88d0105930 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.423+0000 7f88d877d700 1 -- 172.21.15.45:0/2408409592 >> 172.21.15.45:0/2408409592 conn(0x7f88d00ff480 msgr2=0x7f88d01018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.423+0000 7f88d877d700 1 -- 172.21.15.45:0/2408409592 shutdown_connections 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.423+0000 7f88d877d700 1 -- 172.21.15.45:0/2408409592 wait complete. 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d877d700 1 Processor -- start 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d877d700 1 -- start start 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d877d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88d0103d60 0x7f88d0199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d877d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88d01054b0 0x7f88d019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d877d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88d0105e70 0x7f88d019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d877d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d019edf0 con 0x7f88d0105e70 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d877d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d019ef60 con 0x7f88d0103d60 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d877d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d019f120 con 0x7f88d01054b0 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d6d1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88d0105e70 0x7f88d019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d6519700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88d0103d60 0x7f88d0199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d6d1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88d0105e70 0x7f88d019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51242/0 (socket says 172.21.15.45:51242) 2024-01-31T05:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d6d1a700 1 -- 172.21.15.45:0/3629690151 learned_addr learned my addr 172.21.15.45:0/3629690151 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d5d18700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88d01054b0 0x7f88d019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d6d1a700 1 -- 172.21.15.45:0/3629690151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88d01054b0 msgr2=0x7f88d019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d6d1a700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88d01054b0 0x7f88d019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d6d1a700 1 -- 172.21.15.45:0/3629690151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88d0103d60 msgr2=0x7f88d0199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d6d1a700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88d0103d60 0x7f88d0199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.424+0000 7f88d6d1a700 1 -- 172.21.15.45:0/3629690151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c00095f0 con 0x7f88d0105e70 2024-01-31T05:21:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.425+0000 7f88d6d1a700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88d0105e70 0x7f88d019e7e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f88c8007ee0 tx=0x7f88c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.425+0000 7f88c77fe700 1 -- 172.21.15.45:0/3629690151 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8017070 con 0x7f88d0105e70 2024-01-31T05:21:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.425+0000 7f88d877d700 1 -- 172.21.15.45:0/3629690151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d019f400 con 0x7f88d0105e70 2024-01-31T05:21:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.425+0000 7f88c77fe700 1 -- 172.21.15.45:0/3629690151 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88c800f040 con 0x7f88d0105e70 2024-01-31T05:21:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.425+0000 7f88c77fe700 1 -- 172.21.15.45:0/3629690151 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8005740 con 0x7f88d0105e70 2024-01-31T05:21:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.425+0000 7f88d877d700 1 -- 172.21.15.45:0/3629690151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d01a6c60 con 0x7f88d0105e70 2024-01-31T05:21:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.426+0000 7f88d877d700 1 -- 172.21.15.45:0/3629690151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d010c060 con 0x7f88d0105e70 2024-01-31T05:21:31.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.427+0000 7f88c77fe700 1 -- 172.21.15.45:0/3629690151 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f88c800dc60 con 0x7f88d0105e70 2024-01-31T05:21:31.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.427+0000 7f88c77fe700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f88bc0a1400 0x7f88bc0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:31.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.428+0000 7f88d6519700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f88bc0a1400 0x7f88bc0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:31.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.428+0000 7f88c77fe700 1 -- 172.21.15.45:0/3629690151 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f88c80cc370 con 0x7f88d0105e70 2024-01-31T05:21:31.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.428+0000 7f88d6519700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f88bc0a1400 0x7f88bc0a38c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009a10 tx=0x7f88cc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:31.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.432+0000 7f88c77fe700 1 -- 172.21.15.45:0/3629690151 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f88c8095840 con 0x7f88d0105e70 2024-01-31T05:21:31.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.641+0000 7f88d877d700 1 -- 172.21.15.45:0/3629690151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f88d019b270 con 0x7f88d0105e70 2024-01-31T05:21:31.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.642+0000 7f88c77fe700 1 -- 172.21.15.45:0/3629690151 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f88c8098e60 con 0x7f88d0105e70 2024-01-31T05:21:31.641 INFO:teuthology.orchestra.run.smithi045.stdout:257698037798 2024-01-31T05:21:31.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.644+0000 7f88d877d700 1 -- 172.21.15.45:0/3629690151 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f88bc0a1400 msgr2=0x7f88bc0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:31.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.644+0000 7f88d877d700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f88bc0a1400 0x7f88bc0a38c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009a10 tx=0x7f88cc009190 comp rx=0 tx=0).stop 2024-01-31T05:21:31.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.644+0000 7f88d877d700 1 -- 172.21.15.45:0/3629690151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88d0105e70 msgr2=0x7f88d019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:31.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.644+0000 7f88d877d700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88d0105e70 0x7f88d019e7e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f88c8007ee0 tx=0x7f88c800d3b0 comp rx=0 tx=0).stop 2024-01-31T05:21:31.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.644+0000 7f88d877d700 1 -- 172.21.15.45:0/3629690151 shutdown_connections 2024-01-31T05:21:31.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.644+0000 7f88d877d700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f88bc0a1400 0x7f88bc0a38c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:31.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.644+0000 7f88d877d700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88d0105e70 0x7f88d019e7e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.644+0000 7f88d877d700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88d0103d60 0x7f88d0199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.644+0000 7f88d877d700 1 --2- 172.21.15.45:0/3629690151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88d01054b0 0x7f88d019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.644+0000 7f88d877d700 1 -- 172.21.15.45:0/3629690151 >> 172.21.15.45:0/3629690151 conn(0x7f88d00ff480 msgr2=0x7f88d0109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.644+0000 7f88d877d700 1 -- 172.21.15.45:0/3629690151 shutdown_connections 2024-01-31T05:21:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:31.644+0000 7f88d877d700 1 -- 172.21.15.45:0/3629690151 wait complete. 2024-01-31T05:21:31.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:31 smithi045 ceph-mon[31585]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:21:31.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:31 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3629690151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-01-31T05:21:31.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:31 smithi156 ceph-mon[35720]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:21:31.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:31 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3629690151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-01-31T05:21:32.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:31 smithi186 ceph-mon[35659]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:21:32.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:31 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3629690151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-01-31T05:21:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:33.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:33 smithi156 ceph-mon[35720]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:21:33.829 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:33 smithi045 ceph-mon[31585]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:21:33.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:33 smithi186 ceph-mon[35659]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:21:34.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.484+0000 7fb39ac19700 1 -- 172.21.15.45:0/2806296719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb394107790 msgr2=0x7fb394107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:34.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.484+0000 7fb39ac19700 1 --2- 172.21.15.45:0/2806296719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb394107790 0x7fb394107b70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb3880099c0 tx=0x7fb388009cd0 comp rx=0 tx=0).stop 2024-01-31T05:21:34.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.484+0000 7fb39ac19700 1 -- 172.21.15.45:0/2806296719 shutdown_connections 2024-01-31T05:21:34.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.484+0000 7fb39ac19700 1 --2- 172.21.15.45:0/2806296719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb39406a6e0 0x7fb39410e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:34.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.484+0000 7fb39ac19700 1 --2- 172.21.15.45:0/2806296719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb394107790 0x7fb394107b70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:34.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.484+0000 7fb39ac19700 1 --2- 172.21.15.45:0/2806296719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394069d20 0x7fb39406a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:34.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.484+0000 7fb39ac19700 1 -- 172.21.15.45:0/2806296719 >> 172.21.15.45:0/2806296719 conn(0x7fb394076730 msgr2=0x7fb394076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:34.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.484+0000 7fb39ac19700 1 -- 172.21.15.45:0/2806296719 shutdown_connections 2024-01-31T05:21:34.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.484+0000 7fb39ac19700 1 -- 172.21.15.45:0/2806296719 wait complete. 2024-01-31T05:21:34.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.484+0000 7fb39ac19700 1 Processor -- start 2024-01-31T05:21:34.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.484+0000 7fb39ac19700 1 -- start start 2024-01-31T05:21:34.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb39ac19700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb394069d20 0x7fb3941007e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:34.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb39ac19700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39406a6e0 0x7fb394100d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:34.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb39ac19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb394107790 0x7fb394101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:34.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb39ac19700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3941a5f90 con 0x7fb394107790 2024-01-31T05:21:34.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb39ac19700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3941a6100 con 0x7fb39406a6e0 2024-01-31T05:21:34.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb39ac19700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3941a62c0 con 0x7fb394069d20 2024-01-31T05:21:34.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb3991b6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb394107790 0x7fb394101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:34.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb3991b6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb394107790 0x7fb394101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51246/0 (socket says 172.21.15.45:51246) 2024-01-31T05:21:34.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb3991b6700 1 -- 172.21.15.45:0/3199787243 learned_addr learned my addr 172.21.15.45:0/3199787243 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:34.749 INFO:teuthology.orchestra.run.smithi045.stdout:77309411396 2024-01-31T05:21:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb393fff700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39406a6e0 0x7fb394100d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb3989b5700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb394069d20 0x7fb3941007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb3991b6700 1 -- 172.21.15.45:0/3199787243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb394069d20 msgr2=0x7fb3941007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb3991b6700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb394069d20 0x7fb3941007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb3991b6700 1 -- 172.21.15.45:0/3199787243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39406a6e0 msgr2=0x7fb394100d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb3991b6700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39406a6e0 0x7fb394100d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb3991b6700 1 -- 172.21.15.45:0/3199787243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880095f0 con 0x7fb394107790 2024-01-31T05:21:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb3991b6700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb394107790 0x7fb394101260 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb38c007d70 tx=0x7fb38c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb391ffb700 1 -- 172.21.15.45:0/3199787243 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c00ac90 con 0x7fb394107790 2024-01-31T05:21:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb391ffb700 1 -- 172.21.15.45:0/3199787243 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb38c00adf0 con 0x7fb394107790 2024-01-31T05:21:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb39ac19700 1 -- 172.21.15.45:0/3199787243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3941a65a0 con 0x7fb394107790 2024-01-31T05:21:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.485+0000 7fb391ffb700 1 -- 172.21.15.45:0/3199787243 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c0115e0 con 0x7fb394107790 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.486+0000 7fb39ac19700 1 -- 172.21.15.45:0/3199787243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3941a6a50 con 0x7fb394107790 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.487+0000 7fb39ac19700 1 -- 172.21.15.45:0/3199787243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb394109d20 con 0x7fb394107790 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.488+0000 7fb391ffb700 1 -- 172.21.15.45:0/3199787243 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fb38c012420 con 0x7fb394107790 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.488+0000 7fb391ffb700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb3840a1400 0x7fb3840a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.488+0000 7fb3989b5700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb3840a1400 0x7fb3840a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.488+0000 7fb391ffb700 1 -- 172.21.15.45:0/3199787243 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb38c0ccb90 con 0x7fb394107790 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.489+0000 7fb3989b5700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb3840a1400 0x7fb3840a38c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb388000c00 tx=0x7fb388009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.493+0000 7fb391ffb700 1 -- 172.21.15.45:0/3199787243 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb38c096170 con 0x7fb394107790 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.710+0000 7fb39ac19700 1 -- 172.21.15.45:0/3199787243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb394101d10 con 0x7fb394107790 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.711+0000 7fb391ffb700 1 -- 172.21.15.45:0/3199787243 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb38c016020 con 0x7fb394107790 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.713+0000 7fb39ac19700 1 -- 172.21.15.45:0/3199787243 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb3840a1400 msgr2=0x7fb3840a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.713+0000 7fb39ac19700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb3840a1400 0x7fb3840a38c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb388000c00 tx=0x7fb388009f10 comp rx=0 tx=0).stop 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.713+0000 7fb39ac19700 1 -- 172.21.15.45:0/3199787243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb394107790 msgr2=0x7fb394101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.713+0000 7fb39ac19700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb394107790 0x7fb394101260 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb38c007d70 tx=0x7fb38c00a340 comp rx=0 tx=0).stop 2024-01-31T05:21:34.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.713+0000 7fb39ac19700 1 -- 172.21.15.45:0/3199787243 shutdown_connections 2024-01-31T05:21:34.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.713+0000 7fb39ac19700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb3840a1400 0x7fb3840a38c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:34.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.713+0000 7fb39ac19700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb394107790 0x7fb394101260 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:34.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.713+0000 7fb39ac19700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb394069d20 0x7fb3941007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:34.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.713+0000 7fb39ac19700 1 --2- 172.21.15.45:0/3199787243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39406a6e0 0x7fb394100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:34.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.713+0000 7fb39ac19700 1 -- 172.21.15.45:0/3199787243 >> 172.21.15.45:0/3199787243 conn(0x7fb394076730 msgr2=0x7fb3940ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:34.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.713+0000 7fb39ac19700 1 -- 172.21.15.45:0/3199787243 shutdown_connections 2024-01-31T05:21:34.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:34.713+0000 7fb39ac19700 1 -- 172.21.15.45:0/3199787243 wait complete. 2024-01-31T05:21:35.337 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411392 got 77309411396 for osd.2 2024-01-31T05:21:35.337 DEBUG:teuthology.parallel:result is None 2024-01-31T05:21:35.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:35 smithi045 ceph-mon[31585]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:21:35.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:35 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3199787243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-01-31T05:21:35.906 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037796 got 257698037798 for osd.8 2024-01-31T05:21:35.906 DEBUG:teuthology.parallel:result is None 2024-01-31T05:21:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:35 smithi156 ceph-mon[35720]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:21:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:35 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3199787243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-01-31T05:21:36.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.087+0000 7fe7369dd700 1 -- 172.21.15.45:0/42580145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe730106650 msgr2=0x7fe730106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:36.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.087+0000 7fe7369dd700 1 --2- 172.21.15.45:0/42580145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe730106650 0x7fe730106a30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe7200099c0 tx=0x7fe720009cd0 comp rx=0 tx=0).stop 2024-01-31T05:21:36.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.087+0000 7fe7369dd700 1 -- 172.21.15.45:0/42580145 shutdown_connections 2024-01-31T05:21:36.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.087+0000 7fe7369dd700 1 --2- 172.21.15.45:0/42580145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7301021f0 0x7fe73010e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:36.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.087+0000 7fe7369dd700 1 --2- 172.21.15.45:0/42580145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe730106650 0x7fe730106a30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:36.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.087+0000 7fe7369dd700 1 --2- 172.21.15.45:0/42580145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe730101830 0x7fe730101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:36.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.087+0000 7fe7369dd700 1 -- 172.21.15.45:0/42580145 >> 172.21.15.45:0/42580145 conn(0x7fe730078910 msgr2=0x7fe730078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.087+0000 7fe7369dd700 1 -- 172.21.15.45:0/42580145 shutdown_connections 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.087+0000 7fe7369dd700 1 -- 172.21.15.45:0/42580145 wait complete. 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.088+0000 7fe7369dd700 1 Processor -- start 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.088+0000 7fe7369dd700 1 -- start start 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.088+0000 7fe7369dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe730101830 0x7fe730073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.088+0000 7fe7369dd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7301021f0 0x7fe730074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.088+0000 7fe7369dd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe730106650 0x7fe730074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.088+0000 7fe7369dd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7301a5f90 con 0x7fe730101830 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe7369dd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7301a6100 con 0x7fe7301021f0 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe7369dd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7301a62c0 con 0x7fe730106650 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe730101830 0x7fe730073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe730101830 0x7fe730073ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51264/0 (socket says 172.21.15.45:51264) 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72ffff700 1 -- 172.21.15.45:0/1295709442 learned_addr learned my addr 172.21.15.45:0/1295709442 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72f7fe700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7301021f0 0x7fe730074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe734f7a700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe730106650 0x7fe730074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72ffff700 1 -- 172.21.15.45:0/1295709442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe730106650 msgr2=0x7fe730074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72ffff700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe730106650 0x7fe730074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72ffff700 1 -- 172.21.15.45:0/1295709442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7301021f0 msgr2=0x7fe730074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72ffff700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7301021f0 0x7fe730074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72ffff700 1 -- 172.21.15.45:0/1295709442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7200095f0 con 0x7fe730101830 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe734f7a700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe730106650 0x7fe730074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72ffff700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe730101830 0x7fe730073ae0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe7200098e0 tx=0x7fe720005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72d7fa700 1 -- 172.21.15.45:0/1295709442 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72001d070 con 0x7fe730101830 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72d7fa700 1 -- 172.21.15.45:0/1295709442 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe72000fb30 con 0x7fe730101830 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe7369dd700 1 -- 172.21.15.45:0/1295709442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7301a6540 con 0x7fe730101830 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.089+0000 7fe72d7fa700 1 -- 172.21.15.45:0/1295709442 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720022c90 con 0x7fe730101830 2024-01-31T05:21:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.090+0000 7fe7369dd700 1 -- 172.21.15.45:0/1295709442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7301a69b0 con 0x7fe730101830 2024-01-31T05:21:36.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:35 smithi186 ceph-mon[35659]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:21:36.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:35 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3199787243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-01-31T05:21:36.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.090+0000 7fe7369dd700 1 -- 172.21.15.45:0/1295709442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe730109d10 con 0x7fe730101830 2024-01-31T05:21:36.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.091+0000 7fe72d7fa700 1 -- 172.21.15.45:0/1295709442 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fe72000fca0 con 0x7fe730101830 2024-01-31T05:21:36.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.092+0000 7fe72d7fa700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe7180a1480 0x7fe7180a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:36.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.092+0000 7fe72f7fe700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe7180a1480 0x7fe7180a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:36.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.092+0000 7fe72d7fa700 1 -- 172.21.15.45:0/1295709442 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe7200cdeb0 con 0x7fe730101830 2024-01-31T05:21:36.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.093+0000 7fe72f7fe700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe7180a1480 0x7fe7180a3940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe730078ef0 tx=0x7fe72400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:36.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.097+0000 7fe72d7fa700 1 -- 172.21.15.45:0/1295709442 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe720097410 con 0x7fe730101830 2024-01-31T05:21:36.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.303+0000 7fe7369dd700 1 -- 172.21.15.45:0/1295709442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fe730074fc0 con 0x7fe730101830 2024-01-31T05:21:36.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.303+0000 7fe72d7fa700 1 -- 172.21.15.45:0/1295709442 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fe720027020 con 0x7fe730101830 2024-01-31T05:21:36.303 INFO:teuthology.orchestra.run.smithi045.stdout:313532612636 2024-01-31T05:21:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.305+0000 7fe7369dd700 1 -- 172.21.15.45:0/1295709442 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe7180a1480 msgr2=0x7fe7180a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.305+0000 7fe7369dd700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe7180a1480 0x7fe7180a3940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe730078ef0 tx=0x7fe72400a380 comp rx=0 tx=0).stop 2024-01-31T05:21:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.305+0000 7fe7369dd700 1 -- 172.21.15.45:0/1295709442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe730101830 msgr2=0x7fe730073ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.305+0000 7fe7369dd700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe730101830 0x7fe730073ae0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe7200098e0 tx=0x7fe720005740 comp rx=0 tx=0).stop 2024-01-31T05:21:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.306+0000 7fe7369dd700 1 -- 172.21.15.45:0/1295709442 shutdown_connections 2024-01-31T05:21:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.306+0000 7fe7369dd700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe730106650 0x7fe730074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.306+0000 7fe7369dd700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe7180a1480 0x7fe7180a3940 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.306+0000 7fe7369dd700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe730101830 0x7fe730073ae0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.306+0000 7fe7369dd700 1 --2- 172.21.15.45:0/1295709442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7301021f0 0x7fe730074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.306+0000 7fe7369dd700 1 -- 172.21.15.45:0/1295709442 >> 172.21.15.45:0/1295709442 conn(0x7fe730078910 msgr2=0x7fe7300ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.306+0000 7fe7369dd700 1 -- 172.21.15.45:0/1295709442 shutdown_connections 2024-01-31T05:21:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:36.306+0000 7fe7369dd700 1 -- 172.21.15.45:0/1295709442 wait complete. 2024-01-31T05:21:36.788 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:36 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1295709442' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-01-31T05:21:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:36 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1295709442' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-01-31T05:21:37.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:36 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1295709442' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-01-31T05:21:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.515+0000 7f9623135700 1 -- 172.21.15.45:0/1096494565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c100940 msgr2=0x7f961c106480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.515+0000 7f9623135700 1 --2- 172.21.15.45:0/1096494565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c100940 0x7f961c106480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f960c00b0a0 tx=0x7f960c00b3b0 comp rx=0 tx=0).stop 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.515+0000 7f9623135700 1 -- 172.21.15.45:0/1096494565 shutdown_connections 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.515+0000 7f9623135700 1 --2- 172.21.15.45:0/1096494565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f961c106bb0 0x7f961c10e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.515+0000 7f9623135700 1 --2- 172.21.15.45:0/1096494565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f961c0fff90 0x7f961c100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.515+0000 7f9623135700 1 --2- 172.21.15.45:0/1096494565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c100940 0x7f961c106480 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.515+0000 7f9623135700 1 -- 172.21.15.45:0/1096494565 >> 172.21.15.45:0/1096494565 conn(0x7f961c076ce0 msgr2=0x7f961c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.515+0000 7f9623135700 1 -- 172.21.15.45:0/1096494565 shutdown_connections 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.515+0000 7f9623135700 1 -- 172.21.15.45:0/1096494565 wait complete. 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.516+0000 7f9623135700 1 Processor -- start 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.516+0000 7f9623135700 1 -- start start 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.516+0000 7f9623135700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c0fff90 0x7f961c197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.516+0000 7f9623135700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f961c100940 0x7f961c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.516+0000 7f9623135700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f961c106bb0 0x7f961c19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.516+0000 7f9623135700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c19cc50 con 0x7f961c0fff90 2024-01-31T05:21:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.516+0000 7f9623135700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c19cdc0 con 0x7f961c106bb0 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.516+0000 7f9623135700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c19cf80 con 0x7f961c100940 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.516+0000 7f9620ed1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c0fff90 0x7f961c197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.516+0000 7f9620ed1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c0fff90 0x7f961c197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41278/0 (socket says 172.21.15.45:41278) 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.516+0000 7f9620ed1700 1 -- 172.21.15.45:0/1107832238 learned_addr learned my addr 172.21.15.45:0/1107832238 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.516+0000 7f961bfff700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f961c100940 0x7f961c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f96216d2700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f961c106bb0 0x7f961c19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f9620ed1700 1 -- 172.21.15.45:0/1107832238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f961c100940 msgr2=0x7f961c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f9620ed1700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f961c100940 0x7f961c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f9620ed1700 1 -- 172.21.15.45:0/1107832238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f961c106bb0 msgr2=0x7f961c19c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f9620ed1700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f961c106bb0 0x7f961c19c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f9620ed1700 1 -- 172.21.15.45:0/1107832238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c009d00 con 0x7f961c0fff90 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f96216d2700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f961c106bb0 0x7f961c19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f9620ed1700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c0fff90 0x7f961c197d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f961400c920 tx=0x7f961400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f9619ffb700 1 -- 172.21.15.45:0/1107832238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9614007b00 con 0x7f961c0fff90 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f9623135700 1 -- 172.21.15.45:0/1107832238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961c19d260 con 0x7f961c0fff90 2024-01-31T05:21:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f9619ffb700 1 -- 172.21.15.45:0/1107832238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9614004cf0 con 0x7f961c0fff90 2024-01-31T05:21:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f9619ffb700 1 -- 172.21.15.45:0/1107832238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9614005610 con 0x7f961c0fff90 2024-01-31T05:21:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.517+0000 7f9623135700 1 -- 172.21.15.45:0/1107832238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961c1a4ac0 con 0x7f961c0fff90 2024-01-31T05:21:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.518+0000 7f9623135700 1 -- 172.21.15.45:0/1107832238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961c109dd0 con 0x7f961c0fff90 2024-01-31T05:21:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.520+0000 7f9619ffb700 1 -- 172.21.15.45:0/1107832238 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f9614004750 con 0x7f961c0fff90 2024-01-31T05:21:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.520+0000 7f9619ffb700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f96080a1400 0x7f96080a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.521+0000 7f9619ffb700 1 -- 172.21.15.45:0/1107832238 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f96140ccb30 con 0x7f961c0fff90 2024-01-31T05:21:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.521+0000 7f961bfff700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f96080a1400 0x7f96080a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.521+0000 7f961bfff700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f96080a1400 0x7f96080a38c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f961c104320 tx=0x7f960c00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:37.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.524+0000 7f9619ffb700 1 -- 172.21.15.45:0/1107832238 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f9614096000 con 0x7f961c0fff90 2024-01-31T05:21:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.727+0000 7f9623135700 1 -- 172.21.15.45:0/1107832238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f961c1990d0 con 0x7f961c0fff90 2024-01-31T05:21:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.727+0000 7f9619ffb700 1 -- 172.21.15.45:0/1107832238 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f961401d020 con 0x7f961c0fff90 2024-01-31T05:21:37.726 INFO:teuthology.orchestra.run.smithi045.stdout:55834574922 2024-01-31T05:21:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.729+0000 7f9623135700 1 -- 172.21.15.45:0/1107832238 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f96080a1400 msgr2=0x7f96080a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.729+0000 7f9623135700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f96080a1400 0x7f96080a38c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f961c104320 tx=0x7f960c00b7b0 comp rx=0 tx=0).stop 2024-01-31T05:21:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.729+0000 7f9623135700 1 -- 172.21.15.45:0/1107832238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c0fff90 msgr2=0x7f961c197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.729+0000 7f9623135700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c0fff90 0x7f961c197d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f961400c920 tx=0x7f961400cce0 comp rx=0 tx=0).stop 2024-01-31T05:21:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.729+0000 7f9623135700 1 -- 172.21.15.45:0/1107832238 shutdown_connections 2024-01-31T05:21:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.729+0000 7f9623135700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f961c106bb0 0x7f961c19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:37.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.729+0000 7f9623135700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f96080a1400 0x7f96080a38c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:37.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.729+0000 7f9623135700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c0fff90 0x7f961c197d70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:37.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.729+0000 7f9623135700 1 --2- 172.21.15.45:0/1107832238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f961c100940 0x7f961c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:37.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.729+0000 7f9623135700 1 -- 172.21.15.45:0/1107832238 >> 172.21.15.45:0/1107832238 conn(0x7f961c076ce0 msgr2=0x7f961c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:37.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.729+0000 7f9623135700 1 -- 172.21.15.45:0/1107832238 shutdown_connections 2024-01-31T05:21:37.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:37.729+0000 7f9623135700 1 -- 172.21.15.45:0/1107832238 wait complete. 2024-01-31T05:21:37.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:37 smithi045 ceph-mon[31585]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:21:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:37 smithi156 ceph-mon[35720]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:21:38.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:37 smithi186 ceph-mon[35659]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:21:38.364 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574922 for osd.1 2024-01-31T05:21:38.365 DEBUG:teuthology.parallel:result is None 2024-01-31T05:21:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.743+0000 7f1a7866d700 1 -- 172.21.15.45:0/3002223176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a70106850 msgr2=0x7f1a70106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.743+0000 7f1a7866d700 1 --2- 172.21.15.45:0/3002223176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a70106850 0x7f1a70106c30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1a600099c0 tx=0x7f1a60009cd0 comp rx=0 tx=0).stop 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 -- 172.21.15.45:0/3002223176 shutdown_connections 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 --2- 172.21.15.45:0/3002223176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a7006a8e0 0x7f1a7010e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 --2- 172.21.15.45:0/3002223176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a70106850 0x7f1a70106c30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 --2- 172.21.15.45:0/3002223176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a70069f20 0x7f1a7006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 -- 172.21.15.45:0/3002223176 >> 172.21.15.45:0/3002223176 conn(0x7f1a70077540 msgr2=0x7f1a70077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 -- 172.21.15.45:0/3002223176 shutdown_connections 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 -- 172.21.15.45:0/3002223176 wait complete. 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 Processor -- start 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 -- start start 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a70069f20 0x7f1a70100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7006a8e0 0x7f1a70100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a70106850 0x7f1a70101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a701a6060 con 0x7f1a7006a8e0 2024-01-31T05:21:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a701a61d0 con 0x7f1a70106850 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.744+0000 7f1a7866d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a701a6390 con 0x7f1a70069f20 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a75c08700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7006a8e0 0x7f1a70100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a75c08700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7006a8e0 0x7f1a70100fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41292/0 (socket says 172.21.15.45:41292) 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a75c08700 1 -- 172.21.15.45:0/1340891882 learned_addr learned my addr 172.21.15.45:0/1340891882 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a76409700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a70069f20 0x7f1a70100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a76c0a700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a70106850 0x7f1a70101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a75c08700 1 -- 172.21.15.45:0/1340891882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a70069f20 msgr2=0x7f1a70100a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a75c08700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a70069f20 0x7f1a70100a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a75c08700 1 -- 172.21.15.45:0/1340891882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a70106850 msgr2=0x7f1a70101510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a75c08700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a70106850 0x7f1a70101510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a75c08700 1 -- 172.21.15.45:0/1340891882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600095f0 con 0x7f1a7006a8e0 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a76c0a700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a70106850 0x7f1a70101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a75c08700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7006a8e0 0x7f1a70100fd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00b530 tx=0x7f1a6c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:38.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a677fe700 1 -- 172.21.15.45:0/1340891882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c011830 con 0x7f1a7006a8e0 2024-01-31T05:21:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a7866d700 1 -- 172.21.15.45:0/1340891882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a701a6670 con 0x7f1a7006a8e0 2024-01-31T05:21:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a677fe700 1 -- 172.21.15.45:0/1340891882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a6c011e50 con 0x7f1a7006a8e0 2024-01-31T05:21:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a677fe700 1 -- 172.21.15.45:0/1340891882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c00f2e0 con 0x7f1a7006a8e0 2024-01-31T05:21:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.745+0000 7f1a7866d700 1 -- 172.21.15.45:0/1340891882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a701a6bc0 con 0x7f1a7006a8e0 2024-01-31T05:21:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.746+0000 7f1a7866d700 1 -- 172.21.15.45:0/1340891882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a70109e60 con 0x7f1a7006a8e0 2024-01-31T05:21:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.748+0000 7f1a677fe700 1 -- 172.21.15.45:0/1340891882 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f1a6c010380 con 0x7f1a7006a8e0 2024-01-31T05:21:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.748+0000 7f1a677fe700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1a5c0a1400 0x7f1a5c0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.748+0000 7f1a76409700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1a5c0a1400 0x7f1a5c0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.748+0000 7f1a677fe700 1 -- 172.21.15.45:0/1340891882 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1a6c0ccb20 con 0x7f1a7006a8e0 2024-01-31T05:21:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.749+0000 7f1a76409700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1a5c0a1400 0x7f1a5c0a38c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009990 tx=0x7f1a60005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:38.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.752+0000 7f1a677fe700 1 -- 172.21.15.45:0/1340891882 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1a6c096100 con 0x7f1a7006a8e0 2024-01-31T05:21:38.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:38 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1107832238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-01-31T05:21:38.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.968+0000 7f1a7866d700 1 -- 172.21.15.45:0/1340891882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1a70101f70 con 0x7f1a7006a8e0 2024-01-31T05:21:38.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.969+0000 7f1a677fe700 1 -- 172.21.15.45:0/1340891882 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a6c099720 con 0x7f1a7006a8e0 2024-01-31T05:21:38.968 INFO:teuthology.orchestra.run.smithi045.stdout:111669149760 2024-01-31T05:21:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.971+0000 7f1a7866d700 1 -- 172.21.15.45:0/1340891882 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1a5c0a1400 msgr2=0x7f1a5c0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.971+0000 7f1a7866d700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1a5c0a1400 0x7f1a5c0a38c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009990 tx=0x7f1a60005bc0 comp rx=0 tx=0).stop 2024-01-31T05:21:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.971+0000 7f1a7866d700 1 -- 172.21.15.45:0/1340891882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7006a8e0 msgr2=0x7f1a70100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.971+0000 7f1a7866d700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7006a8e0 0x7f1a70100fd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00b530 tx=0x7f1a6c00b840 comp rx=0 tx=0).stop 2024-01-31T05:21:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.971+0000 7f1a7866d700 1 -- 172.21.15.45:0/1340891882 shutdown_connections 2024-01-31T05:21:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.971+0000 7f1a7866d700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a70106850 0x7f1a70101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.971+0000 7f1a7866d700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a70069f20 0x7f1a70100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.971+0000 7f1a7866d700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1a5c0a1400 0x7f1a5c0a38c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.971+0000 7f1a7866d700 1 --2- 172.21.15.45:0/1340891882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7006a8e0 0x7f1a70100fd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.971+0000 7f1a7866d700 1 -- 172.21.15.45:0/1340891882 >> 172.21.15.45:0/1340891882 conn(0x7f1a70077540 msgr2=0x7f1a700fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.971+0000 7f1a7866d700 1 -- 172.21.15.45:0/1340891882 shutdown_connections 2024-01-31T05:21:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:38.971+0000 7f1a7866d700 1 -- 172.21.15.45:0/1340891882 wait complete. 2024-01-31T05:21:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:38 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1107832238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-01-31T05:21:39.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:38 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1107832238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-01-31T05:21:39.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.449+0000 7f70502cb700 1 -- 172.21.15.45:0/4017220114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480ffdd0 msgr2=0x7f7048100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:39.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.449+0000 7f70502cb700 1 --2- 172.21.15.45:0/4017220114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480ffdd0 0x7f7048100250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7038009990 tx=0x7f7038009ca0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.449+0000 7f70502cb700 1 -- 172.21.15.45:0/4017220114 shutdown_connections 2024-01-31T05:21:39.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.449+0000 7f70502cb700 1 --2- 172.21.15.45:0/4017220114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7048100790 0x7f704810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.449+0000 7f70502cb700 1 --2- 172.21.15.45:0/4017220114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7048106850 0x7f7048106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.449+0000 7f70502cb700 1 --2- 172.21.15.45:0/4017220114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480ffdd0 0x7f7048100250 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.449+0000 7f70502cb700 1 -- 172.21.15.45:0/4017220114 >> 172.21.15.45:0/4017220114 conn(0x7f7048076ce0 msgr2=0x7f70480fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.449+0000 7f70502cb700 1 -- 172.21.15.45:0/4017220114 shutdown_connections 2024-01-31T05:21:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.449+0000 7f70502cb700 1 -- 172.21.15.45:0/4017220114 wait complete. 2024-01-31T05:21:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f70502cb700 1 Processor -- start 2024-01-31T05:21:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f70502cb700 1 -- start start 2024-01-31T05:21:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f70502cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480ffdd0 0x7f7048100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f70502cb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7048100790 0x7f70481014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f70502cb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7048106850 0x7f7048101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f70502cb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70481a6180 con 0x7f70480ffdd0 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f70502cb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70481a62f0 con 0x7f7048106850 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f70502cb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70481a64b0 con 0x7f7048100790 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f704e067700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480ffdd0 0x7f7048100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f704e067700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480ffdd0 0x7f7048100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41310/0 (socket says 172.21.15.45:41310) 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f704e067700 1 -- 172.21.15.45:0/2744700695 learned_addr learned my addr 172.21.15.45:0/2744700695 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f704d866700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7048100790 0x7f70481014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f704e868700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7048106850 0x7f7048101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f704e067700 1 -- 172.21.15.45:0/2744700695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7048100790 msgr2=0x7f70481014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f704e067700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7048100790 0x7f70481014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f704e067700 1 -- 172.21.15.45:0/2744700695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7048106850 msgr2=0x7f7048101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f704e067700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7048106850 0x7f7048101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.450+0000 7f704e067700 1 -- 172.21.15.45:0/2744700695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70380095f0 con 0x7f70480ffdd0 2024-01-31T05:21:39.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.451+0000 7f704e067700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480ffdd0 0x7f7048100fb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f704400d980 tx=0x7f704400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:39.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.451+0000 7f703f7fe700 1 -- 172.21.15.45:0/2744700695 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044004220 con 0x7f70480ffdd0 2024-01-31T05:21:39.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.451+0000 7f70502cb700 1 -- 172.21.15.45:0/2744700695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70481a6790 con 0x7f70480ffdd0 2024-01-31T05:21:39.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.451+0000 7f703f7fe700 1 -- 172.21.15.45:0/2744700695 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7044007640 con 0x7f70480ffdd0 2024-01-31T05:21:39.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.451+0000 7f703f7fe700 1 -- 172.21.15.45:0/2744700695 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044005690 con 0x7f70480ffdd0 2024-01-31T05:21:39.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.451+0000 7f70502cb700 1 -- 172.21.15.45:0/2744700695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70481a6c60 con 0x7f70480ffdd0 2024-01-31T05:21:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.452+0000 7f70502cb700 1 -- 172.21.15.45:0/2744700695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7048109d70 con 0x7f70480ffdd0 2024-01-31T05:21:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.453+0000 7f703f7fe700 1 -- 172.21.15.45:0/2744700695 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f7044004750 con 0x7f70480ffdd0 2024-01-31T05:21:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.454+0000 7f703f7fe700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f70340a1400 0x7f70340a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.454+0000 7f704d866700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f70340a1400 0x7f70340a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.454+0000 7f703f7fe700 1 -- 172.21.15.45:0/2744700695 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f70440cc890 con 0x7f70480ffdd0 2024-01-31T05:21:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.454+0000 7f704d866700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f70340a1400 0x7f70340a38c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f70380098e0 tx=0x7f7038005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:39.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.462+0000 7f703f7fe700 1 -- 172.21.15.45:0/2744700695 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7044095d60 con 0x7f70480ffdd0 2024-01-31T05:21:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.667+0000 7f70502cb700 1 -- 172.21.15.45:0/2744700695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f70481024e0 con 0x7f70480ffdd0 2024-01-31T05:21:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.668+0000 7f703f7fe700 1 -- 172.21.15.45:0/2744700695 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7044016090 con 0x7f70480ffdd0 2024-01-31T05:21:39.667 INFO:teuthology.orchestra.run.smithi045.stdout:292057776163 2024-01-31T05:21:39.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.670+0000 7f70502cb700 1 -- 172.21.15.45:0/2744700695 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f70340a1400 msgr2=0x7f70340a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:39.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.670+0000 7f70502cb700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f70340a1400 0x7f70340a38c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f70380098e0 tx=0x7f7038005960 comp rx=0 tx=0).stop 2024-01-31T05:21:39.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.670+0000 7f70502cb700 1 -- 172.21.15.45:0/2744700695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480ffdd0 msgr2=0x7f7048100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:39.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.670+0000 7f70502cb700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480ffdd0 0x7f7048100fb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f704400d980 tx=0x7f704400dc90 comp rx=0 tx=0).stop 2024-01-31T05:21:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.670+0000 7f70502cb700 1 -- 172.21.15.45:0/2744700695 shutdown_connections 2024-01-31T05:21:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.670+0000 7f70502cb700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7048106850 0x7f7048101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.670+0000 7f70502cb700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f70340a1400 0x7f70340a38c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.670+0000 7f70502cb700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480ffdd0 0x7f7048100fb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.670+0000 7f70502cb700 1 --2- 172.21.15.45:0/2744700695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7048100790 0x7f70481014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.670+0000 7f70502cb700 1 -- 172.21.15.45:0/2744700695 >> 172.21.15.45:0/2744700695 conn(0x7f7048076ce0 msgr2=0x7f70480fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.671+0000 7f70502cb700 1 -- 172.21.15.45:0/2744700695 shutdown_connections 2024-01-31T05:21:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.671+0000 7f70502cb700 1 -- 172.21.15.45:0/2744700695 wait complete. 2024-01-31T05:21:39.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.684+0000 7f4d38418700 1 -- 172.21.15.45:0/2174309044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d30106680 msgr2=0x7f4d30106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:39.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.684+0000 7f4d38418700 1 --2- 172.21.15.45:0/2174309044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d30106680 0x7f4d30106a60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009990 tx=0x7f4d28009ca0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.684+0000 7f4d38418700 1 -- 172.21.15.45:0/2174309044 shutdown_connections 2024-01-31T05:21:39.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.684+0000 7f4d38418700 1 --2- 172.21.15.45:0/2174309044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d301010d0 0x7f4d3010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.684+0000 7f4d38418700 1 --2- 172.21.15.45:0/2174309044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d30106680 0x7f4d30106a60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.684+0000 7f4d38418700 1 --2- 172.21.15.45:0/2174309044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d30100710 0x7f4d30100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.684+0000 7f4d38418700 1 -- 172.21.15.45:0/2174309044 >> 172.21.15.45:0/2174309044 conn(0x7f4d30076730 msgr2=0x7f4d30076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:39.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.684+0000 7f4d38418700 1 -- 172.21.15.45:0/2174309044 shutdown_connections 2024-01-31T05:21:39.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.684+0000 7f4d38418700 1 -- 172.21.15.45:0/2174309044 wait complete. 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.685+0000 7f4d38418700 1 Processor -- start 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.685+0000 7f4d38418700 1 -- start start 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.685+0000 7f4d38418700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d30100710 0x7f4d30196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.685+0000 7f4d38418700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d301010d0 0x7f4d30196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.685+0000 7f4d38418700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d30106680 0x7f4d30197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.685+0000 7f4d38418700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d301a5fd0 con 0x7f4d301010d0 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.685+0000 7f4d38418700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d301a6140 con 0x7f4d30100710 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.685+0000 7f4d38418700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d301a6300 con 0x7f4d30106680 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.685+0000 7f4d359b3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d301010d0 0x7f4d30196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.685+0000 7f4d359b3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d301010d0 0x7f4d30196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41334/0 (socket says 172.21.15.45:41334) 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.685+0000 7f4d359b3700 1 -- 172.21.15.45:0/2549366874 learned_addr learned my addr 172.21.15.45:0/2549366874 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.685+0000 7f4d361b4700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d30100710 0x7f4d30196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.686+0000 7f4d369b5700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d30106680 0x7f4d30197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.686+0000 7f4d359b3700 1 -- 172.21.15.45:0/2549366874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d30106680 msgr2=0x7f4d30197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.686+0000 7f4d359b3700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d30106680 0x7f4d30197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.686+0000 7f4d359b3700 1 -- 172.21.15.45:0/2549366874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d30100710 msgr2=0x7f4d30196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.686+0000 7f4d359b3700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d30100710 0x7f4d30196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.686+0000 7f4d359b3700 1 -- 172.21.15.45:0/2549366874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d280095f0 con 0x7f4d301010d0 2024-01-31T05:21:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.686+0000 7f4d359b3700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d301010d0 0x7f4d30196dd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400d950 tx=0x7f4d2400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.686+0000 7f4d2f7fe700 1 -- 172.21.15.45:0/2549366874 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24009940 con 0x7f4d301010d0 2024-01-31T05:21:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.686+0000 7f4d38418700 1 -- 172.21.15.45:0/2549366874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d301a65e0 con 0x7f4d301010d0 2024-01-31T05:21:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.686+0000 7f4d2f7fe700 1 -- 172.21.15.45:0/2549366874 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d24010460 con 0x7f4d301010d0 2024-01-31T05:21:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.686+0000 7f4d2f7fe700 1 -- 172.21.15.45:0/2549366874 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2400f620 con 0x7f4d301010d0 2024-01-31T05:21:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.686+0000 7f4d38418700 1 -- 172.21.15.45:0/2549366874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d301a6ae0 con 0x7f4d301010d0 2024-01-31T05:21:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.687+0000 7f4d38418700 1 -- 172.21.15.45:0/2549366874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d30108c80 con 0x7f4d301010d0 2024-01-31T05:21:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.689+0000 7f4d2f7fe700 1 -- 172.21.15.45:0/2549366874 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f4d240108d0 con 0x7f4d301010d0 2024-01-31T05:21:39.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.689+0000 7f4d2f7fe700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f4d180a1400 0x7f4d180a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:39.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.690+0000 7f4d361b4700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f4d180a1400 0x7f4d180a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:39.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.690+0000 7f4d2f7fe700 1 -- 172.21.15.45:0/2549366874 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4d240cccf0 con 0x7f4d301010d0 2024-01-31T05:21:39.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.690+0000 7f4d361b4700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f4d180a1400 0x7f4d180a38c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800b2e0 tx=0x7f4d28005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:39.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.694+0000 7f4d2f7fe700 1 -- 172.21.15.45:0/2549366874 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f4d240961c0 con 0x7f4d301010d0 2024-01-31T05:21:39.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.896+0000 7f4d38418700 1 -- 172.21.15.45:0/2549366874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4d3006a310 con 0x7f4d301010d0 2024-01-31T05:21:39.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.897+0000 7f4d2f7fe700 1 -- 172.21.15.45:0/2549366874 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4d24020090 con 0x7f4d301010d0 2024-01-31T05:21:39.896 INFO:teuthology.orchestra.run.smithi045.stdout:34359738446 2024-01-31T05:21:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.899+0000 7f4d38418700 1 -- 172.21.15.45:0/2549366874 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f4d180a1400 msgr2=0x7f4d180a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.899+0000 7f4d38418700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f4d180a1400 0x7f4d180a38c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800b2e0 tx=0x7f4d28005960 comp rx=0 tx=0).stop 2024-01-31T05:21:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.899+0000 7f4d38418700 1 -- 172.21.15.45:0/2549366874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d301010d0 msgr2=0x7f4d30196dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.899+0000 7f4d38418700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d301010d0 0x7f4d30196dd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400d950 tx=0x7f4d2400dc60 comp rx=0 tx=0).stop 2024-01-31T05:21:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.899+0000 7f4d38418700 1 -- 172.21.15.45:0/2549366874 shutdown_connections 2024-01-31T05:21:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.899+0000 7f4d38418700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d30106680 0x7f4d30197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.899+0000 7f4d38418700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d30100710 0x7f4d30196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.899+0000 7f4d38418700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f4d180a1400 0x7f4d180a38c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.899+0000 7f4d38418700 1 --2- 172.21.15.45:0/2549366874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d301010d0 0x7f4d30196dd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.899+0000 7f4d38418700 1 -- 172.21.15.45:0/2549366874 >> 172.21.15.45:0/2549366874 conn(0x7f4d30076730 msgr2=0x7f4d300fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.899+0000 7f4d38418700 1 -- 172.21.15.45:0/2549366874 shutdown_connections 2024-01-31T05:21:39.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:39.899+0000 7f4d38418700 1 -- 172.21.15.45:0/2549366874 wait complete. 2024-01-31T05:21:39.926 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:39 smithi186 ceph-mon[35659]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:21:39.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:39 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1340891882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-01-31T05:21:39.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:39 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:21:39.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:39 smithi156 ceph-mon[35720]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:21:39.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:39 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1340891882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-01-31T05:21:39.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:39 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:21:39.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:39 smithi045 ceph-mon[31585]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:21:39.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:39 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1340891882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-01-31T05:21:39.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:39 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:21:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:40 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2744700695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-01-31T05:21:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:40 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2549366874' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-01-31T05:21:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:40 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:40 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:40 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:40 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:41.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:40 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2744700695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-01-31T05:21:41.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:40 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2549366874' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-01-31T05:21:41.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:40 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:41.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:40 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:41.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:40 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:41.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:40 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:41.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:40 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2744700695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-01-31T05:21:41.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:40 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2549366874' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-01-31T05:21:41.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:40 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:41.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:40 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:41.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:40 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:41.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:40 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:41.378 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776160 got 292057776163 for osd.9 2024-01-31T05:21:41.378 DEBUG:teuthology.parallel:result is None 2024-01-31T05:21:41.970 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:41 smithi045 ceph-mon[31585]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:21:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:41 smithi156 ceph-mon[35720]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:21:41.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.981+0000 7fe04cf77700 1 -- 172.21.15.45:0/1769870958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe048101860 msgr2=0x7fe048101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:41.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.981+0000 7fe04cf77700 1 --2- 172.21.15.45:0/1769870958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe048101860 0x7fe048101ce0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe030009990 tx=0x7fe030009ca0 comp rx=0 tx=0).stop 2024-01-31T05:21:41.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.981+0000 7fe04cf77700 1 -- 172.21.15.45:0/1769870958 shutdown_connections 2024-01-31T05:21:41.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.981+0000 7fe04cf77700 1 --2- 172.21.15.45:0/1769870958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe048102220 0x7fe04810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:41.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.981+0000 7fe04cf77700 1 --2- 172.21.15.45:0/1769870958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0481077c0 0x7fe048107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:41.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.981+0000 7fe04cf77700 1 --2- 172.21.15.45:0/1769870958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe048101860 0x7fe048101ce0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:41.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.981+0000 7fe04cf77700 1 -- 172.21.15.45:0/1769870958 >> 172.21.15.45:0/1769870958 conn(0x7fe048078870 msgr2=0x7fe048078c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:41.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.981+0000 7fe04cf77700 1 -- 172.21.15.45:0/1769870958 shutdown_connections 2024-01-31T05:21:41.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.981+0000 7fe04cf77700 1 -- 172.21.15.45:0/1769870958 wait complete. 2024-01-31T05:21:41.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.982+0000 7fe04cf77700 1 Processor -- start 2024-01-31T05:21:41.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.982+0000 7fe04cf77700 1 -- start start 2024-01-31T05:21:41.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.982+0000 7fe04cf77700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe048101860 0x7fe04819d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:41.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.982+0000 7fe04cf77700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe048102220 0x7fe04819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:41.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.982+0000 7fe04cf77700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0481077c0 0x7fe048197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:41.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.982+0000 7fe04cf77700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0481980b0 con 0x7fe0481077c0 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.982+0000 7fe04cf77700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0481981f0 con 0x7fe048101860 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.982+0000 7fe04cf77700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0481983b0 con 0x7fe048102220 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.982+0000 7fe046d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0481077c0 0x7fe048197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.982+0000 7fe046d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0481077c0 0x7fe048197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41354/0 (socket says 172.21.15.45:41354) 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.982+0000 7fe046d9d700 1 -- 172.21.15.45:0/2472963960 learned_addr learned my addr 172.21.15.45:0/2472963960 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe04659c700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe048101860 0x7fe04819d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe045d9b700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe048102220 0x7fe04819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe046d9d700 1 -- 172.21.15.45:0/2472963960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe048102220 msgr2=0x7fe04819ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe046d9d700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe048102220 0x7fe04819ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe046d9d700 1 -- 172.21.15.45:0/2472963960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe048101860 msgr2=0x7fe04819d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe046d9d700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe048101860 0x7fe04819d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe046d9d700 1 -- 172.21.15.45:0/2472963960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0300095f0 con 0x7fe0481077c0 2024-01-31T05:21:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe045d9b700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe048102220 0x7fe04819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:21:41.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe046d9d700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0481077c0 0x7fe048197a50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b970 tx=0x7fe03800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:41.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe03f7fe700 1 -- 172.21.15.45:0/2472963960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03800c930 con 0x7fe0481077c0 2024-01-31T05:21:41.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe03f7fe700 1 -- 172.21.15.45:0/2472963960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe038014440 con 0x7fe0481077c0 2024-01-31T05:21:41.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe04cf77700 1 -- 172.21.15.45:0/2472963960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe048198690 con 0x7fe0481077c0 2024-01-31T05:21:41.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe03f7fe700 1 -- 172.21.15.45:0/2472963960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0380075d0 con 0x7fe0481077c0 2024-01-31T05:21:41.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.983+0000 7fe04cf77700 1 -- 172.21.15.45:0/2472963960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0481a4930 con 0x7fe0481077c0 2024-01-31T05:21:41.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.984+0000 7fe04cf77700 1 -- 172.21.15.45:0/2472963960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe048109d00 con 0x7fe0481077c0 2024-01-31T05:21:41.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.985+0000 7fe03f7fe700 1 -- 172.21.15.45:0/2472963960 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fe0380145b0 con 0x7fe0481077c0 2024-01-31T05:21:41.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.986+0000 7fe03f7fe700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe0340a1480 0x7fe0340a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:41.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.986+0000 7fe04659c700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe0340a1480 0x7fe0340a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:41.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.986+0000 7fe03f7fe700 1 -- 172.21.15.45:0/2472963960 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe038006e40 con 0x7fe0481077c0 2024-01-31T05:21:41.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.986+0000 7fe04659c700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe0340a1480 0x7fe0340a3940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe048198b90 tx=0x7fe04000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:41.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:41.991+0000 7fe03f7fe700 1 -- 172.21.15.45:0/2472963960 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe0380955f0 con 0x7fe0481077c0 2024-01-31T05:21:42.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:41 smithi186 ceph-mon[35659]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:21:42.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.196+0000 7fe04cf77700 1 -- 172.21.15.45:0/2472963960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe048198fb0 con 0x7fe0481077c0 2024-01-31T05:21:42.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.196+0000 7fe03f7fe700 1 -- 172.21.15.45:0/2472963960 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe038019020 con 0x7fe0481077c0 2024-01-31T05:21:42.196 INFO:teuthology.orchestra.run.smithi045.stdout:201863462961 2024-01-31T05:21:42.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.198+0000 7fe04cf77700 1 -- 172.21.15.45:0/2472963960 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe0340a1480 msgr2=0x7fe0340a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:42.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.198+0000 7fe04cf77700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe0340a1480 0x7fe0340a3940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe048198b90 tx=0x7fe04000a380 comp rx=0 tx=0).stop 2024-01-31T05:21:42.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.198+0000 7fe04cf77700 1 -- 172.21.15.45:0/2472963960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0481077c0 msgr2=0x7fe048197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:42.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.198+0000 7fe04cf77700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0481077c0 0x7fe048197a50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b970 tx=0x7fe03800bc80 comp rx=0 tx=0).stop 2024-01-31T05:21:42.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.199+0000 7fe04cf77700 1 -- 172.21.15.45:0/2472963960 shutdown_connections 2024-01-31T05:21:42.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.199+0000 7fe04cf77700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe0340a1480 0x7fe0340a3940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.199+0000 7fe04cf77700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0481077c0 0x7fe048197a50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.199+0000 7fe04cf77700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe048101860 0x7fe04819d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.199+0000 7fe04cf77700 1 --2- 172.21.15.45:0/2472963960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe048102220 0x7fe04819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.199+0000 7fe04cf77700 1 -- 172.21.15.45:0/2472963960 >> 172.21.15.45:0/2472963960 conn(0x7fe048078870 msgr2=0x7fe0481004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:42.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.199+0000 7fe04cf77700 1 -- 172.21.15.45:0/2472963960 shutdown_connections 2024-01-31T05:21:42.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.199+0000 7fe04cf77700 1 -- 172.21.15.45:0/2472963960 wait complete. 2024-01-31T05:21:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.410+0000 7f2291e53700 1 -- 172.21.15.45:0/465700017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c073a10 msgr2=0x7f228c073df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.410+0000 7f2291e53700 1 --2- 172.21.15.45:0/465700017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c073a10 0x7f228c073df0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f227c00b0a0 tx=0x7f227c00b3b0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.410+0000 7f2291e53700 1 -- 172.21.15.45:0/465700017 shutdown_connections 2024-01-31T05:21:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.410+0000 7f2291e53700 1 --2- 172.21.15.45:0/465700017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f228c10e210 0x7f228c110600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.410+0000 7f2291e53700 1 --2- 172.21.15.45:0/465700017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c073a10 0x7f228c073df0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.410+0000 7f2291e53700 1 --2- 172.21.15.45:0/465700017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f228c074330 0x7f228c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.410+0000 7f2291e53700 1 -- 172.21.15.45:0/465700017 >> 172.21.15.45:0/465700017 conn(0x7f228c0788a0 msgr2=0x7f228c078cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.410+0000 7f2291e53700 1 -- 172.21.15.45:0/465700017 shutdown_connections 2024-01-31T05:21:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.410+0000 7f2291e53700 1 -- 172.21.15.45:0/465700017 wait complete. 2024-01-31T05:21:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f2291e53700 1 Processor -- start 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f2291e53700 1 -- start start 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f2291e53700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f228c073a10 0x7f228c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f2291e53700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f228c074330 0x7f228c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f2291e53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c10e210 0x7f228c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f2291e53700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c19ef70 con 0x7f228c10e210 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f2291e53700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c19f0e0 con 0x7f228c073a10 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f2291e53700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c19f2a0 con 0x7f228c074330 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f228bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c10e210 0x7f228c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f228bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c10e210 0x7f228c19e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41372/0 (socket says 172.21.15.45:41372) 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f228bfff700 1 -- 172.21.15.45:0/3513094547 learned_addr learned my addr 172.21.15.45:0/3513094547 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f228b7fe700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f228c073a10 0x7f228c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f228affd700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f228c074330 0x7f228c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f228bfff700 1 -- 172.21.15.45:0/3513094547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f228c074330 msgr2=0x7f228c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f228bfff700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f228c074330 0x7f228c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f228bfff700 1 -- 172.21.15.45:0/3513094547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f228c073a10 msgr2=0x7f228c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f228bfff700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f228c073a10 0x7f228c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.411+0000 7f228bfff700 1 -- 172.21.15.45:0/3513094547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c009d00 con 0x7f228c10e210 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.412+0000 7f228bfff700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c10e210 0x7f228c19e960 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f228400b480 tx=0x7f228400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.412+0000 7f2288ff9700 1 -- 172.21.15.45:0/3513094547 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284011830 con 0x7f228c10e210 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.412+0000 7f2288ff9700 1 -- 172.21.15.45:0/3513094547 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2284011e50 con 0x7f228c10e210 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.412+0000 7f2291e53700 1 -- 172.21.15.45:0/3513094547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c19f580 con 0x7f228c10e210 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.412+0000 7f2288ff9700 1 -- 172.21.15.45:0/3513094547 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228400f2e0 con 0x7f228c10e210 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.412+0000 7f2291e53700 1 -- 172.21.15.45:0/3513094547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c1a6de0 con 0x7f228c10e210 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.413+0000 7f2291e53700 1 -- 172.21.15.45:0/3513094547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c109420 con 0x7f228c10e210 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.414+0000 7f2288ff9700 1 -- 172.21.15.45:0/3513094547 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f22840105a0 con 0x7f228c10e210 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.414+0000 7f2288ff9700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f22780a1400 0x7f22780a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.414+0000 7f228b7fe700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f22780a1400 0x7f22780a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.414+0000 7f2288ff9700 1 -- 172.21.15.45:0/3513094547 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f22840cc420 con 0x7f228c10e210 2024-01-31T05:21:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.415+0000 7f228b7fe700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f22780a1400 0x7f22780a38c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f227c009fd0 tx=0x7f227c00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.419+0000 7f2288ff9700 1 -- 172.21.15.45:0/3513094547 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f2284095a00 con 0x7f228c10e210 2024-01-31T05:21:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.624+0000 7f2291e53700 1 -- 172.21.15.45:0/3513094547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f228c1139f0 con 0x7f228c10e210 2024-01-31T05:21:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.624+0000 7f2288ff9700 1 -- 172.21.15.45:0/3513094547 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2284099020 con 0x7f228c10e210 2024-01-31T05:21:42.625 INFO:teuthology.orchestra.run.smithi045.stdout:146028888124 2024-01-31T05:21:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.626+0000 7f2291e53700 1 -- 172.21.15.45:0/3513094547 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f22780a1400 msgr2=0x7f22780a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.626+0000 7f2291e53700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f22780a1400 0x7f22780a38c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f227c009fd0 tx=0x7f227c00b7b0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.626+0000 7f2291e53700 1 -- 172.21.15.45:0/3513094547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c10e210 msgr2=0x7f228c19e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.626+0000 7f2291e53700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c10e210 0x7f228c19e960 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f228400b480 tx=0x7f228400b790 comp rx=0 tx=0).stop 2024-01-31T05:21:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.626+0000 7f2291e53700 1 -- 172.21.15.45:0/3513094547 shutdown_connections 2024-01-31T05:21:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.626+0000 7f2291e53700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f22780a1400 0x7f22780a38c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.626+0000 7f2291e53700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c10e210 0x7f228c19e960 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.626+0000 7f2291e53700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f228c073a10 0x7f228c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.626+0000 7f2291e53700 1 --2- 172.21.15.45:0/3513094547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f228c074330 0x7f228c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.626+0000 7f2291e53700 1 -- 172.21.15.45:0/3513094547 >> 172.21.15.45:0/3513094547 conn(0x7f228c0788a0 msgr2=0x7f228c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.627+0000 7f2291e53700 1 -- 172.21.15.45:0/3513094547 shutdown_connections 2024-01-31T05:21:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:42.627+0000 7f2291e53700 1 -- 172.21.15.45:0/3513094547 wait complete. 2024-01-31T05:21:42.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:42 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2472963960' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-01-31T05:21:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:42 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3513094547' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-01-31T05:21:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:42 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2472963960' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-01-31T05:21:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:42 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3513094547' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-01-31T05:21:43.022 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149753 got 111669149760 for osd.3 2024-01-31T05:21:43.022 DEBUG:teuthology.parallel:result is None 2024-01-31T05:21:43.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:42 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2472963960' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-01-31T05:21:43.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:42 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3513094547' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-01-31T05:21:43.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:43 smithi045 ceph-mon[31585]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:21:43.947 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462954 got 201863462961 for osd.6 2024-01-31T05:21:43.947 DEBUG:teuthology.parallel:result is None 2024-01-31T05:21:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:43 smithi156 ceph-mon[35720]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:21:44.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:43 smithi186 ceph-mon[35659]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:21:44.534 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612633 got 313532612636 for osd.10 2024-01-31T05:21:44.534 DEBUG:teuthology.parallel:result is None 2024-01-31T05:21:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.691+0000 7fd9ba52a700 1 -- 172.21.15.45:0/2448675655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b41068d0 msgr2=0x7fd9b4106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.691+0000 7fd9ba52a700 1 --2- 172.21.15.45:0/2448675655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b41068d0 0x7fd9b4106cb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0099c0 tx=0x7fd99c009cd0 comp rx=0 tx=0).stop 2024-01-31T05:21:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.691+0000 7fd9ba52a700 1 -- 172.21.15.45:0/2448675655 shutdown_connections 2024-01-31T05:21:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.691+0000 7fd9ba52a700 1 --2- 172.21.15.45:0/2448675655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9b4101380 0x7fd9b410d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.691+0000 7fd9ba52a700 1 --2- 172.21.15.45:0/2448675655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b41068d0 0x7fd9b4106cb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.691+0000 7fd9ba52a700 1 --2- 172.21.15.45:0/2448675655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b41009c0 0x7fd9b4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.691+0000 7fd9ba52a700 1 -- 172.21.15.45:0/2448675655 >> 172.21.15.45:0/2448675655 conn(0x7fd9b4077540 msgr2=0x7fd9b4077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.691+0000 7fd9ba52a700 1 -- 172.21.15.45:0/2448675655 shutdown_connections 2024-01-31T05:21:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.691+0000 7fd9ba52a700 1 -- 172.21.15.45:0/2448675655 wait complete. 2024-01-31T05:21:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.691+0000 7fd9ba52a700 1 Processor -- start 2024-01-31T05:21:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.691+0000 7fd9ba52a700 1 -- start start 2024-01-31T05:21:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9ba52a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b41009c0 0x7fd9b419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9ba52a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9b4101380 0x7fd9b419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9ba52a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b41068d0 0x7fd9b4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9ba52a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b41980e0 con 0x7fd9b41068d0 2024-01-31T05:21:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9ba52a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b4198220 con 0x7fd9b4101380 2024-01-31T05:21:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9ba52a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b41983e0 con 0x7fd9b41009c0 2024-01-31T05:21:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b8ac7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b41068d0 0x7fd9b4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b8ac7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b41068d0 0x7fd9b4197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41396/0 (socket says 172.21.15.45:41396) 2024-01-31T05:21:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b8ac7700 1 -- 172.21.15.45:0/706825352 learned_addr learned my addr 172.21.15.45:0/706825352 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b3fff700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b41009c0 0x7fd9b419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b37fe700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9b4101380 0x7fd9b419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b8ac7700 1 -- 172.21.15.45:0/706825352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b41009c0 msgr2=0x7fd9b419d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b8ac7700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b41009c0 0x7fd9b419d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b8ac7700 1 -- 172.21.15.45:0/706825352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9b4101380 msgr2=0x7fd9b419de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b8ac7700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9b4101380 0x7fd9b419de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b8ac7700 1 -- 172.21.15.45:0/706825352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c0095f0 con 0x7fd9b41068d0 2024-01-31T05:21:44.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b37fe700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9b4101380 0x7fd9b419de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:21:44.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b3fff700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b41009c0 0x7fd9b419d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:21:44.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.692+0000 7fd9b8ac7700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b41068d0 0x7fd9b4197a80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00b9e0 tx=0x7fd9ac00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:44.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.693+0000 7fd9b17fa700 1 -- 172.21.15.45:0/706825352 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00c950 con 0x7fd9b41068d0 2024-01-31T05:21:44.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.693+0000 7fd9ba52a700 1 -- 172.21.15.45:0/706825352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b41986c0 con 0x7fd9b41068d0 2024-01-31T05:21:44.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.693+0000 7fd9b17fa700 1 -- 172.21.15.45:0/706825352 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9ac014440 con 0x7fd9b41068d0 2024-01-31T05:21:44.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.693+0000 7fd9b17fa700 1 -- 172.21.15.45:0/706825352 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0075d0 con 0x7fd9b41068d0 2024-01-31T05:21:44.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.693+0000 7fd9ba52a700 1 -- 172.21.15.45:0/706825352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b41a4960 con 0x7fd9b41068d0 2024-01-31T05:21:44.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.693+0000 7fd9ba52a700 1 -- 172.21.15.45:0/706825352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b40684a0 con 0x7fd9b41068d0 2024-01-31T05:21:44.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.695+0000 7fd9b17fa700 1 -- 172.21.15.45:0/706825352 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fd9ac0145b0 con 0x7fd9b41068d0 2024-01-31T05:21:44.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.695+0000 7fd9b17fa700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd9a00a1400 0x7fd9a00a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:44.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.695+0000 7fd9b3fff700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd9a00a1400 0x7fd9a00a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:44.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.695+0000 7fd9b17fa700 1 -- 172.21.15.45:0/706825352 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd9ac006e40 con 0x7fd9b41068d0 2024-01-31T05:21:44.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.696+0000 7fd9b3fff700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd9a00a1400 0x7fd9a00a38c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd99c000c00 tx=0x7fd99c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:44.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.699+0000 7fd9b17fa700 1 -- 172.21.15.45:0/706825352 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd9ac095b50 con 0x7fd9b41068d0 2024-01-31T05:21:44.725 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.911+0000 7fd9ba52a700 1 -- 172.21.15.45:0/706825352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd9b4198fb0 con 0x7fd9b41068d0 2024-01-31T05:21:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.911+0000 7fd9b17fa700 1 -- 172.21.15.45:0/706825352 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd9ac099170 con 0x7fd9b41068d0 2024-01-31T05:21:44.910 INFO:teuthology.orchestra.run.smithi045.stdout:167503724600 2024-01-31T05:21:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.913+0000 7fd9ba52a700 1 -- 172.21.15.45:0/706825352 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd9a00a1400 msgr2=0x7fd9a00a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.913+0000 7fd9ba52a700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd9a00a1400 0x7fd9a00a38c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd99c000c00 tx=0x7fd99c005fb0 comp rx=0 tx=0).stop 2024-01-31T05:21:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.913+0000 7fd9ba52a700 1 -- 172.21.15.45:0/706825352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b41068d0 msgr2=0x7fd9b4197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.913+0000 7fd9ba52a700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b41068d0 0x7fd9b4197a80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00b9e0 tx=0x7fd9ac00bda0 comp rx=0 tx=0).stop 2024-01-31T05:21:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.913+0000 7fd9ba52a700 1 -- 172.21.15.45:0/706825352 shutdown_connections 2024-01-31T05:21:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.913+0000 7fd9ba52a700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd9a00a1400 0x7fd9a00a38c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.913+0000 7fd9ba52a700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b41068d0 0x7fd9b4197a80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:44.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.913+0000 7fd9ba52a700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b41009c0 0x7fd9b419d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:44.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.913+0000 7fd9ba52a700 1 --2- 172.21.15.45:0/706825352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9b4101380 0x7fd9b419de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:44.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.913+0000 7fd9ba52a700 1 -- 172.21.15.45:0/706825352 >> 172.21.15.45:0/706825352 conn(0x7fd9b4077540 msgr2=0x7fd9b40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:44.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.913+0000 7fd9ba52a700 1 -- 172.21.15.45:0/706825352 shutdown_connections 2024-01-31T05:21:44.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:44.913+0000 7fd9ba52a700 1 -- 172.21.15.45:0/706825352 wait complete. 2024-01-31T05:21:45.146 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738442 got 34359738446 for osd.0 2024-01-31T05:21:45.146 DEBUG:teuthology.parallel:result is None 2024-01-31T05:21:45.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:45 smithi045 ceph-mon[31585]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:21:45.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:45 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/706825352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-01-31T05:21:45.821 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724594 got 167503724600 for osd.5 2024-01-31T05:21:45.821 DEBUG:teuthology.parallel:result is None 2024-01-31T05:21:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:45 smithi156 ceph-mon[35720]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:21:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:45 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/706825352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-01-31T05:21:46.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:45 smithi186 ceph-mon[35659]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:21:46.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:45 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/706825352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-01-31T05:21:46.805 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-01-31T05:21:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-01-31T05:21:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-01-31T05:21:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-01-31T05:21:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-01-31T05:21:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-01-31T05:21:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:46 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-01-31T05:21:47.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-01-31T05:21:47.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-01-31T05:21:47.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-01-31T05:21:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-01-31T05:21:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-01-31T05:21:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-01-31T05:21:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:46 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-01-31T05:21:47.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-01-31T05:21:47.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-01-31T05:21:47.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-01-31T05:21:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-01-31T05:21:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-01-31T05:21:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-01-31T05:21:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:46 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-01-31T05:21:47.172 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888119 got 146028888124 for osd.4 2024-01-31T05:21:47.172 DEBUG:teuthology.parallel:result is None 2024-01-31T05:21:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:47 smithi156 ceph-mon[35720]: pgmap v316: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:21:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-01-31T05:21:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-01-31T05:21:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-01-31T05:21:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-01-31T05:21:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-01-31T05:21:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-01-31T05:21:47.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:47 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-01-31T05:21:47.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:47 smithi156 ceph-mon[35720]: osdmap e87: 12 total, 12 up, 12 in 2024-01-31T05:21:48.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:47 smithi186 ceph-mon[35659]: pgmap v316: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:21:48.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-01-31T05:21:48.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-01-31T05:21:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-01-31T05:21:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-01-31T05:21:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-01-31T05:21:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-01-31T05:21:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:47 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-01-31T05:21:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:47 smithi186 ceph-mon[35659]: osdmap e87: 12 total, 12 up, 12 in 2024-01-31T05:21:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:47 smithi045 ceph-mon[31585]: pgmap v316: 97 pgs: 97 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:21:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-01-31T05:21:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-01-31T05:21:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-01-31T05:21:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-01-31T05:21:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-01-31T05:21:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-01-31T05:21:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:47 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-01-31T05:21:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:47 smithi045 ceph-mon[31585]: osdmap e87: 12 total, 12 up, 12 in 2024-01-31T05:21:48.933 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:48 smithi045 ceph-mon[31585]: osdmap e88: 12 total, 12 up, 12 in 2024-01-31T05:21:48.934 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:48 smithi045 ceph-mon[31585]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-01-31T05:21:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:48 smithi156 ceph-mon[35720]: osdmap e88: 12 total, 12 up, 12 in 2024-01-31T05:21:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:48 smithi156 ceph-mon[35720]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-01-31T05:21:49.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:48 smithi186 ceph-mon[35659]: osdmap e88: 12 total, 12 up, 12 in 2024-01-31T05:21:49.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:48 smithi186 ceph-mon[35659]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-01-31T05:21:49.972 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:49 smithi045 ceph-mon[31585]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:21:49.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:49 smithi156 ceph-mon[35720]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:21:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.076+0000 7f1cea112700 1 -- 172.21.15.45:0/1975864174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce4101080 msgr2=0x7f1ce410d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.076+0000 7f1cea112700 1 --2- 172.21.15.45:0/1975864174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce4101080 0x7f1ce410d520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00b0a0 tx=0x7f1cdc00b3b0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.076+0000 7f1cea112700 1 -- 172.21.15.45:0/1975864174 shutdown_connections 2024-01-31T05:21:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.076+0000 7f1cea112700 1 --2- 172.21.15.45:0/1975864174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce4101080 0x7f1ce410d520 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.076+0000 7f1cea112700 1 --2- 172.21.15.45:0/1975864174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ce4105590 0x7f1ce4105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.076+0000 7f1cea112700 1 --2- 172.21.15.45:0/1975864174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce41006c0 0x7f1ce4100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.076+0000 7f1cea112700 1 -- 172.21.15.45:0/1975864174 >> 172.21.15.45:0/1975864174 conn(0x7f1ce40776f0 msgr2=0x7f1ce4077b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:50.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.076+0000 7f1cea112700 1 -- 172.21.15.45:0/1975864174 shutdown_connections 2024-01-31T05:21:50.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.076+0000 7f1cea112700 1 -- 172.21.15.45:0/1975864174 wait complete. 2024-01-31T05:21:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.076+0000 7f1cea112700 1 Processor -- start 2024-01-31T05:21:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1cea112700 1 -- start start 2024-01-31T05:21:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1cea112700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ce41006c0 0x7f1ce419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1cea112700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce4101080 0x7f1ce419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1cea112700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce4105590 0x7f1ce4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1cea112700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce41980e0 con 0x7f1ce4101080 2024-01-31T05:21:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1cea112700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce4198220 con 0x7f1ce4105590 2024-01-31T05:21:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1cea112700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce41983e0 con 0x7f1ce41006c0 2024-01-31T05:21:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1ce2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce4101080 0x7f1ce419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1ce2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce4101080 0x7f1ce419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44504/0 (socket says 172.21.15.45:44504) 2024-01-31T05:21:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1ce2ffd700 1 -- 172.21.15.45:0/2200991023 learned_addr learned my addr 172.21.15.45:0/2200991023 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1ce3fff700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce4105590 0x7f1ce4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1ce37fe700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ce41006c0 0x7f1ce419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1ce2ffd700 1 -- 172.21.15.45:0/2200991023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ce41006c0 msgr2=0x7f1ce419d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1ce2ffd700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ce41006c0 0x7f1ce419d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1ce2ffd700 1 -- 172.21.15.45:0/2200991023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce4105590 msgr2=0x7f1ce4197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1ce2ffd700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce4105590 0x7f1ce4197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1ce2ffd700 1 -- 172.21.15.45:0/2200991023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc009d00 con 0x7f1ce4101080 2024-01-31T05:21:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.077+0000 7f1ce2ffd700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce4101080 0x7f1ce419de00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800c9b0 tx=0x7f1cd800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.078+0000 7f1ce0ff9700 1 -- 172.21.15.45:0/2200991023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8007b00 con 0x7f1ce4101080 2024-01-31T05:21:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.078+0000 7f1ce0ff9700 1 -- 172.21.15.45:0/2200991023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cd8007c60 con 0x7f1ce4101080 2024-01-31T05:21:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.078+0000 7f1cea112700 1 -- 172.21.15.45:0/2200991023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce41986c0 con 0x7f1ce4101080 2024-01-31T05:21:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.078+0000 7f1ce0ff9700 1 -- 172.21.15.45:0/2200991023 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8018660 con 0x7f1ce4101080 2024-01-31T05:21:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.078+0000 7f1cea112700 1 -- 172.21.15.45:0/2200991023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce41a4960 con 0x7f1ce4101080 2024-01-31T05:21:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.079+0000 7f1cea112700 1 -- 172.21.15.45:0/2200991023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce4108be0 con 0x7f1ce4101080 2024-01-31T05:21:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.080+0000 7f1ce0ff9700 1 -- 172.21.15.45:0/2200991023 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f1cd8018880 con 0x7f1ce4101080 2024-01-31T05:21:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.080+0000 7f1ce0ff9700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ccc0a13f0 0x7f1ccc0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.080+0000 7f1ce37fe700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ccc0a13f0 0x7f1ccc0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.080+0000 7f1ce0ff9700 1 -- 172.21.15.45:0/2200991023 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1cd80ccb20 con 0x7f1ce4101080 2024-01-31T05:21:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.081+0000 7f1ce37fe700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ccc0a13f0 0x7f1ccc0a38b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4005fd0 tx=0x7f1cd4005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.085+0000 7f1ce0ff9700 1 -- 172.21.15.45:0/2200991023 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1cd8095fd0 con 0x7f1ce4101080 2024-01-31T05:21:50.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:49 smithi186 ceph-mon[35659]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:21:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.287+0000 7f1cea112700 1 -- 172.21.15.45:0/2200991023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1ce4198fb0 con 0x7f1ce4101080 2024-01-31T05:21:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.287+0000 7f1ce0ff9700 1 -- 172.21.15.45:0/2200991023 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1cd80995f0 con 0x7f1ce4101080 2024-01-31T05:21:50.287 INFO:teuthology.orchestra.run.smithi045.stdout:223338299439 2024-01-31T05:21:50.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.289+0000 7f1cea112700 1 -- 172.21.15.45:0/2200991023 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ccc0a13f0 msgr2=0x7f1ccc0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:50.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.289+0000 7f1cea112700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ccc0a13f0 0x7f1ccc0a38b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4005fd0 tx=0x7f1cd4005db0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.289+0000 7f1cea112700 1 -- 172.21.15.45:0/2200991023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce4101080 msgr2=0x7f1ce419de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:50.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.289+0000 7f1cea112700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce4101080 0x7f1ce419de00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800c9b0 tx=0x7f1cd800cd70 comp rx=0 tx=0).stop 2024-01-31T05:21:50.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.289+0000 7f1cea112700 1 -- 172.21.15.45:0/2200991023 shutdown_connections 2024-01-31T05:21:50.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.289+0000 7f1cea112700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce4105590 0x7f1ce4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.289+0000 7f1cea112700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ce41006c0 0x7f1ce419d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.289+0000 7f1cea112700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ccc0a13f0 0x7f1ccc0a38b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.289+0000 7f1cea112700 1 --2- 172.21.15.45:0/2200991023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce4101080 0x7f1ce419de00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.289+0000 7f1cea112700 1 -- 172.21.15.45:0/2200991023 >> 172.21.15.45:0/2200991023 conn(0x7f1ce40776f0 msgr2=0x7f1ce40fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:50.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.290+0000 7f1cea112700 1 -- 172.21.15.45:0/2200991023 shutdown_connections 2024-01-31T05:21:50.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.290+0000 7f1cea112700 1 -- 172.21.15.45:0/2200991023 wait complete. 2024-01-31T05:21:50.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.302+0000 7f87723c9700 1 -- 172.21.15.45:0/2488492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f876c1067b0 msgr2=0x7f876c106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:50.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.302+0000 7f87723c9700 1 --2- 172.21.15.45:0/2488492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f876c1067b0 0x7f876c106b90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f875c009990 tx=0x7f875c009ca0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.302+0000 7f87723c9700 1 -- 172.21.15.45:0/2488492 shutdown_connections 2024-01-31T05:21:50.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.302+0000 7f87723c9700 1 --2- 172.21.15.45:0/2488492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f876c1018a0 0x7f876c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.302+0000 7f87723c9700 1 --2- 172.21.15.45:0/2488492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f876c1067b0 0x7f876c106b90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.302+0000 7f87723c9700 1 --2- 172.21.15.45:0/2488492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876c100ee0 0x7f876c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.302+0000 7f87723c9700 1 -- 172.21.15.45:0/2488492 >> 172.21.15.45:0/2488492 conn(0x7f876c076ce0 msgr2=0x7f876c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:50.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.302+0000 7f87723c9700 1 -- 172.21.15.45:0/2488492 shutdown_connections 2024-01-31T05:21:50.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.302+0000 7f87723c9700 1 -- 172.21.15.45:0/2488492 wait complete. 2024-01-31T05:21:50.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.302+0000 7f87723c9700 1 Processor -- start 2024-01-31T05:21:50.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.302+0000 7f87723c9700 1 -- start start 2024-01-31T05:21:50.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f87723c9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f876c100ee0 0x7f876c19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:50.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f87723c9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876c1018a0 0x7f876c19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:50.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f87723c9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f876c1067b0 0x7f876c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:50.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f87723c9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c198130 con 0x7f876c100ee0 2024-01-31T05:21:50.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f87723c9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c198270 con 0x7f876c1018a0 2024-01-31T05:21:50.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f87723c9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c198430 con 0x7f876c1067b0 2024-01-31T05:21:50.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f876bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f876c100ee0 0x7f876c19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:50.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f876bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f876c100ee0 0x7f876c19d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44526/0 (socket says 172.21.15.45:44526) 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f876bfff700 1 -- 172.21.15.45:0/2097169340 learned_addr learned my addr 172.21.15.45:0/2097169340 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f876b7fe700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876c1018a0 0x7f876c19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f8770966700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f876c1067b0 0x7f876c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f876bfff700 1 -- 172.21.15.45:0/2097169340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f876c1067b0 msgr2=0x7f876c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f876bfff700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f876c1067b0 0x7f876c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f876bfff700 1 -- 172.21.15.45:0/2097169340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876c1018a0 msgr2=0x7f876c19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f876bfff700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876c1018a0 0x7f876c19de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.303+0000 7f876bfff700 1 -- 172.21.15.45:0/2097169340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c0095f0 con 0x7f876c100ee0 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.304+0000 7f876bfff700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f876c100ee0 0x7f876c19d8c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f875c003730 tx=0x7f875c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.304+0000 7f87697fa700 1 -- 172.21.15.45:0/2097169340 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c01d070 con 0x7f876c100ee0 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.304+0000 7f87697fa700 1 -- 172.21.15.45:0/2097169340 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f875c005030 con 0x7f876c100ee0 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.304+0000 7f87723c9700 1 -- 172.21.15.45:0/2097169340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876c1986b0 con 0x7f876c100ee0 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.304+0000 7f87697fa700 1 -- 172.21.15.45:0/2097169340 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c004a40 con 0x7f876c100ee0 2024-01-31T05:21:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.304+0000 7f87723c9700 1 -- 172.21.15.45:0/2097169340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876c1a4960 con 0x7f876c100ee0 2024-01-31T05:21:50.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.305+0000 7f87723c9700 1 -- 172.21.15.45:0/2097169340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876c101ef0 con 0x7f876c100ee0 2024-01-31T05:21:50.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.306+0000 7f87697fa700 1 -- 172.21.15.45:0/2097169340 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f875c004ba0 con 0x7f876c100ee0 2024-01-31T05:21:50.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.307+0000 7f87697fa700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f87580a1370 0x7f87580a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:50.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.307+0000 7f876b7fe700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f87580a1370 0x7f87580a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:50.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.307+0000 7f87697fa700 1 -- 172.21.15.45:0/2097169340 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f875c0cdd80 con 0x7f876c100ee0 2024-01-31T05:21:50.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.307+0000 7f876b7fe700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f87580a1370 0x7f87580a3830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f876c105dc0 tx=0x7f875400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:50.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.311+0000 7f87697fa700 1 -- 172.21.15.45:0/2097169340 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f875c097260 con 0x7f876c100ee0 2024-01-31T05:21:50.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.514+0000 7f87723c9700 1 -- 172.21.15.45:0/2097169340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f876c0684a0 con 0x7f876c100ee0 2024-01-31T05:21:50.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.515+0000 7f87697fa700 1 -- 172.21.15.45:0/2097169340 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f875c09a880 con 0x7f876c100ee0 2024-01-31T05:21:50.514 INFO:teuthology.orchestra.run.smithi045.stdout:347892351003 2024-01-31T05:21:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.517+0000 7f87723c9700 1 -- 172.21.15.45:0/2097169340 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f87580a1370 msgr2=0x7f87580a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.517+0000 7f87723c9700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f87580a1370 0x7f87580a3830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f876c105dc0 tx=0x7f875400b410 comp rx=0 tx=0).stop 2024-01-31T05:21:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.517+0000 7f87723c9700 1 -- 172.21.15.45:0/2097169340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f876c100ee0 msgr2=0x7f876c19d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.517+0000 7f87723c9700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f876c100ee0 0x7f876c19d8c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f875c003730 tx=0x7f875c00f690 comp rx=0 tx=0).stop 2024-01-31T05:21:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.517+0000 7f87723c9700 1 -- 172.21.15.45:0/2097169340 shutdown_connections 2024-01-31T05:21:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.517+0000 7f87723c9700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f876c1067b0 0x7f876c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.517+0000 7f87723c9700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f87580a1370 0x7f87580a3830 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.517+0000 7f87723c9700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f876c100ee0 0x7f876c19d8c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.517+0000 7f87723c9700 1 --2- 172.21.15.45:0/2097169340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876c1018a0 0x7f876c19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.517+0000 7f87723c9700 1 -- 172.21.15.45:0/2097169340 >> 172.21.15.45:0/2097169340 conn(0x7f876c076ce0 msgr2=0x7f876c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.517+0000 7f87723c9700 1 -- 172.21.15.45:0/2097169340 shutdown_connections 2024-01-31T05:21:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:50.517+0000 7f87723c9700 1 -- 172.21.15.45:0/2097169340 wait complete. 2024-01-31T05:21:50.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:50 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2200991023' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-01-31T05:21:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:50 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2097169340' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-01-31T05:21:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:50 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2200991023' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-01-31T05:21:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:50 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2097169340' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-01-31T05:21:51.011 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299434 got 223338299439 for osd.7 2024-01-31T05:21:51.012 DEBUG:teuthology.parallel:result is None 2024-01-31T05:21:51.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:50 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2200991023' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-01-31T05:21:51.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:50 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2097169340' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-01-31T05:21:51.701 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350998 got 347892351003 for osd.11 2024-01-31T05:21:51.701 DEBUG:teuthology.parallel:result is None 2024-01-31T05:21:51.702 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-01-31T05:21:51.702 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph pg dump --format=json 2024-01-31T05:21:51.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:51 smithi156 ceph-mon[35720]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-01-31T05:21:51.977 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:52.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:51 smithi186 ceph-mon[35659]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-01-31T05:21:52.100 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:51 smithi045 ceph-mon[31585]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-01-31T05:21:53.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:53.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:53.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:53 smithi186 ceph-mon[35659]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-01-31T05:21:53.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:21:53.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:21:53.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:53 smithi186 ceph-mon[35659]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:53.357 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:53.357 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:53.357 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:53 smithi045 ceph-mon[31585]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-01-31T05:21:53.358 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:21:53.358 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:21:53.358 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:53 smithi045 ceph-mon[31585]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:53.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:53.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:53.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:53 smithi156 ceph-mon[35720]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-01-31T05:21:53.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:21:53.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:21:53.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:53 smithi156 ceph-mon[35720]: from='mgr.14152 172.21.15.45:0/1224428565' entity='mgr.x' 2024-01-31T05:21:53.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.579+0000 7fb85fca7700 1 -- 172.21.15.45:0/4126016671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb858100790 msgr2=0x7fb85810e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:53.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.579+0000 7fb85fca7700 1 --2- 172.21.15.45:0/4126016671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb858100790 0x7fb85810e6f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb85000b0a0 tx=0x7fb85000b3b0 comp rx=0 tx=0).stop 2024-01-31T05:21:53.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.579+0000 7fb85fca7700 1 -- 172.21.15.45:0/4126016671 shutdown_connections 2024-01-31T05:21:53.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.579+0000 7fb85fca7700 1 --2- 172.21.15.45:0/4126016671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb858100790 0x7fb85810e6f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.579+0000 7fb85fca7700 1 --2- 172.21.15.45:0/4126016671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb858106850 0x7fb858106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.579+0000 7fb85fca7700 1 --2- 172.21.15.45:0/4126016671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8580ffdd0 0x7fb858100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.579+0000 7fb85fca7700 1 -- 172.21.15.45:0/4126016671 >> 172.21.15.45:0/4126016671 conn(0x7fb858076ce0 msgr2=0x7fb8580fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.579+0000 7fb85fca7700 1 -- 172.21.15.45:0/4126016671 shutdown_connections 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.579+0000 7fb85fca7700 1 -- 172.21.15.45:0/4126016671 wait complete. 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85fca7700 1 Processor -- start 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85fca7700 1 -- start start 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85fca7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8580ffdd0 0x7fb858197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85fca7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb858100790 0x7fb858198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85fca7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb858106850 0x7fb85819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85fca7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85819cd00 con 0x7fb858100790 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85fca7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85819ce70 con 0x7fb858106850 2024-01-31T05:21:53.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85fca7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85819d030 con 0x7fb8580ffdd0 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85d242700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb858100790 0x7fb858198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85d242700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb858100790 0x7fb858198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44542/0 (socket says 172.21.15.45:44542) 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85d242700 1 -- 172.21.15.45:0/3878822431 learned_addr learned my addr 172.21.15.45:0/3878822431 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85e244700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb858106850 0x7fb85819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85da43700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8580ffdd0 0x7fb858197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85d242700 1 -- 172.21.15.45:0/3878822431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8580ffdd0 msgr2=0x7fb858197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85d242700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8580ffdd0 0x7fb858197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85d242700 1 -- 172.21.15.45:0/3878822431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb858106850 msgr2=0x7fb85819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85d242700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb858106850 0x7fb85819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85d242700 1 -- 172.21.15.45:0/3878822431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb850009d00 con 0x7fb858100790 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.580+0000 7fb85da43700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8580ffdd0 0x7fb858197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.581+0000 7fb85d242700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb858100790 0x7fb858198360 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb85400e9a0 tx=0x7fb85400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:53.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.581+0000 7fb84effd700 1 -- 172.21.15.45:0/3878822431 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854009790 con 0x7fb858100790 2024-01-31T05:21:53.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.581+0000 7fb85fca7700 1 -- 172.21.15.45:0/3878822431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85819d310 con 0x7fb858100790 2024-01-31T05:21:53.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.581+0000 7fb84effd700 1 -- 172.21.15.45:0/3878822431 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb85400cbd0 con 0x7fb858100790 2024-01-31T05:21:53.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.581+0000 7fb84effd700 1 -- 172.21.15.45:0/3878822431 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854018680 con 0x7fb858100790 2024-01-31T05:21:53.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.581+0000 7fb85fca7700 1 -- 172.21.15.45:0/3878822431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8581a4b70 con 0x7fb858100790 2024-01-31T05:21:53.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.582+0000 7fb85fca7700 1 -- 172.21.15.45:0/3878822431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb858109e40 con 0x7fb858100790 2024-01-31T05:21:53.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.583+0000 7fb84effd700 1 -- 172.21.15.45:0/3878822431 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fb85400cd40 con 0x7fb858100790 2024-01-31T05:21:53.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.583+0000 7fb84effd700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8440a1370 0x7fb8440a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:53.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.584+0000 7fb85da43700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8440a1370 0x7fb8440a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:53.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.584+0000 7fb84effd700 1 -- 172.21.15.45:0/3878822431 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb854014070 con 0x7fb858100790 2024-01-31T05:21:53.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.584+0000 7fb85da43700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8440a1370 0x7fb8440a3830 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb848009cb0 tx=0x7fb848009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:53.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.587+0000 7fb84effd700 1 -- 172.21.15.45:0/3878822431 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb854096520 con 0x7fb858100790 2024-01-31T05:21:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.785+0000 7fb85fca7700 1 -- 172.21.15.45:0/3878822431 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb858199180 con 0x7fb8440a1370 2024-01-31T05:21:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.796+0000 7fb84effd700 1 -- 172.21.15.45:0/3878822431 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344789 (secure 0 0 0) 0x7fb858199180 con 0x7fb8440a1370 2024-01-31T05:21:53.800 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:21:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.799+0000 7fb85fca7700 1 -- 172.21.15.45:0/3878822431 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8440a1370 msgr2=0x7fb8440a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.799+0000 7fb85fca7700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8440a1370 0x7fb8440a3830 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb848009cb0 tx=0x7fb848009260 comp rx=0 tx=0).stop 2024-01-31T05:21:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.799+0000 7fb85fca7700 1 -- 172.21.15.45:0/3878822431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb858100790 msgr2=0x7fb858198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.799+0000 7fb85fca7700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb858100790 0x7fb858198360 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb85400e9a0 tx=0x7fb85400ecb0 comp rx=0 tx=0).stop 2024-01-31T05:21:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.799+0000 7fb85fca7700 1 -- 172.21.15.45:0/3878822431 shutdown_connections 2024-01-31T05:21:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.799+0000 7fb85fca7700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb858106850 0x7fb85819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.799+0000 7fb85fca7700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8580ffdd0 0x7fb858197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.799+0000 7fb85fca7700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb8440a1370 0x7fb8440a3830 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.799+0000 7fb85fca7700 1 --2- 172.21.15.45:0/3878822431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb858100790 0x7fb858198360 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.799+0000 7fb85fca7700 1 -- 172.21.15.45:0/3878822431 >> 172.21.15.45:0/3878822431 conn(0x7fb858076ce0 msgr2=0x7fb8581048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:53.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.799+0000 7fb85fca7700 1 -- 172.21.15.45:0/3878822431 shutdown_connections 2024-01-31T05:21:53.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:53.799+0000 7fb85fca7700 1 -- 172.21.15.45:0/3878822431 wait complete. 2024-01-31T05:21:53.805 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-01-31T05:21:54.521 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-01-31T05:21:52.608377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653372,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":563,"num_read_kb":400,"num_write":339,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":773,"ondisk_log_size":773,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":295,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404060,"kb_used_data":13248,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537764,"statfs":{"total":1151940427776,"available":1151526670336,"internally_reserved":0,"allocated":13565952,"data_stored":7452242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,14,6,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":31,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002230"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689535+0000","last_change":"2024-01-31T05:20:40.720465+0000","last_active":"2024-01-31T05:21:47.689535+0000","last_peered":"2024-01-31T05:21:47.689535+0000","last_clean":"2024-01-31T05:21:47.689535+0000","last_became_active":"2024-01-31T05:20:40.717615+0000","last_became_peered":"2024-01-31T05:20:40.717615+0000","last_unstale":"2024-01-31T05:21:47.689535+0000","last_undegraded":"2024-01-31T05:21:47.689535+0000","last_fullsized":"2024-01-31T05:21:47.689535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:30:01.564811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691731+0000","last_change":"2024-01-31T05:20:40.729011+0000","last_active":"2024-01-31T05:21:47.691731+0000","last_peered":"2024-01-31T05:21:47.691731+0000","last_clean":"2024-01-31T05:21:47.691731+0000","last_became_active":"2024-01-31T05:20:40.728021+0000","last_became_peered":"2024-01-31T05:20:40.728021+0000","last_unstale":"2024-01-31T05:21:47.691731+0000","last_undegraded":"2024-01-31T05:21:47.691731+0000","last_fullsized":"2024-01-31T05:21:47.691731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:23:06.641571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691590+0000","last_change":"2024-01-31T05:20:40.729136+0000","last_active":"2024-01-31T05:21:47.691590+0000","last_peered":"2024-01-31T05:21:47.691590+0000","last_clean":"2024-01-31T05:21:47.691590+0000","last_became_active":"2024-01-31T05:20:40.728273+0000","last_became_peered":"2024-01-31T05:20:40.728273+0000","last_unstale":"2024-01-31T05:21:47.691590+0000","last_undegraded":"2024-01-31T05:21:47.691590+0000","last_fullsized":"2024-01-31T05:21:47.691590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:17:43.625443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692995+0000","last_change":"2024-01-31T05:21:47.692585+0000","last_active":"2024-01-31T05:21:47.692995+0000","last_peered":"2024-01-31T05:21:47.692995+0000","last_clean":"2024-01-31T05:21:47.692995+0000","last_became_active":"2024-01-31T05:21:47.692150+0000","last_became_peered":"2024-01-31T05:21:47.692150+0000","last_unstale":"2024-01-31T05:21:47.692995+0000","last_undegraded":"2024-01-31T05:21:47.692995+0000","last_fullsized":"2024-01-31T05:21:47.692995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:56:06.261960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688566+0000","last_change":"2024-01-31T05:20:40.724551+0000","last_active":"2024-01-31T05:21:47.688566+0000","last_peered":"2024-01-31T05:21:47.688566+0000","last_clean":"2024-01-31T05:21:47.688566+0000","last_became_active":"2024-01-31T05:20:40.724172+0000","last_became_peered":"2024-01-31T05:20:40.724172+0000","last_unstale":"2024-01-31T05:21:47.688566+0000","last_undegraded":"2024-01-31T05:21:47.688566+0000","last_fullsized":"2024-01-31T05:21:47.688566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:45:28.748481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895954+0000","last_change":"2024-01-31T05:20:40.721982+0000","last_active":"2024-01-31T05:21:47.895954+0000","last_peered":"2024-01-31T05:21:47.895954+0000","last_clean":"2024-01-31T05:21:47.895954+0000","last_became_active":"2024-01-31T05:20:40.721649+0000","last_became_peered":"2024-01-31T05:20:40.721649+0000","last_unstale":"2024-01-31T05:21:47.895954+0000","last_undegraded":"2024-01-31T05:21:47.895954+0000","last_fullsized":"2024-01-31T05:21:47.895954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:47:32.776711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686428+0000","last_change":"2024-01-31T05:20:40.718340+0000","last_active":"2024-01-31T05:21:47.686428+0000","last_peered":"2024-01-31T05:21:47.686428+0000","last_clean":"2024-01-31T05:21:47.686428+0000","last_became_active":"2024-01-31T05:20:40.718017+0000","last_became_peered":"2024-01-31T05:20:40.718017+0000","last_unstale":"2024-01-31T05:21:47.686428+0000","last_undegraded":"2024-01-31T05:21:47.686428+0000","last_fullsized":"2024-01-31T05:21:47.686428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:17:45.606831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695366+0000","last_change":"2024-01-31T05:20:40.720271+0000","last_active":"2024-01-31T05:21:47.695366+0000","last_peered":"2024-01-31T05:21:47.695366+0000","last_clean":"2024-01-31T05:21:47.695366+0000","last_became_active":"2024-01-31T05:20:40.719960+0000","last_became_peered":"2024-01-31T05:20:40.719960+0000","last_unstale":"2024-01-31T05:21:47.695366+0000","last_undegraded":"2024-01-31T05:21:47.695366+0000","last_fullsized":"2024-01-31T05:21:47.695366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:59:16.243268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689929+0000","last_change":"2024-01-31T05:20:40.725779+0000","last_active":"2024-01-31T05:21:47.689929+0000","last_peered":"2024-01-31T05:21:47.689929+0000","last_clean":"2024-01-31T05:21:47.689929+0000","last_became_active":"2024-01-31T05:20:40.725423+0000","last_became_peered":"2024-01-31T05:20:40.725423+0000","last_unstale":"2024-01-31T05:21:47.689929+0000","last_undegraded":"2024-01-31T05:21:47.689929+0000","last_fullsized":"2024-01-31T05:21:47.689929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:11:46.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.896186+0000","last_change":"2024-01-31T05:20:40.723144+0000","last_active":"2024-01-31T05:21:47.896186+0000","last_peered":"2024-01-31T05:21:47.896186+0000","last_clean":"2024-01-31T05:21:47.896186+0000","last_became_active":"2024-01-31T05:20:40.722806+0000","last_became_peered":"2024-01-31T05:20:40.722806+0000","last_unstale":"2024-01-31T05:21:47.896186+0000","last_undegraded":"2024-01-31T05:21:47.896186+0000","last_fullsized":"2024-01-31T05:21:47.896186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:33.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686816+0000","last_change":"2024-01-31T05:20:40.721083+0000","last_active":"2024-01-31T05:21:47.686816+0000","last_peered":"2024-01-31T05:21:47.686816+0000","last_clean":"2024-01-31T05:21:47.686816+0000","last_became_active":"2024-01-31T05:20:40.720773+0000","last_became_peered":"2024-01-31T05:20:40.720773+0000","last_unstale":"2024-01-31T05:21:47.686816+0000","last_undegraded":"2024-01-31T05:21:47.686816+0000","last_fullsized":"2024-01-31T05:21:47.686816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:39:08.114126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692642+0000","last_change":"2024-01-31T05:20:40.723630+0000","last_active":"2024-01-31T05:21:47.692642+0000","last_peered":"2024-01-31T05:21:47.692642+0000","last_clean":"2024-01-31T05:21:47.692642+0000","last_became_active":"2024-01-31T05:20:40.723340+0000","last_became_peered":"2024-01-31T05:20:40.723340+0000","last_unstale":"2024-01-31T05:21:47.692642+0000","last_undegraded":"2024-01-31T05:21:47.692642+0000","last_fullsized":"2024-01-31T05:21:47.692642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:53:21.199111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895922+0000","last_change":"2024-01-31T05:20:40.712602+0000","last_active":"2024-01-31T05:21:47.895922+0000","last_peered":"2024-01-31T05:21:47.895922+0000","last_clean":"2024-01-31T05:21:47.895922+0000","last_became_active":"2024-01-31T05:20:40.712261+0000","last_became_peered":"2024-01-31T05:20:40.712261+0000","last_unstale":"2024-01-31T05:21:47.895922+0000","last_undegraded":"2024-01-31T05:21:47.895922+0000","last_fullsized":"2024-01-31T05:21:47.895922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:30:05.638679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691375+0000","last_change":"2024-01-31T05:20:40.723759+0000","last_active":"2024-01-31T05:21:47.691375+0000","last_peered":"2024-01-31T05:21:47.691375+0000","last_clean":"2024-01-31T05:21:47.691375+0000","last_became_active":"2024-01-31T05:20:40.723421+0000","last_became_peered":"2024-01-31T05:20:40.723421+0000","last_unstale":"2024-01-31T05:21:47.691375+0000","last_undegraded":"2024-01-31T05:21:47.691375+0000","last_fullsized":"2024-01-31T05:21:47.691375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:02:27.348273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691456+0000","last_change":"2024-01-31T05:20:40.726482+0000","last_active":"2024-01-31T05:21:47.691456+0000","last_peered":"2024-01-31T05:21:47.691456+0000","last_clean":"2024-01-31T05:21:47.691456+0000","last_became_active":"2024-01-31T05:20:40.726086+0000","last_became_peered":"2024-01-31T05:20:40.726086+0000","last_unstale":"2024-01-31T05:21:47.691456+0000","last_undegraded":"2024-01-31T05:21:47.691456+0000","last_fullsized":"2024-01-31T05:21:47.691456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:02:28.142740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689383+0000","last_change":"2024-01-31T05:20:40.728363+0000","last_active":"2024-01-31T05:21:47.689383+0000","last_peered":"2024-01-31T05:21:47.689383+0000","last_clean":"2024-01-31T05:21:47.689383+0000","last_became_active":"2024-01-31T05:20:40.727795+0000","last_became_peered":"2024-01-31T05:20:40.727795+0000","last_unstale":"2024-01-31T05:21:47.689383+0000","last_undegraded":"2024-01-31T05:21:47.689383+0000","last_fullsized":"2024-01-31T05:21:47.689383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:43:08.123159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689646+0000","last_change":"2024-01-31T05:20:40.720902+0000","last_active":"2024-01-31T05:21:47.689646+0000","last_peered":"2024-01-31T05:21:47.689646+0000","last_clean":"2024-01-31T05:21:47.689646+0000","last_became_active":"2024-01-31T05:20:40.720554+0000","last_became_peered":"2024-01-31T05:20:40.720554+0000","last_unstale":"2024-01-31T05:21:47.689646+0000","last_undegraded":"2024-01-31T05:21:47.689646+0000","last_fullsized":"2024-01-31T05:21:47.689646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:56:49.441368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.685578+0000","last_change":"2024-01-31T05:20:40.710504+0000","last_active":"2024-01-31T05:21:47.685578+0000","last_peered":"2024-01-31T05:21:47.685578+0000","last_clean":"2024-01-31T05:21:47.685578+0000","last_became_active":"2024-01-31T05:20:40.710162+0000","last_became_peered":"2024-01-31T05:20:40.710162+0000","last_unstale":"2024-01-31T05:21:47.685578+0000","last_undegraded":"2024-01-31T05:21:47.685578+0000","last_fullsized":"2024-01-31T05:21:47.685578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:49:49.930573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688084+0000","last_change":"2024-01-31T05:20:40.729023+0000","last_active":"2024-01-31T05:21:47.688084+0000","last_peered":"2024-01-31T05:21:47.688084+0000","last_clean":"2024-01-31T05:21:47.688084+0000","last_became_active":"2024-01-31T05:20:40.728672+0000","last_became_peered":"2024-01-31T05:20:40.728672+0000","last_unstale":"2024-01-31T05:21:47.688084+0000","last_undegraded":"2024-01-31T05:21:47.688084+0000","last_fullsized":"2024-01-31T05:21:47.688084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:56:41.578440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.685542+0000","last_change":"2024-01-31T05:20:39.716771+0000","last_active":"2024-01-31T05:21:47.685542+0000","last_peered":"2024-01-31T05:21:47.685542+0000","last_clean":"2024-01-31T05:21:47.685542+0000","last_became_active":"2024-01-31T05:20:39.716439+0000","last_became_peered":"2024-01-31T05:20:39.716439+0000","last_unstale":"2024-01-31T05:21:47.685542+0000","last_undegraded":"2024-01-31T05:21:47.685542+0000","last_fullsized":"2024-01-31T05:21:47.685542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:14:24.317293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695802+0000","last_change":"2024-01-31T05:20:40.730695+0000","last_active":"2024-01-31T05:21:47.695802+0000","last_peered":"2024-01-31T05:21:47.695802+0000","last_clean":"2024-01-31T05:21:47.695802+0000","last_became_active":"2024-01-31T05:20:40.730343+0000","last_became_peered":"2024-01-31T05:20:40.730343+0000","last_unstale":"2024-01-31T05:21:47.695802+0000","last_undegraded":"2024-01-31T05:21:47.695802+0000","last_fullsized":"2024-01-31T05:21:47.695802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:32.308239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695968+0000","last_change":"2024-01-31T05:20:39.711082+0000","last_active":"2024-01-31T05:21:47.695968+0000","last_peered":"2024-01-31T05:21:47.695968+0000","last_clean":"2024-01-31T05:21:47.695968+0000","last_became_active":"2024-01-31T05:20:39.710741+0000","last_became_peered":"2024-01-31T05:20:39.710741+0000","last_unstale":"2024-01-31T05:21:47.695968+0000","last_undegraded":"2024-01-31T05:21:47.695968+0000","last_fullsized":"2024-01-31T05:21:47.695968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:20:01.934837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691726+0000","last_change":"2024-01-31T05:20:40.729269+0000","last_active":"2024-01-31T05:21:47.691726+0000","last_peered":"2024-01-31T05:21:47.691726+0000","last_clean":"2024-01-31T05:21:47.691726+0000","last_became_active":"2024-01-31T05:20:40.728910+0000","last_became_peered":"2024-01-31T05:20:40.728910+0000","last_unstale":"2024-01-31T05:21:47.691726+0000","last_undegraded":"2024-01-31T05:21:47.691726+0000","last_fullsized":"2024-01-31T05:21:47.691726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:00:24.711108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.703286+0000","last_change":"2024-01-31T05:21:47.702095+0000","last_active":"2024-01-31T05:21:47.703286+0000","last_peered":"2024-01-31T05:21:47.703286+0000","last_clean":"2024-01-31T05:21:47.703286+0000","last_became_active":"2024-01-31T05:21:47.692164+0000","last_became_peered":"2024-01-31T05:21:47.692164+0000","last_unstale":"2024-01-31T05:21:47.703286+0000","last_undegraded":"2024-01-31T05:21:47.703286+0000","last_fullsized":"2024-01-31T05:21:47.703286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:31.642391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689477+0000","last_change":"2024-01-31T05:20:40.727934+0000","last_active":"2024-01-31T05:21:47.689477+0000","last_peered":"2024-01-31T05:21:47.689477+0000","last_clean":"2024-01-31T05:21:47.689477+0000","last_became_active":"2024-01-31T05:20:40.727517+0000","last_became_peered":"2024-01-31T05:20:40.727517+0000","last_unstale":"2024-01-31T05:21:47.689477+0000","last_undegraded":"2024-01-31T05:21:47.689477+0000","last_fullsized":"2024-01-31T05:21:47.689477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:35:46.329596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691357+0000","last_change":"2024-01-31T05:20:39.709541+0000","last_active":"2024-01-31T05:21:47.691357+0000","last_peered":"2024-01-31T05:21:47.691357+0000","last_clean":"2024-01-31T05:21:47.691357+0000","last_became_active":"2024-01-31T05:20:39.708918+0000","last_became_peered":"2024-01-31T05:20:39.708918+0000","last_unstale":"2024-01-31T05:21:47.691357+0000","last_undegraded":"2024-01-31T05:21:47.691357+0000","last_fullsized":"2024-01-31T05:21:47.691357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:55:51.545144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.693894+0000","last_change":"2024-01-31T05:20:40.729677+0000","last_active":"2024-01-31T05:21:47.693894+0000","last_peered":"2024-01-31T05:21:47.693894+0000","last_clean":"2024-01-31T05:21:47.693894+0000","last_became_active":"2024-01-31T05:20:40.727880+0000","last_became_peered":"2024-01-31T05:20:40.727880+0000","last_unstale":"2024-01-31T05:21:47.693894+0000","last_undegraded":"2024-01-31T05:21:47.693894+0000","last_fullsized":"2024-01-31T05:21:47.693894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:49:55.935842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:50.574828+0000","last_change":"2024-01-31T05:20:39.711310+0000","last_active":"2024-01-31T05:21:50.574828+0000","last_peered":"2024-01-31T05:21:50.574828+0000","last_clean":"2024-01-31T05:21:50.574828+0000","last_became_active":"2024-01-31T05:20:39.710108+0000","last_became_peered":"2024-01-31T05:20:39.710108+0000","last_unstale":"2024-01-31T05:21:50.574828+0000","last_undegraded":"2024-01-31T05:21:50.574828+0000","last_fullsized":"2024-01-31T05:21:50.574828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:03:25.550777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689420+0000","last_change":"2024-01-31T05:20:40.722095+0000","last_active":"2024-01-31T05:21:47.689420+0000","last_peered":"2024-01-31T05:21:47.689420+0000","last_clean":"2024-01-31T05:21:47.689420+0000","last_became_active":"2024-01-31T05:20:40.721778+0000","last_became_peered":"2024-01-31T05:20:40.721778+0000","last_unstale":"2024-01-31T05:21:47.689420+0000","last_undegraded":"2024-01-31T05:21:47.689420+0000","last_fullsized":"2024-01-31T05:21:47.689420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:48:04.284745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895807+0000","last_change":"2024-01-31T05:20:39.705704+0000","last_active":"2024-01-31T05:21:47.895807+0000","last_peered":"2024-01-31T05:21:47.895807+0000","last_clean":"2024-01-31T05:21:47.895807+0000","last_became_active":"2024-01-31T05:20:39.705304+0000","last_became_peered":"2024-01-31T05:20:39.705304+0000","last_unstale":"2024-01-31T05:21:47.895807+0000","last_undegraded":"2024-01-31T05:21:47.895807+0000","last_fullsized":"2024-01-31T05:21:47.895807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:01:40.116085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.897314+0000","last_change":"2024-01-31T05:21:47.897314+0000","last_active":"2024-01-31T05:21:47.897314+0000","last_peered":"2024-01-31T05:21:47.897314+0000","last_clean":"2024-01-31T05:21:47.897314+0000","last_became_active":"2024-01-31T05:21:47.896873+0000","last_became_peered":"2024-01-31T05:21:47.896873+0000","last_unstale":"2024-01-31T05:21:47.897314+0000","last_undegraded":"2024-01-31T05:21:47.897314+0000","last_fullsized":"2024-01-31T05:21:47.897314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:56:25.953686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895551+0000","last_change":"2024-01-31T05:20:39.705157+0000","last_active":"2024-01-31T05:21:47.895551+0000","last_peered":"2024-01-31T05:21:47.895551+0000","last_clean":"2024-01-31T05:21:47.895551+0000","last_became_active":"2024-01-31T05:20:39.704794+0000","last_became_peered":"2024-01-31T05:20:39.704794+0000","last_unstale":"2024-01-31T05:21:47.895551+0000","last_undegraded":"2024-01-31T05:21:47.895551+0000","last_fullsized":"2024-01-31T05:21:47.895551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:21:20.314515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.696058+0000","last_change":"2024-01-31T05:20:40.723906+0000","last_active":"2024-01-31T05:21:47.696058+0000","last_peered":"2024-01-31T05:21:47.696058+0000","last_clean":"2024-01-31T05:21:47.696058+0000","last_became_active":"2024-01-31T05:20:40.723612+0000","last_became_peered":"2024-01-31T05:20:40.723612+0000","last_unstale":"2024-01-31T05:21:47.696058+0000","last_undegraded":"2024-01-31T05:21:47.696058+0000","last_fullsized":"2024-01-31T05:21:47.696058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:42:42.509983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691991+0000","last_change":"2024-01-31T05:20:39.709315+0000","last_active":"2024-01-31T05:21:47.691991+0000","last_peered":"2024-01-31T05:21:47.691991+0000","last_clean":"2024-01-31T05:21:47.691991+0000","last_became_active":"2024-01-31T05:20:39.709033+0000","last_became_peered":"2024-01-31T05:20:39.709033+0000","last_unstale":"2024-01-31T05:21:47.691991+0000","last_undegraded":"2024-01-31T05:21:47.691991+0000","last_fullsized":"2024-01-31T05:21:47.691991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:43:29.540029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.685448+0000","last_change":"2024-01-31T05:20:40.717396+0000","last_active":"2024-01-31T05:21:47.685448+0000","last_peered":"2024-01-31T05:21:47.685448+0000","last_clean":"2024-01-31T05:21:47.685448+0000","last_became_active":"2024-01-31T05:20:40.717076+0000","last_became_peered":"2024-01-31T05:20:40.717076+0000","last_unstale":"2024-01-31T05:21:47.685448+0000","last_undegraded":"2024-01-31T05:21:47.685448+0000","last_fullsized":"2024-01-31T05:21:47.685448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:42:51.667070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:49.572793+0000","last_change":"2024-01-31T05:20:39.716940+0000","last_active":"2024-01-31T05:21:49.572793+0000","last_peered":"2024-01-31T05:21:49.572793+0000","last_clean":"2024-01-31T05:21:49.572793+0000","last_became_active":"2024-01-31T05:20:39.716691+0000","last_became_peered":"2024-01-31T05:20:39.716691+0000","last_unstale":"2024-01-31T05:21:49.572793+0000","last_undegraded":"2024-01-31T05:21:49.572793+0000","last_fullsized":"2024-01-31T05:21:49.572793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:38:23.786807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689798+0000","last_change":"2024-01-31T05:20:40.727618+0000","last_active":"2024-01-31T05:21:47.689798+0000","last_peered":"2024-01-31T05:21:47.689798+0000","last_clean":"2024-01-31T05:21:47.689798+0000","last_became_active":"2024-01-31T05:20:40.727282+0000","last_became_peered":"2024-01-31T05:20:40.727282+0000","last_unstale":"2024-01-31T05:21:47.689798+0000","last_undegraded":"2024-01-31T05:21:47.689798+0000","last_fullsized":"2024-01-31T05:21:47.689798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:07:12.416575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686860+0000","last_change":"2024-01-31T05:20:39.709584+0000","last_active":"2024-01-31T05:21:47.686860+0000","last_peered":"2024-01-31T05:21:47.686860+0000","last_clean":"2024-01-31T05:21:47.686860+0000","last_became_active":"2024-01-31T05:20:39.708660+0000","last_became_peered":"2024-01-31T05:20:39.708660+0000","last_unstale":"2024-01-31T05:21:47.686860+0000","last_undegraded":"2024-01-31T05:21:47.686860+0000","last_fullsized":"2024-01-31T05:21:47.686860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:21:06.450913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691837+0000","last_change":"2024-01-31T05:20:40.734040+0000","last_active":"2024-01-31T05:21:47.691837+0000","last_peered":"2024-01-31T05:21:47.691837+0000","last_clean":"2024-01-31T05:21:47.691837+0000","last_became_active":"2024-01-31T05:20:40.733307+0000","last_became_peered":"2024-01-31T05:20:40.733307+0000","last_unstale":"2024-01-31T05:21:47.691837+0000","last_undegraded":"2024-01-31T05:21:47.691837+0000","last_fullsized":"2024-01-31T05:21:47.691837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:31:01.584900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.693932+0000","last_change":"2024-01-31T05:20:39.708004+0000","last_active":"2024-01-31T05:21:47.693932+0000","last_peered":"2024-01-31T05:21:47.693932+0000","last_clean":"2024-01-31T05:21:47.693932+0000","last_became_active":"2024-01-31T05:20:39.707681+0000","last_became_peered":"2024-01-31T05:20:39.707681+0000","last_unstale":"2024-01-31T05:21:47.693932+0000","last_undegraded":"2024-01-31T05:21:47.693932+0000","last_fullsized":"2024-01-31T05:21:47.693932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:24:10.003803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689601+0000","last_change":"2024-01-31T05:20:40.714449+0000","last_active":"2024-01-31T05:21:47.689601+0000","last_peered":"2024-01-31T05:21:47.689601+0000","last_clean":"2024-01-31T05:21:47.689601+0000","last_became_active":"2024-01-31T05:20:40.714064+0000","last_became_peered":"2024-01-31T05:20:40.714064+0000","last_unstale":"2024-01-31T05:21:47.689601+0000","last_undegraded":"2024-01-31T05:21:47.689601+0000","last_fullsized":"2024-01-31T05:21:47.689601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:17:04.803852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":190,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:50.574127+0000","last_change":"2024-01-31T05:20:39.709050+0000","last_active":"2024-01-31T05:21:50.574127+0000","last_peered":"2024-01-31T05:21:50.574127+0000","last_clean":"2024-01-31T05:21:50.574127+0000","last_became_active":"2024-01-31T05:20:39.708674+0000","last_became_peered":"2024-01-31T05:20:39.708674+0000","last_unstale":"2024-01-31T05:21:50.574127+0000","last_undegraded":"2024-01-31T05:21:50.574127+0000","last_fullsized":"2024-01-31T05:21:50.574127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:08:34.994683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59160,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":128,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688902+0000","last_change":"2024-01-31T05:20:40.727191+0000","last_active":"2024-01-31T05:21:47.688902+0000","last_peered":"2024-01-31T05:21:47.688902+0000","last_clean":"2024-01-31T05:21:47.688902+0000","last_became_active":"2024-01-31T05:20:40.725114+0000","last_became_peered":"2024-01-31T05:20:40.725114+0000","last_unstale":"2024-01-31T05:21:47.688902+0000","last_undegraded":"2024-01-31T05:21:47.688902+0000","last_fullsized":"2024-01-31T05:21:47.688902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:00.758702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688113+0000","last_change":"2024-01-31T05:20:39.719349+0000","last_active":"2024-01-31T05:21:47.688113+0000","last_peered":"2024-01-31T05:21:47.688113+0000","last_clean":"2024-01-31T05:21:47.688113+0000","last_became_active":"2024-01-31T05:20:39.718918+0000","last_became_peered":"2024-01-31T05:20:39.718918+0000","last_unstale":"2024-01-31T05:21:47.688113+0000","last_undegraded":"2024-01-31T05:21:47.688113+0000","last_fullsized":"2024-01-31T05:21:47.688113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:47:48.521638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895635+0000","last_change":"2024-01-31T05:20:40.716868+0000","last_active":"2024-01-31T05:21:47.895635+0000","last_peered":"2024-01-31T05:21:47.895635+0000","last_clean":"2024-01-31T05:21:47.895635+0000","last_became_active":"2024-01-31T05:20:40.716519+0000","last_became_peered":"2024-01-31T05:20:40.716519+0000","last_unstale":"2024-01-31T05:21:47.895635+0000","last_undegraded":"2024-01-31T05:21:47.895635+0000","last_fullsized":"2024-01-31T05:21:47.895635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:21:05.365719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692637+0000","last_change":"2024-01-31T05:20:39.716631+0000","last_active":"2024-01-31T05:21:47.692637+0000","last_peered":"2024-01-31T05:21:47.692637+0000","last_clean":"2024-01-31T05:21:47.692637+0000","last_became_active":"2024-01-31T05:20:39.716331+0000","last_became_peered":"2024-01-31T05:20:39.716331+0000","last_unstale":"2024-01-31T05:21:47.692637+0000","last_undegraded":"2024-01-31T05:21:47.692637+0000","last_fullsized":"2024-01-31T05:21:47.692637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:05:59.555000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686625+0000","last_change":"2024-01-31T05:20:40.721548+0000","last_active":"2024-01-31T05:21:47.686625+0000","last_peered":"2024-01-31T05:21:47.686625+0000","last_clean":"2024-01-31T05:21:47.686625+0000","last_became_active":"2024-01-31T05:20:40.720296+0000","last_became_peered":"2024-01-31T05:20:40.720296+0000","last_unstale":"2024-01-31T05:21:47.686625+0000","last_undegraded":"2024-01-31T05:21:47.686625+0000","last_fullsized":"2024-01-31T05:21:47.686625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:04:08.406863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":88,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692796+0000","last_change":"2024-01-31T05:20:39.707644+0000","last_active":"2024-01-31T05:21:47.692796+0000","last_peered":"2024-01-31T05:21:47.692796+0000","last_clean":"2024-01-31T05:21:47.692796+0000","last_became_active":"2024-01-31T05:20:39.707334+0000","last_became_peered":"2024-01-31T05:20:39.707334+0000","last_unstale":"2024-01-31T05:21:47.692796+0000","last_undegraded":"2024-01-31T05:21:47.692796+0000","last_fullsized":"2024-01-31T05:21:47.692796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:13:25.630415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686277+0000","last_change":"2024-01-31T05:20:40.715835+0000","last_active":"2024-01-31T05:21:47.686277+0000","last_peered":"2024-01-31T05:21:47.686277+0000","last_clean":"2024-01-31T05:21:47.686277+0000","last_became_active":"2024-01-31T05:20:40.715467+0000","last_became_peered":"2024-01-31T05:20:40.715467+0000","last_unstale":"2024-01-31T05:21:47.686277+0000","last_undegraded":"2024-01-31T05:21:47.686277+0000","last_fullsized":"2024-01-31T05:21:47.686277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:40:33.583238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688427+0000","last_change":"2024-01-31T05:20:39.717929+0000","last_active":"2024-01-31T05:21:47.688427+0000","last_peered":"2024-01-31T05:21:47.688427+0000","last_clean":"2024-01-31T05:21:47.688427+0000","last_became_active":"2024-01-31T05:20:39.717644+0000","last_became_peered":"2024-01-31T05:20:39.717644+0000","last_unstale":"2024-01-31T05:21:47.688427+0000","last_undegraded":"2024-01-31T05:21:47.688427+0000","last_fullsized":"2024-01-31T05:21:47.688427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:24:41.296706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692360+0000","last_change":"2024-01-31T05:20:40.726182+0000","last_active":"2024-01-31T05:21:47.692360+0000","last_peered":"2024-01-31T05:21:47.692360+0000","last_clean":"2024-01-31T05:21:47.692360+0000","last_became_active":"2024-01-31T05:20:40.725858+0000","last_became_peered":"2024-01-31T05:20:40.725858+0000","last_unstale":"2024-01-31T05:21:47.692360+0000","last_undegraded":"2024-01-31T05:21:47.692360+0000","last_fullsized":"2024-01-31T05:21:47.692360+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:27.209744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689293+0000","last_change":"2024-01-31T05:20:39.706132+0000","last_active":"2024-01-31T05:21:47.689293+0000","last_peered":"2024-01-31T05:21:47.689293+0000","last_clean":"2024-01-31T05:21:47.689293+0000","last_became_active":"2024-01-31T05:20:39.705791+0000","last_became_peered":"2024-01-31T05:20:39.705791+0000","last_unstale":"2024-01-31T05:21:47.689293+0000","last_undegraded":"2024-01-31T05:21:47.689293+0000","last_fullsized":"2024-01-31T05:21:47.689293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:54:57.857011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.696014+0000","last_change":"2024-01-31T05:21:47.696014+0000","last_active":"2024-01-31T05:21:47.696014+0000","last_peered":"2024-01-31T05:21:47.696014+0000","last_clean":"2024-01-31T05:21:47.696014+0000","last_became_active":"2024-01-31T05:21:47.695012+0000","last_became_peered":"2024-01-31T05:21:47.695012+0000","last_unstale":"2024-01-31T05:21:47.696014+0000","last_undegraded":"2024-01-31T05:21:47.696014+0000","last_fullsized":"2024-01-31T05:21:47.696014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:58:49.034777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.685513+0000","last_change":"2024-01-31T05:20:39.710195+0000","last_active":"2024-01-31T05:21:47.685513+0000","last_peered":"2024-01-31T05:21:47.685513+0000","last_clean":"2024-01-31T05:21:47.685513+0000","last_became_active":"2024-01-31T05:20:39.709787+0000","last_became_peered":"2024-01-31T05:20:39.709787+0000","last_unstale":"2024-01-31T05:21:47.685513+0000","last_undegraded":"2024-01-31T05:21:47.685513+0000","last_fullsized":"2024-01-31T05:21:47.685513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:28:36.283468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.896149+0000","last_change":"2024-01-31T05:20:40.716243+0000","last_active":"2024-01-31T05:21:47.896149+0000","last_peered":"2024-01-31T05:21:47.896149+0000","last_clean":"2024-01-31T05:21:47.896149+0000","last_became_active":"2024-01-31T05:20:40.715927+0000","last_became_peered":"2024-01-31T05:20:40.715927+0000","last_unstale":"2024-01-31T05:21:47.896149+0000","last_undegraded":"2024-01-31T05:21:47.896149+0000","last_fullsized":"2024-01-31T05:21:47.896149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:20:19.289014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686547+0000","last_change":"2024-01-31T05:20:39.712333+0000","last_active":"2024-01-31T05:21:47.686547+0000","last_peered":"2024-01-31T05:21:47.686547+0000","last_clean":"2024-01-31T05:21:47.686547+0000","last_became_active":"2024-01-31T05:20:39.712008+0000","last_became_peered":"2024-01-31T05:20:39.712008+0000","last_unstale":"2024-01-31T05:21:47.686547+0000","last_undegraded":"2024-01-31T05:21:47.686547+0000","last_fullsized":"2024-01-31T05:21:47.686547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:04:52.962130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.896096+0000","last_change":"2024-01-31T05:20:40.725131+0000","last_active":"2024-01-31T05:21:47.896096+0000","last_peered":"2024-01-31T05:21:47.896096+0000","last_clean":"2024-01-31T05:21:47.896096+0000","last_became_active":"2024-01-31T05:20:40.724797+0000","last_became_peered":"2024-01-31T05:20:40.724797+0000","last_unstale":"2024-01-31T05:21:47.896096+0000","last_undegraded":"2024-01-31T05:21:47.896096+0000","last_fullsized":"2024-01-31T05:21:47.896096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:23:26.535023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692221+0000","last_change":"2024-01-31T05:20:40.726857+0000","last_active":"2024-01-31T05:21:47.692221+0000","last_peered":"2024-01-31T05:21:47.692221+0000","last_clean":"2024-01-31T05:21:47.692221+0000","last_became_active":"2024-01-31T05:20:40.726545+0000","last_became_peered":"2024-01-31T05:20:40.726545+0000","last_unstale":"2024-01-31T05:21:47.692221+0000","last_undegraded":"2024-01-31T05:21:47.692221+0000","last_fullsized":"2024-01-31T05:21:47.692221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:43:55.852474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:50.573500+0000","last_change":"2024-01-31T05:20:39.713920+0000","last_active":"2024-01-31T05:21:50.573500+0000","last_peered":"2024-01-31T05:21:50.573500+0000","last_clean":"2024-01-31T05:21:50.573500+0000","last_became_active":"2024-01-31T05:20:39.713582+0000","last_became_peered":"2024-01-31T05:20:39.713582+0000","last_unstale":"2024-01-31T05:21:50.573500+0000","last_undegraded":"2024-01-31T05:21:50.573500+0000","last_fullsized":"2024-01-31T05:21:50.573500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:20:18.932993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695600+0000","last_change":"2024-01-31T05:20:40.720442+0000","last_active":"2024-01-31T05:21:47.695600+0000","last_peered":"2024-01-31T05:21:47.695600+0000","last_clean":"2024-01-31T05:21:47.695600+0000","last_became_active":"2024-01-31T05:20:40.720114+0000","last_became_peered":"2024-01-31T05:20:40.720114+0000","last_unstale":"2024-01-31T05:21:47.695600+0000","last_undegraded":"2024-01-31T05:21:47.695600+0000","last_fullsized":"2024-01-31T05:21:47.695600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:11:01.536820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692617+0000","last_change":"2024-01-31T05:20:39.706791+0000","last_active":"2024-01-31T05:21:47.692617+0000","last_peered":"2024-01-31T05:21:47.692617+0000","last_clean":"2024-01-31T05:21:47.692617+0000","last_became_active":"2024-01-31T05:20:39.706447+0000","last_became_peered":"2024-01-31T05:20:39.706447+0000","last_unstale":"2024-01-31T05:21:47.692617+0000","last_undegraded":"2024-01-31T05:21:47.692617+0000","last_fullsized":"2024-01-31T05:21:47.692617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:41:05.118138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686420+0000","last_change":"2024-01-31T05:19:28.446661+0000","last_active":"2024-01-31T05:21:47.686420+0000","last_peered":"2024-01-31T05:21:47.686420+0000","last_clean":"2024-01-31T05:21:47.686420+0000","last_became_active":"2024-01-31T05:19:28.446238+0000","last_became_peered":"2024-01-31T05:19:28.446238+0000","last_unstale":"2024-01-31T05:21:47.686420+0000","last_undegraded":"2024-01-31T05:21:47.686420+0000","last_fullsized":"2024-01-31T05:21:47.686420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:16:11.415995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:16:11.415995+0000","last_clean_scrub_stamp":"2024-01-31T05:16:11.415995+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:56:53.665942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895436+0000","last_change":"2024-01-31T05:20:40.714343+0000","last_active":"2024-01-31T05:21:47.895436+0000","last_peered":"2024-01-31T05:21:47.895436+0000","last_clean":"2024-01-31T05:21:47.895436+0000","last_became_active":"2024-01-31T05:20:40.713820+0000","last_became_peered":"2024-01-31T05:20:40.713820+0000","last_unstale":"2024-01-31T05:21:47.895436+0000","last_undegraded":"2024-01-31T05:21:47.895436+0000","last_fullsized":"2024-01-31T05:21:47.895436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:44:16.525005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686643+0000","last_change":"2024-01-31T05:20:39.712068+0000","last_active":"2024-01-31T05:21:47.686643+0000","last_peered":"2024-01-31T05:21:47.686643+0000","last_clean":"2024-01-31T05:21:47.686643+0000","last_became_active":"2024-01-31T05:20:39.711641+0000","last_became_peered":"2024-01-31T05:20:39.711641+0000","last_unstale":"2024-01-31T05:21:47.686643+0000","last_undegraded":"2024-01-31T05:21:47.686643+0000","last_fullsized":"2024-01-31T05:21:47.686643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:27.199351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689032+0000","last_change":"2024-01-31T05:20:39.716003+0000","last_active":"2024-01-31T05:21:47.689032+0000","last_peered":"2024-01-31T05:21:47.689032+0000","last_clean":"2024-01-31T05:21:47.689032+0000","last_became_active":"2024-01-31T05:20:39.715653+0000","last_became_peered":"2024-01-31T05:20:39.715653+0000","last_unstale":"2024-01-31T05:21:47.689032+0000","last_undegraded":"2024-01-31T05:21:47.689032+0000","last_fullsized":"2024-01-31T05:21:47.689032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:45:56.886077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895762+0000","last_change":"2024-01-31T05:20:40.712298+0000","last_active":"2024-01-31T05:21:47.895762+0000","last_peered":"2024-01-31T05:21:47.895762+0000","last_clean":"2024-01-31T05:21:47.895762+0000","last_became_active":"2024-01-31T05:20:40.711907+0000","last_became_peered":"2024-01-31T05:20:40.711907+0000","last_unstale":"2024-01-31T05:21:47.895762+0000","last_undegraded":"2024-01-31T05:21:47.895762+0000","last_fullsized":"2024-01-31T05:21:47.895762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:04:13.482003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.685696+0000","last_change":"2024-01-31T05:20:39.720639+0000","last_active":"2024-01-31T05:21:47.685696+0000","last_peered":"2024-01-31T05:21:47.685696+0000","last_clean":"2024-01-31T05:21:47.685696+0000","last_became_active":"2024-01-31T05:20:39.720302+0000","last_became_peered":"2024-01-31T05:20:39.720302+0000","last_unstale":"2024-01-31T05:21:47.685696+0000","last_undegraded":"2024-01-31T05:21:47.685696+0000","last_fullsized":"2024-01-31T05:21:47.685696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:11:31.507908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695719+0000","last_change":"2024-01-31T05:20:40.725324+0000","last_active":"2024-01-31T05:21:47.695719+0000","last_peered":"2024-01-31T05:21:47.695719+0000","last_clean":"2024-01-31T05:21:47.695719+0000","last_became_active":"2024-01-31T05:20:40.725027+0000","last_became_peered":"2024-01-31T05:20:40.725027+0000","last_unstale":"2024-01-31T05:21:47.695719+0000","last_undegraded":"2024-01-31T05:21:47.695719+0000","last_fullsized":"2024-01-31T05:21:47.695719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:06:24.129276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692814+0000","last_change":"2024-01-31T05:20:39.705805+0000","last_active":"2024-01-31T05:21:47.692814+0000","last_peered":"2024-01-31T05:21:47.692814+0000","last_clean":"2024-01-31T05:21:47.692814+0000","last_became_active":"2024-01-31T05:20:39.705447+0000","last_became_peered":"2024-01-31T05:20:39.705447+0000","last_unstale":"2024-01-31T05:21:47.692814+0000","last_undegraded":"2024-01-31T05:21:47.692814+0000","last_fullsized":"2024-01-31T05:21:47.692814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:24:47.856773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691840+0000","last_change":"2024-01-31T05:20:40.733818+0000","last_active":"2024-01-31T05:21:47.691840+0000","last_peered":"2024-01-31T05:21:47.691840+0000","last_clean":"2024-01-31T05:21:47.691840+0000","last_became_active":"2024-01-31T05:20:40.733272+0000","last_became_peered":"2024-01-31T05:20:40.733272+0000","last_unstale":"2024-01-31T05:21:47.691840+0000","last_undegraded":"2024-01-31T05:21:47.691840+0000","last_fullsized":"2024-01-31T05:21:47.691840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:19:14.144194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.699674+0000","last_change":"2024-01-31T05:21:47.699674+0000","last_active":"2024-01-31T05:21:47.699674+0000","last_peered":"2024-01-31T05:21:47.699674+0000","last_clean":"2024-01-31T05:21:47.699674+0000","last_became_active":"2024-01-31T05:21:47.693473+0000","last_became_peered":"2024-01-31T05:21:47.693473+0000","last_unstale":"2024-01-31T05:21:47.699674+0000","last_undegraded":"2024-01-31T05:21:47.699674+0000","last_fullsized":"2024-01-31T05:21:47.699674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:57:20.931061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688519+0000","last_change":"2024-01-31T05:20:40.722376+0000","last_active":"2024-01-31T05:21:47.688519+0000","last_peered":"2024-01-31T05:21:47.688519+0000","last_clean":"2024-01-31T05:21:47.688519+0000","last_became_active":"2024-01-31T05:20:40.722055+0000","last_became_peered":"2024-01-31T05:20:40.722055+0000","last_unstale":"2024-01-31T05:21:47.688519+0000","last_undegraded":"2024-01-31T05:21:47.688519+0000","last_fullsized":"2024-01-31T05:21:47.688519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:11:47.198866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695502+0000","last_change":"2024-01-31T05:20:39.714498+0000","last_active":"2024-01-31T05:21:47.695502+0000","last_peered":"2024-01-31T05:21:47.695502+0000","last_clean":"2024-01-31T05:21:47.695502+0000","last_became_active":"2024-01-31T05:20:39.714179+0000","last_became_peered":"2024-01-31T05:20:39.714179+0000","last_unstale":"2024-01-31T05:21:47.695502+0000","last_undegraded":"2024-01-31T05:21:47.695502+0000","last_fullsized":"2024-01-31T05:21:47.695502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:42:55.913696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689706+0000","last_change":"2024-01-31T05:20:40.716878+0000","last_active":"2024-01-31T05:21:47.689706+0000","last_peered":"2024-01-31T05:21:47.689706+0000","last_clean":"2024-01-31T05:21:47.689706+0000","last_became_active":"2024-01-31T05:20:40.715998+0000","last_became_peered":"2024-01-31T05:20:40.715998+0000","last_unstale":"2024-01-31T05:21:47.689706+0000","last_undegraded":"2024-01-31T05:21:47.689706+0000","last_fullsized":"2024-01-31T05:21:47.689706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:59:31.030147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.687838+0000","last_change":"2024-01-31T05:20:39.719267+0000","last_active":"2024-01-31T05:21:47.687838+0000","last_peered":"2024-01-31T05:21:47.687838+0000","last_clean":"2024-01-31T05:21:47.687838+0000","last_became_active":"2024-01-31T05:20:39.718964+0000","last_became_peered":"2024-01-31T05:20:39.718964+0000","last_unstale":"2024-01-31T05:21:47.687838+0000","last_undegraded":"2024-01-31T05:21:47.687838+0000","last_fullsized":"2024-01-31T05:21:47.687838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:31.150796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695812+0000","last_change":"2024-01-31T05:20:40.727530+0000","last_active":"2024-01-31T05:21:47.695812+0000","last_peered":"2024-01-31T05:21:47.695812+0000","last_clean":"2024-01-31T05:21:47.695812+0000","last_became_active":"2024-01-31T05:20:40.727016+0000","last_became_peered":"2024-01-31T05:20:40.727016+0000","last_unstale":"2024-01-31T05:21:47.695812+0000","last_undegraded":"2024-01-31T05:21:47.695812+0000","last_fullsized":"2024-01-31T05:21:47.695812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:15:25.851713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688698+0000","last_change":"2024-01-31T05:20:39.711083+0000","last_active":"2024-01-31T05:21:47.688698+0000","last_peered":"2024-01-31T05:21:47.688698+0000","last_clean":"2024-01-31T05:21:47.688698+0000","last_became_active":"2024-01-31T05:20:39.710732+0000","last_became_peered":"2024-01-31T05:20:39.710732+0000","last_unstale":"2024-01-31T05:21:47.688698+0000","last_undegraded":"2024-01-31T05:21:47.688698+0000","last_fullsized":"2024-01-31T05:21:47.688698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:43.846910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686565+0000","last_change":"2024-01-31T05:20:40.723667+0000","last_active":"2024-01-31T05:21:47.686565+0000","last_peered":"2024-01-31T05:21:47.686565+0000","last_clean":"2024-01-31T05:21:47.686565+0000","last_became_active":"2024-01-31T05:20:40.723343+0000","last_became_peered":"2024-01-31T05:20:40.723343+0000","last_unstale":"2024-01-31T05:21:47.686565+0000","last_undegraded":"2024-01-31T05:21:47.686565+0000","last_fullsized":"2024-01-31T05:21:47.686565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:12.983137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688823+0000","last_change":"2024-01-31T05:20:39.717263+0000","last_active":"2024-01-31T05:21:47.688823+0000","last_peered":"2024-01-31T05:21:47.688823+0000","last_clean":"2024-01-31T05:21:47.688823+0000","last_became_active":"2024-01-31T05:20:39.716666+0000","last_became_peered":"2024-01-31T05:20:39.716666+0000","last_unstale":"2024-01-31T05:21:47.688823+0000","last_undegraded":"2024-01-31T05:21:47.688823+0000","last_fullsized":"2024-01-31T05:21:47.688823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:10:41.789503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686865+0000","last_change":"2024-01-31T05:20:40.721728+0000","last_active":"2024-01-31T05:21:47.686865+0000","last_peered":"2024-01-31T05:21:47.686865+0000","last_clean":"2024-01-31T05:21:47.686865+0000","last_became_active":"2024-01-31T05:20:40.717082+0000","last_became_peered":"2024-01-31T05:20:40.717082+0000","last_unstale":"2024-01-31T05:21:47.686865+0000","last_undegraded":"2024-01-31T05:21:47.686865+0000","last_fullsized":"2024-01-31T05:21:47.686865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:15:55.823060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.693723+0000","last_change":"2024-01-31T05:20:39.715450+0000","last_active":"2024-01-31T05:21:47.693723+0000","last_peered":"2024-01-31T05:21:47.693723+0000","last_clean":"2024-01-31T05:21:47.693723+0000","last_became_active":"2024-01-31T05:20:39.715140+0000","last_became_peered":"2024-01-31T05:20:39.715140+0000","last_unstale":"2024-01-31T05:21:47.693723+0000","last_undegraded":"2024-01-31T05:21:47.693723+0000","last_fullsized":"2024-01-31T05:21:47.693723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:34:01.299228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686541+0000","last_change":"2024-01-31T05:20:40.723855+0000","last_active":"2024-01-31T05:21:47.686541+0000","last_peered":"2024-01-31T05:21:47.686541+0000","last_clean":"2024-01-31T05:21:47.686541+0000","last_became_active":"2024-01-31T05:20:40.723532+0000","last_became_peered":"2024-01-31T05:20:40.723532+0000","last_unstale":"2024-01-31T05:21:47.686541+0000","last_undegraded":"2024-01-31T05:21:47.686541+0000","last_fullsized":"2024-01-31T05:21:47.686541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:48:59.700145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895790+0000","last_change":"2024-01-31T05:20:39.713941+0000","last_active":"2024-01-31T05:21:47.895790+0000","last_peered":"2024-01-31T05:21:47.895790+0000","last_clean":"2024-01-31T05:21:47.895790+0000","last_became_active":"2024-01-31T05:20:39.713610+0000","last_became_peered":"2024-01-31T05:20:39.713610+0000","last_unstale":"2024-01-31T05:21:47.895790+0000","last_undegraded":"2024-01-31T05:21:47.895790+0000","last_fullsized":"2024-01-31T05:21:47.895790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:48:13.722639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686946+0000","last_change":"2024-01-31T05:20:40.716485+0000","last_active":"2024-01-31T05:21:47.686946+0000","last_peered":"2024-01-31T05:21:47.686946+0000","last_clean":"2024-01-31T05:21:47.686946+0000","last_became_active":"2024-01-31T05:20:40.716177+0000","last_became_peered":"2024-01-31T05:20:40.716177+0000","last_unstale":"2024-01-31T05:21:47.686946+0000","last_undegraded":"2024-01-31T05:21:47.686946+0000","last_fullsized":"2024-01-31T05:21:47.686946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:25:37.359617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689813+0000","last_change":"2024-01-31T05:20:40.710376+0000","last_active":"2024-01-31T05:21:47.689813+0000","last_peered":"2024-01-31T05:21:47.689813+0000","last_clean":"2024-01-31T05:21:47.689813+0000","last_became_active":"2024-01-31T05:20:40.710017+0000","last_became_peered":"2024-01-31T05:20:40.710017+0000","last_unstale":"2024-01-31T05:21:47.689813+0000","last_undegraded":"2024-01-31T05:21:47.689813+0000","last_fullsized":"2024-01-31T05:21:47.689813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:48:37.519561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695196+0000","last_change":"2024-01-31T05:21:47.695196+0000","last_active":"2024-01-31T05:21:47.695196+0000","last_peered":"2024-01-31T05:21:47.695196+0000","last_clean":"2024-01-31T05:21:47.695196+0000","last_became_active":"2024-01-31T05:21:47.694661+0000","last_became_peered":"2024-01-31T05:21:47.694661+0000","last_unstale":"2024-01-31T05:21:47.695196+0000","last_undegraded":"2024-01-31T05:21:47.695196+0000","last_fullsized":"2024-01-31T05:21:47.695196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:15:56.062684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.687683+0000","last_change":"2024-01-31T05:20:40.725974+0000","last_active":"2024-01-31T05:21:47.687683+0000","last_peered":"2024-01-31T05:21:47.687683+0000","last_clean":"2024-01-31T05:21:47.687683+0000","last_became_active":"2024-01-31T05:20:40.725586+0000","last_became_peered":"2024-01-31T05:20:40.725586+0000","last_unstale":"2024-01-31T05:21:47.687683+0000","last_undegraded":"2024-01-31T05:21:47.687683+0000","last_fullsized":"2024-01-31T05:21:47.687683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:03:06.413502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691589+0000","last_change":"2024-01-31T05:20:40.733524+0000","last_active":"2024-01-31T05:21:47.691589+0000","last_peered":"2024-01-31T05:21:47.691589+0000","last_clean":"2024-01-31T05:21:47.691589+0000","last_became_active":"2024-01-31T05:20:40.733162+0000","last_became_peered":"2024-01-31T05:20:40.733162+0000","last_unstale":"2024-01-31T05:21:47.691589+0000","last_undegraded":"2024-01-31T05:21:47.691589+0000","last_fullsized":"2024-01-31T05:21:47.691589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:12:03.146533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895685+0000","last_change":"2024-01-31T05:20:40.715460+0000","last_active":"2024-01-31T05:21:47.895685+0000","last_peered":"2024-01-31T05:21:47.895685+0000","last_clean":"2024-01-31T05:21:47.895685+0000","last_became_active":"2024-01-31T05:20:40.715103+0000","last_became_peered":"2024-01-31T05:20:40.715103+0000","last_unstale":"2024-01-31T05:21:47.895685+0000","last_undegraded":"2024-01-31T05:21:47.895685+0000","last_fullsized":"2024-01-31T05:21:47.895685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:41:43.178835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688467+0000","last_change":"2024-01-31T05:20:40.714317+0000","last_active":"2024-01-31T05:21:47.688467+0000","last_peered":"2024-01-31T05:21:47.688467+0000","last_clean":"2024-01-31T05:21:47.688467+0000","last_became_active":"2024-01-31T05:20:40.714008+0000","last_became_peered":"2024-01-31T05:20:40.714008+0000","last_unstale":"2024-01-31T05:21:47.688467+0000","last_undegraded":"2024-01-31T05:21:47.688467+0000","last_fullsized":"2024-01-31T05:21:47.688467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:36:39.740278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688019+0000","last_change":"2024-01-31T05:20:40.710901+0000","last_active":"2024-01-31T05:21:47.688019+0000","last_peered":"2024-01-31T05:21:47.688019+0000","last_clean":"2024-01-31T05:21:47.688019+0000","last_became_active":"2024-01-31T05:20:40.710412+0000","last_became_peered":"2024-01-31T05:20:40.710412+0000","last_unstale":"2024-01-31T05:21:47.688019+0000","last_undegraded":"2024-01-31T05:21:47.688019+0000","last_fullsized":"2024-01-31T05:21:47.688019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:39.966723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695601+0000","last_change":"2024-01-31T05:20:40.732753+0000","last_active":"2024-01-31T05:21:47.695601+0000","last_peered":"2024-01-31T05:21:47.695601+0000","last_clean":"2024-01-31T05:21:47.695601+0000","last_became_active":"2024-01-31T05:20:40.732330+0000","last_became_peered":"2024-01-31T05:20:40.732330+0000","last_unstale":"2024-01-31T05:21:47.695601+0000","last_undegraded":"2024-01-31T05:21:47.695601+0000","last_fullsized":"2024-01-31T05:21:47.695601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:54:01.933393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692108+0000","last_change":"2024-01-31T05:20:40.723129+0000","last_active":"2024-01-31T05:21:47.692108+0000","last_peered":"2024-01-31T05:21:47.692108+0000","last_clean":"2024-01-31T05:21:47.692108+0000","last_became_active":"2024-01-31T05:20:40.722796+0000","last_became_peered":"2024-01-31T05:20:40.722796+0000","last_unstale":"2024-01-31T05:21:47.692108+0000","last_undegraded":"2024-01-31T05:21:47.692108+0000","last_fullsized":"2024-01-31T05:21:47.692108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:18.120495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691136+0000","last_change":"2024-01-31T05:21:47.690719+0000","last_active":"2024-01-31T05:21:47.691136+0000","last_peered":"2024-01-31T05:21:47.691136+0000","last_clean":"2024-01-31T05:21:47.691136+0000","last_became_active":"2024-01-31T05:21:47.690347+0000","last_became_peered":"2024-01-31T05:21:47.690347+0000","last_unstale":"2024-01-31T05:21:47.691136+0000","last_undegraded":"2024-01-31T05:21:47.691136+0000","last_fullsized":"2024-01-31T05:21:47.691136+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:05.607970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689591+0000","last_change":"2024-01-31T05:20:40.720798+0000","last_active":"2024-01-31T05:21:47.689591+0000","last_peered":"2024-01-31T05:21:47.689591+0000","last_clean":"2024-01-31T05:21:47.689591+0000","last_became_active":"2024-01-31T05:20:40.720393+0000","last_became_peered":"2024-01-31T05:20:40.720393+0000","last_unstale":"2024-01-31T05:21:47.689591+0000","last_undegraded":"2024-01-31T05:21:47.689591+0000","last_fullsized":"2024-01-31T05:21:47.689591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:05:51.442874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895940+0000","last_change":"2024-01-31T05:20:40.715469+0000","last_active":"2024-01-31T05:21:47.895940+0000","last_peered":"2024-01-31T05:21:47.895940+0000","last_clean":"2024-01-31T05:21:47.895940+0000","last_became_active":"2024-01-31T05:20:40.715124+0000","last_became_peered":"2024-01-31T05:20:40.715124+0000","last_unstale":"2024-01-31T05:21:47.895940+0000","last_undegraded":"2024-01-31T05:21:47.895940+0000","last_fullsized":"2024-01-31T05:21:47.895940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:54:32.258584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688626+0000","last_change":"2024-01-31T05:20:40.719815+0000","last_active":"2024-01-31T05:21:47.688626+0000","last_peered":"2024-01-31T05:21:47.688626+0000","last_clean":"2024-01-31T05:21:47.688626+0000","last_became_active":"2024-01-31T05:20:40.719438+0000","last_became_peered":"2024-01-31T05:20:40.719438+0000","last_unstale":"2024-01-31T05:21:47.688626+0000","last_undegraded":"2024-01-31T05:21:47.688626+0000","last_fullsized":"2024-01-31T05:21:47.688626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:24:44.814291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62770,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":427,"num_read_kb":281,"num_write":156,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":188310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":458961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.60699999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.626}]},{"osd":2,"last update":"Wed Jan 31 05:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Jan 31 05:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.45800000000000002}]},{"osd":6,"last update":"Wed Jan 31 05:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.434},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.70499999999999996}]},{"osd":7,"last update":"Wed Jan 31 05:20:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":11.733000000000001,"5min":11.733000000000001,"15min":11.733000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.68500000000000005}]},{"osd":8,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.63600000000000001}]},{"osd":9,"last update":"Wed Jan 31 05:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.75},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.76500000000000001}]},{"osd":10,"last update":"Wed Jan 31 05:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.79200000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612640,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41112,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704040,"statfs":{"total":95995035648,"available":95952936960,"internally_reserved":0,"allocated":1646592,"data_stored":1109602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.503},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.51500000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.32400000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69699999999999995,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39300000000000002}]},{"osd":2,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.47999999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.46100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.55000000000000004}]},{"osd":3,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.752,"5min":0.878,"15min":0.878},"last":0.53400000000000003}]},{"osd":4,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.314,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.78400000000000003,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.45200000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89000000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47399999999999998}]},{"osd":6,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.498,"5min":0.374,"15min":0.374},"max":{"1min":0.86299999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.74399999999999999,"15min":0.753},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85899999999999999,"5min":1.044,"15min":1.044},"last":0.64500000000000002}]},{"osd":7,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.439},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.67200000000000004}]},{"osd":8,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91800000000000004,"5min":0.995,"15min":0.995},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.81599999999999995,"15min":0.83099999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.92800000000000005,"5min":1.028,"15min":1.028},"last":0.621}]},{"osd":9,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81699999999999995,"15min":0.82799999999999996},"min":{"1min":0.54100000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.88900000000000001,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.84199999999999997,"15min":0.86199999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.90500000000000003,"5min":1.073,"15min":1.073},"last":0.72099999999999997}]},{"osd":11,"last update":"Wed Jan 31 05:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.74199999999999999}]}]},{"osd":9,"up_from":68,"seq":292057776166,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40444,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704708,"statfs":{"total":95995035648,"available":95953620992,"internally_reserved":0,"allocated":966656,"data_stored":459197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.45800000000000002,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.51900000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.309},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.29399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81899999999999995,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.45000000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93200000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.39000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83199999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56399999999999995}]},{"osd":2,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0569999999999999,"5min":1.089,"15min":1.089},"last":6.343},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.51200000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.56499999999999995,"15min":0.58299999999999996},"min":{"1min":0.313,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69199999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75700000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.35499999999999998}]},{"osd":4,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68400000000000005,"15min":0.69199999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80800000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70499999999999996,"15min":0.71999999999999997},"min":{"1min":0.443,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77700000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.55100000000000005}]},{"osd":5,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.71999999999999997,"15min":0.73999999999999999},"min":{"1min":0.48299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77400000000000002,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.78900000000000003,"15min":0.81499999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81499999999999995,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.76000000000000001,"15min":0.78400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94499999999999995,"5min":1.298,"15min":1.298},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.69999999999999996,"15min":0.72099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80000000000000004,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.40600000000000003}]},{"osd":7,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.80800000000000005,"15min":0.82899999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.85399999999999998,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.61399999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.86499999999999999,"5min":1.619,"15min":1.619},"last":0.67600000000000005}]},{"osd":8,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.80600000000000005,"15min":0.83199999999999996},"min":{"1min":0.48099999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83499999999999996,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1120000000000001,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.73999999999999999}]},{"osd":10,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.78400000000000003,"15min":0.79700000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.93799999999999994,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79600000000000004,"15min":0.80700000000000005},"min":{"1min":0.57099999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.996,"5min":1.083,"15min":1.083},"last":0.77800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]},{"osd":8,"up_from":60,"seq":257698037803,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":459117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.78300000000000003,"15min":0.66500000000000004},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83599999999999997,"5min":17.460999999999999,"15min":17.460999999999999},"last":0.34599999999999997}]},{"osd":1,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.59499999999999997,"15min":0.57299999999999995},"min":{"1min":0.47799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.56899999999999995},"min":{"1min":0.48899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.77100000000000002}]},{"osd":2,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72599999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48299999999999998},"min":{"1min":0.38500000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80200000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.375}]},{"osd":3,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46700000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.44700000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.45400000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.504,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.621},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.64600000000000002,"15min":0.63100000000000001},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.64300000000000002}]},{"osd":5,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57199999999999995,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.85499999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56599999999999995}]},{"osd":6,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.84599999999999997,"15min":0.72899999999999998},"min":{"1min":0.47699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.093,"5min":17.439,"15min":17.439},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.53300000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.502}]},{"osd":7,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.69999999999999996,"15min":0.69099999999999995},"min":{"1min":0.59699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.93500000000000005,"15min":0.81799999999999995},"min":{"1min":0.623,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0349999999999999,"5min":17.420000000000002,"15min":17.420000000000002},"last":0.79000000000000004}]},{"osd":9,"last update":"Wed Jan 31 05:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":1.355,"15min":1.4510000000000001},"min":{"1min":0.58599999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2470000000000001,"5min":17.396999999999998,"15min":17.396999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":1.4079999999999999,"15min":1.508},"min":{"1min":0.60999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.296,"5min":17.489999999999998,"15min":17.489999999999998},"last":0.69799999999999995}]},{"osd":10,"last update":"Wed Jan 31 05:21:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.68500000000000005,"15min":0.66100000000000003},"min":{"1min":0.60699999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.71699999999999997,"15min":0.69599999999999995},"min":{"1min":0.65500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.80400000000000005}]},{"osd":11,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.754},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.68300000000000005}]}]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1560576,"data_stored":1050656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.49299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.59199999999999997}]},{"osd":1,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40600000000000003},"min":{"1min":0.30099999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.436},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.44,"15min":0.436},"min":{"1min":0.32800000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.39900000000000002}]},{"osd":2,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83899999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60599999999999998,"15min":0.57999999999999996},"min":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85399999999999998,"5min":1.131,"15min":1.131},"last":0.745}]},{"osd":3,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.70099999999999996,"15min":0.64200000000000002},"min":{"1min":0.39100000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92300000000000004,"5min":7.883,"15min":7.883},"last":0.69699999999999995}]},{"osd":4,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.34899999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.46400000000000002}]},{"osd":5,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60799999999999998,"15min":0.58599999999999997},"min":{"1min":0.52300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84299999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.47599999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95899999999999996,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.75700000000000001,"15min":0.70099999999999996},"min":{"1min":0.49399999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0780000000000001,"5min":7.9299999999999997,"15min":7.9299999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59899999999999998}]},{"osd":8,"last update":"Wed Jan 31 05:21:39 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70999999999999996,"15min":0.69899999999999995},"min":{"1min":0.61699999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.95899999999999996,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.70499999999999996,"15min":0.69599999999999995},"min":{"1min":0.61899999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.92900000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.85899999999999999}]},{"osd":9,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.73199999999999998,"15min":0.747},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":1.02,"15min":1.079},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.044,"5min":7.9020000000000001,"15min":7.9020000000000001},"last":0.67600000000000005}]},{"osd":10,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.751,"15min":0.755},"min":{"1min":0.58599999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91400000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76200000000000001,"15min":0.77100000000000002},"min":{"1min":0.499,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0369999999999999,"5min":1.163,"15min":1.163},"last":0.82399999999999995}]},{"osd":11,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.94699999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40436,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704716,"statfs":{"total":95995035648,"available":95953629184,"internally_reserved":0,"allocated":954368,"data_stored":458586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.29599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.41999999999999998,"15min":0.434},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.438,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.438}]},{"osd":1,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.41999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60999999999999999,"15min":0.59199999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65400000000000003}]},{"osd":3,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.46600000000000003,"15min":0.48599999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.52400000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.5},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.51500000000000001,"15min":0.52200000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.59199999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88300000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66500000000000004,"15min":0.66900000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81299999999999994,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.63900000000000001,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72999999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed Jan 31 05:21:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67500000000000004,"15min":0.65200000000000002},"min":{"1min":0.52500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98099999999999998,"5min":1.169,"15min":1.169},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96699999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.70599999999999996}]},{"osd":8,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.70399999999999996,"15min":0.72699999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91700000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71799999999999997,"15min":0.72899999999999998},"min":{"1min":0.53300000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76200000000000001}]},{"osd":9,"last update":"Wed Jan 31 05:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72799999999999998,"15min":0.74299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94399999999999995,"5min":1.091,"15min":1.091},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.71499999999999997,"15min":0.73299999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72999999999999998}]},{"osd":10,"last update":"Wed Jan 31 05:21:26 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81499999999999995,"15min":0.82499999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0580000000000001,"5min":1.131,"15min":1.131},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.81699999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.99299999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.747}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":517539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70899999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.30499999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Jan 31 05:21:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.56200000000000006,"15min":0.51700000000000002},"min":{"1min":0.42199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.53500000000000003,"15min":0.50700000000000001},"min":{"1min":0.38700000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.62}]},{"osd":3,"last update":"Wed Jan 31 05:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.52500000000000002},"min":{"1min":0.497,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.501},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.61899999999999999,"15min":0.56100000000000005},"min":{"1min":0.52600000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.44400000000000001,"15min":0.39900000000000002},"min":{"1min":0.30399999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.94999999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.436},{"interface":"front","average":{"1min":0.499,"5min":0.44600000000000001,"15min":0.40400000000000003},"min":{"1min":0.28799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76200000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.45400000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:21:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.63100000000000001,"15min":0.57499999999999996},"min":{"1min":0.58399999999999996,"5min":0.38,"15min":0.38},"max":{"1min":1.014,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.61699999999999999,"15min":0.57299999999999995},"min":{"1min":0.51300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97599999999999998,"5min":1.028,"15min":1.028},"last":0.77800000000000002}]},{"osd":6,"last update":"Wed Jan 31 05:20:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.38,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86799999999999999,"5min":0.873,"15min":0.873},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.629},"min":{"1min":0.38400000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79500000000000004}]},{"osd":7,"last update":"Wed Jan 31 05:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56599999999999995,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.41799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68700000000000006}]},{"osd":8,"last update":"Wed Jan 31 05:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.67200000000000004,"15min":0.65100000000000002},"min":{"1min":0.51700000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99099999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.73699999999999999,"15min":0.70599999999999996},"min":{"1min":0.69399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.879}]},{"osd":9,"last update":"Wed Jan 31 05:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.73699999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.75800000000000001}]},{"osd":10,"last update":"Wed Jan 31 05:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.75700000000000001,"15min":0.73899999999999999},"min":{"1min":0.60299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81999999999999995}]},{"osd":11,"last update":"Wed Jan 31 05:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.86099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40452,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704700,"statfs":{"total":95995035648,"available":95953612800,"internally_reserved":0,"allocated":974848,"data_stored":459857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.56000000000000005,"15min":0.53000000000000003},"min":{"1min":0.32000000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":0.441},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.60199999999999998,"15min":0.53300000000000003},"min":{"1min":0.377,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":7.835,"5min":7.835,"15min":7.835},"last":0.72199999999999998}]},{"osd":2,"last update":"Wed Jan 31 05:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77000000000000002,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":3.4529999999999998,"5min":18.466999999999999,"15min":18.466999999999999},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Jan 31 05:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.56499999999999995}]},{"osd":4,"last update":"Wed Jan 31 05:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58999999999999997,"15min":0.52300000000000002},"min":{"1min":0.42799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.972,"5min":2.972,"15min":2.972},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.55300000000000005,"15min":0.501},"min":{"1min":0.39800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:21:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.63600000000000001,"15min":0.55200000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":7.9130000000000003,"5min":7.9130000000000003,"15min":7.9130000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81299999999999994,"15min":0.63600000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.359,"5min":18.541,"15min":18.541},"last":0.58999999999999997}]},{"osd":6,"last update":"Wed Jan 31 05:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48799999999999999,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71099999999999997,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.60799999999999998,"15min":0.53200000000000003},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.81799999999999995,"5min":8.6560000000000006,"15min":8.6560000000000006},"last":0.61599999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.56799999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":3.1789999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.63700000000000001,"15min":0.622},"min":{"1min":0.502,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":2.105}]},{"osd":8,"last update":"Wed Jan 31 05:21:39 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.76600000000000001,"15min":0.73299999999999998},"min":{"1min":0.51200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":8.6039999999999992,"5min":8.6039999999999992,"15min":8.6039999999999992},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.67000000000000004}]},{"osd":9,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66200000000000003,"15min":0.66100000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.77400000000000002,"15min":0.74399999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.7050000000000001,"5min":8.7050000000000001,"15min":8.7050000000000001},"last":0.68799999999999994}]},{"osd":10,"last update":"Wed Jan 31 05:21:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.73299999999999998}]},{"osd":11,"last update":"Wed Jan 31 05:20:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.626},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.6749999999999998,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":0.753}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":518889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54000000000000004,"15min":0.51200000000000001},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.76600000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.504},"min":{"1min":0.39200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82199999999999995,"5min":1.617,"15min":1.617},"last":0.42999999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57499999999999996,"15min":0.53500000000000003},"min":{"1min":0.45000000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76300000000000001,"5min":1.04,"15min":1.04},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.57899999999999996,"15min":0.53400000000000003},"min":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88300000000000001,"5min":1.661,"15min":1.661},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.58199999999999996,"15min":0.56799999999999995},"min":{"1min":0.42599999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.61599999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.88200000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.44800000000000001,"15min":0.42399999999999999},"min":{"1min":0.39600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.48099999999999998,"15min":0.45900000000000002},"min":{"1min":0.40000000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45300000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.60299999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.45800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.51000000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.746,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.878,"15min":0.878},"last":0.66800000000000004}]},{"osd":8,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57399999999999995,"15min":0.57699999999999996},"min":{"1min":0.40600000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.65900000000000003}]},{"osd":9,"last update":"Wed Jan 31 05:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.52200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63600000000000001}]},{"osd":10,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.60699999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.59299999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.84499999999999997}]},{"osd":11,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.79800000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":450475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,14,6,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.48699999999999999,"15min":0.373},"min":{"1min":0.28399999999999997,"5min":0.189,"15min":0.189},"max":{"1min":10.337999999999999,"5min":10.337999999999999,"15min":10.337999999999999},"last":10.337999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.40200000000000002,"15min":0.34399999999999997},"min":{"1min":0.27900000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.0430000000000001,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":3.0430000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:21:46 2024","interfaces":[{"interface":"back","average":{"1min":2.0449999999999999,"5min":0.871,"15min":0.64500000000000002},"min":{"1min":0.45400000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":23.84,"5min":23.84,"15min":23.84},"last":23.84},{"interface":"front","average":{"1min":1.651,"5min":0.77800000000000002,"15min":0.60999999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":17.474,"5min":17.474,"15min":17.474},"last":17.474}]},{"osd":2,"last update":"Wed Jan 31 05:21:46 2024","interfaces":[{"interface":"back","average":{"1min":2.0089999999999999,"5min":0.86699999999999999,"15min":0.64400000000000002},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":23.823,"5min":23.823,"15min":23.823},"last":23.823},{"interface":"front","average":{"1min":1.6779999999999999,"5min":0.78700000000000003,"15min":0.60899999999999999},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":18.242000000000001,"5min":18.242000000000001,"15min":18.242000000000001},"last":18.242000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:21:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.35099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.80900000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":15.603},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.876,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":20.015999999999998}]},{"osd":5,"last update":"Wed Jan 31 05:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.38900000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.76100000000000001,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":21.213000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.875,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":19.803000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48299999999999998},"min":{"1min":0.29599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70899999999999996,"5min":0.872,"15min":0.872},"last":19.306000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":19.219000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82499999999999996,"5min":0.88,"15min":0.88},"last":17.434000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.55000000000000004,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85199999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":15.638}]},{"osd":8,"last update":"Wed Jan 31 05:20:35 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":0.92300000000000004,"15min":0.77700000000000002},"min":{"1min":0.504,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":17.98,"5min":17.98,"15min":17.98},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.82699999999999996}]},{"osd":9,"last update":"Wed Jan 31 05:21:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66800000000000004,"15min":0.67300000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":14.872},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92300000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":13.359}]},{"osd":10,"last update":"Wed Jan 31 05:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.77800000000000002,"15min":0.79400000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.82199999999999995,"5min":1.137,"15min":1.137},"last":22.329999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.78000000000000003,"15min":0.79400000000000004},"min":{"1min":0.58499999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90400000000000003,"5min":1.014,"15min":1.014},"last":18.007999999999999}]},{"osd":11,"last update":"Wed Jan 31 05:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":14.914999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":10.461}]}]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40432,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704720,"statfs":{"total":95995035648,"available":95953633280,"internally_reserved":0,"allocated":958464,"data_stored":459023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53400000000000003,"15min":0.50800000000000001},"min":{"1min":0.38400000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78900000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.443},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.49199999999999999},"min":{"1min":0.45200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.753,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.43099999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.497},{"interface":"front","average":{"1min":0.437,"5min":0.41199999999999998,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.72899999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.47499999999999998}]},{"osd":2,"last update":"Wed Jan 31 05:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.48299999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78900000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70299999999999996}]},{"osd":3,"last update":"Wed Jan 31 05:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72899999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.747}]},{"osd":5,"last update":"Wed Jan 31 05:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.45900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.622}]},{"osd":6,"last update":"Wed Jan 31 05:20:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.73199999999999998}]},{"osd":7,"last update":"Wed Jan 31 05:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.45000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.879,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.93000000000000005}]},{"osd":8,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.54700000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93799999999999994,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.57699999999999996,"5min":0.495,"15min":0.495},"max":{"1min":0.92500000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.85799999999999998}]},{"osd":9,"last update":"Wed Jan 31 05:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.76400000000000001},"min":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.81699999999999995}]},{"osd":10,"last update":"Wed Jan 31 05:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.58499999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.57399999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.89400000000000002}]},{"osd":11,"last update":"Wed Jan 31 05:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.71599999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1572864,"data_stored":1050340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.53300000000000003,"15min":0.498},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.38600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82299999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.58399999999999996},"min":{"1min":0.53500000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.80400000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57399999999999995,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69699999999999995}]},{"osd":2,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.41299999999999998,"15min":0.39600000000000002},"min":{"1min":0.32400000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.67700000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.41199999999999998,"15min":0.39000000000000001},"min":{"1min":0.34499999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.65900000000000003,"5min":0.75,"15min":0.75},"last":0.621}]},{"osd":3,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86599999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.65400000000000003}]},{"osd":4,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.60699999999999998,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82099999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.5,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.78700000000000003}]},{"osd":6,"last update":"Wed Jan 31 05:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60799999999999998,"15min":0.59499999999999997},"min":{"1min":0.47499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63100000000000001,"15min":0.61699999999999999},"min":{"1min":0.49199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.79900000000000004}]},{"osd":7,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65500000000000003,"15min":0.65000000000000002},"min":{"1min":0.47399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.77200000000000002}]},{"osd":8,"last update":"Wed Jan 31 05:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.495,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.60699999999999998,"15min":0.59399999999999997},"min":{"1min":0.52000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.67000000000000004}]},{"osd":9,"last update":"Wed Jan 31 05:21:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69499999999999995,"15min":0.69199999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.81499999999999995}]},{"osd":10,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.745,"15min":0.73299999999999998},"min":{"1min":0.627,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.65700000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.83799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:21:54.527 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph pg dump --format=json 2024-01-31T05:21:54.774 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:55.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:55 smithi045 ceph-mon[31585]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:21:55.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:55 smithi045 ceph-mon[31585]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-01-31T05:21:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:55 smithi156 ceph-mon[35720]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:21:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:55 smithi156 ceph-mon[35720]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-01-31T05:21:56.045 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:55 smithi186 ceph-mon[35659]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:21:56.045 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:55 smithi186 ceph-mon[35659]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-01-31T05:21:56.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.445+0000 7f1651e18700 1 -- 172.21.15.45:0/2995769791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c06a8e0 msgr2=0x7f164c10e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:56.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.445+0000 7f1651e18700 1 --2- 172.21.15.45:0/2995769791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c06a8e0 0x7f164c10e730 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f164400b0d0 tx=0x7f164400b3e0 comp rx=0 tx=0).stop 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.445+0000 7f1651e18700 1 -- 172.21.15.45:0/2995769791 shutdown_connections 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.445+0000 7f1651e18700 1 --2- 172.21.15.45:0/2995769791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c06a8e0 0x7f164c10e730 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.445+0000 7f1651e18700 1 --2- 172.21.15.45:0/2995769791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164c106850 0x7f164c106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.445+0000 7f1651e18700 1 --2- 172.21.15.45:0/2995769791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f164c069f20 0x7f164c06a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.445+0000 7f1651e18700 1 -- 172.21.15.45:0/2995769791 >> 172.21.15.45:0/2995769791 conn(0x7f164c077540 msgr2=0x7f164c077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.445+0000 7f1651e18700 1 -- 172.21.15.45:0/2995769791 shutdown_connections 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.445+0000 7f1651e18700 1 -- 172.21.15.45:0/2995769791 wait complete. 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.446+0000 7f1651e18700 1 Processor -- start 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.446+0000 7f1651e18700 1 -- start start 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.446+0000 7f1651e18700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164c069f20 0x7f164c100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.446+0000 7f1651e18700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f164c06a8e0 0x7f164c100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.446+0000 7f1651e18700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c106850 0x7f164c101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.446+0000 7f1651e18700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c1a62b0 con 0x7f164c106850 2024-01-31T05:21:56.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.446+0000 7f1651e18700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c1a6420 con 0x7f164c069f20 2024-01-31T05:21:56.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.446+0000 7f1651e18700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c1a65e0 con 0x7f164c06a8e0 2024-01-31T05:21:56.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.446+0000 7f164bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c106850 0x7f164c101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:56.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.446+0000 7f164bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c106850 0x7f164c101510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44564/0 (socket says 172.21.15.45:44564) 2024-01-31T05:21:56.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.446+0000 7f164bfff700 1 -- 172.21.15.45:0/2374815797 learned_addr learned my addr 172.21.15.45:0/2374815797 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:56.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f164b7fe700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164c069f20 0x7f164c100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:56.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f1642ffd700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f164c06a8e0 0x7f164c100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:56.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f164bfff700 1 -- 172.21.15.45:0/2374815797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f164c06a8e0 msgr2=0x7f164c100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:56.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f164bfff700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f164c06a8e0 0x7f164c100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:56.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f164bfff700 1 -- 172.21.15.45:0/2374815797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164c069f20 msgr2=0x7f164c100a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:56.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f164bfff700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164c069f20 0x7f164c100a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:56.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f164bfff700 1 -- 172.21.15.45:0/2374815797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1644009d00 con 0x7f164c106850 2024-01-31T05:21:56.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f164bfff700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c106850 0x7f164c101510 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1644007fd0 tx=0x7f16440094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:56.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f16497fa700 1 -- 172.21.15.45:0/2374815797 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164400e040 con 0x7f164c106850 2024-01-31T05:21:56.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f1651e18700 1 -- 172.21.15.45:0/2374815797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164c1a6860 con 0x7f164c106850 2024-01-31T05:21:56.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f16497fa700 1 -- 172.21.15.45:0/2374815797 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1644005050 con 0x7f164c106850 2024-01-31T05:21:56.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f16497fa700 1 -- 172.21.15.45:0/2374815797 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164401b690 con 0x7f164c106850 2024-01-31T05:21:56.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.447+0000 7f1651e18700 1 -- 172.21.15.45:0/2374815797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164c1a6d20 con 0x7f164c106850 2024-01-31T05:21:56.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.448+0000 7f1651e18700 1 -- 172.21.15.45:0/2374815797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164c109dd0 con 0x7f164c106850 2024-01-31T05:21:56.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.449+0000 7f16497fa700 1 -- 172.21.15.45:0/2374815797 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f1644004a50 con 0x7f164c106850 2024-01-31T05:21:56.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.450+0000 7f16497fa700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f16380a1370 0x7f16380a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:56.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.450+0000 7f164b7fe700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f16380a1370 0x7f16380a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:56.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.450+0000 7f16497fa700 1 -- 172.21.15.45:0/2374815797 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f16440cdf20 con 0x7f164c106850 2024-01-31T05:21:56.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.450+0000 7f164b7fe700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f16380a1370 0x7f16380a3830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f163c009a10 tx=0x7f163c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:56.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.454+0000 7f16497fa700 1 -- 172.21.15.45:0/2374815797 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1644097400 con 0x7f164c106850 2024-01-31T05:21:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.651+0000 7f1651e18700 1 -- 172.21.15.45:0/2374815797 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f164c101fc0 con 0x7f16380a1370 2024-01-31T05:21:56.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.662+0000 7f16497fa700 1 -- 172.21.15.45:0/2374815797 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344895 (secure 0 0 0) 0x7f164c101fc0 con 0x7f16380a1370 2024-01-31T05:21:56.666 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:21:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.666+0000 7f1651e18700 1 -- 172.21.15.45:0/2374815797 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f16380a1370 msgr2=0x7f16380a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.666+0000 7f1651e18700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f16380a1370 0x7f16380a3830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f163c009a10 tx=0x7f163c009190 comp rx=0 tx=0).stop 2024-01-31T05:21:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.666+0000 7f1651e18700 1 -- 172.21.15.45:0/2374815797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c106850 msgr2=0x7f164c101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.666+0000 7f1651e18700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c106850 0x7f164c101510 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1644007fd0 tx=0x7f16440094c0 comp rx=0 tx=0).stop 2024-01-31T05:21:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.666+0000 7f1651e18700 1 -- 172.21.15.45:0/2374815797 shutdown_connections 2024-01-31T05:21:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.666+0000 7f1651e18700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f16380a1370 0x7f16380a3830 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.666+0000 7f1651e18700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c106850 0x7f164c101510 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.666+0000 7f1651e18700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f164c069f20 0x7f164c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.666+0000 7f1651e18700 1 --2- 172.21.15.45:0/2374815797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f164c06a8e0 0x7f164c100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.666+0000 7f1651e18700 1 -- 172.21.15.45:0/2374815797 >> 172.21.15.45:0/2374815797 conn(0x7f164c077540 msgr2=0x7f164c0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.666+0000 7f1651e18700 1 -- 172.21.15.45:0/2374815797 shutdown_connections 2024-01-31T05:21:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:56.666+0000 7f1651e18700 1 -- 172.21.15.45:0/2374815797 wait complete. 2024-01-31T05:21:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-01-31T05:21:57.348 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-01-31T05:21:56.609283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653372,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":583,"num_read_kb":410,"num_write":339,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":773,"ondisk_log_size":773,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404076,"kb_used_data":13260,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537748,"statfs":{"total":1151940427776,"available":1151526653952,"internally_reserved":0,"allocated":13578240,"data_stored":7460664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.926324"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689535+0000","last_change":"2024-01-31T05:20:40.720465+0000","last_active":"2024-01-31T05:21:47.689535+0000","last_peered":"2024-01-31T05:21:47.689535+0000","last_clean":"2024-01-31T05:21:47.689535+0000","last_became_active":"2024-01-31T05:20:40.717615+0000","last_became_peered":"2024-01-31T05:20:40.717615+0000","last_unstale":"2024-01-31T05:21:47.689535+0000","last_undegraded":"2024-01-31T05:21:47.689535+0000","last_fullsized":"2024-01-31T05:21:47.689535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:30:01.564811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691731+0000","last_change":"2024-01-31T05:20:40.729011+0000","last_active":"2024-01-31T05:21:47.691731+0000","last_peered":"2024-01-31T05:21:47.691731+0000","last_clean":"2024-01-31T05:21:47.691731+0000","last_became_active":"2024-01-31T05:20:40.728021+0000","last_became_peered":"2024-01-31T05:20:40.728021+0000","last_unstale":"2024-01-31T05:21:47.691731+0000","last_undegraded":"2024-01-31T05:21:47.691731+0000","last_fullsized":"2024-01-31T05:21:47.691731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:23:06.641571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691590+0000","last_change":"2024-01-31T05:20:40.729136+0000","last_active":"2024-01-31T05:21:47.691590+0000","last_peered":"2024-01-31T05:21:47.691590+0000","last_clean":"2024-01-31T05:21:47.691590+0000","last_became_active":"2024-01-31T05:20:40.728273+0000","last_became_peered":"2024-01-31T05:20:40.728273+0000","last_unstale":"2024-01-31T05:21:47.691590+0000","last_undegraded":"2024-01-31T05:21:47.691590+0000","last_fullsized":"2024-01-31T05:21:47.691590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:17:43.625443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692995+0000","last_change":"2024-01-31T05:21:47.692585+0000","last_active":"2024-01-31T05:21:47.692995+0000","last_peered":"2024-01-31T05:21:47.692995+0000","last_clean":"2024-01-31T05:21:47.692995+0000","last_became_active":"2024-01-31T05:21:47.692150+0000","last_became_peered":"2024-01-31T05:21:47.692150+0000","last_unstale":"2024-01-31T05:21:47.692995+0000","last_undegraded":"2024-01-31T05:21:47.692995+0000","last_fullsized":"2024-01-31T05:21:47.692995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:56:06.261960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688566+0000","last_change":"2024-01-31T05:20:40.724551+0000","last_active":"2024-01-31T05:21:47.688566+0000","last_peered":"2024-01-31T05:21:47.688566+0000","last_clean":"2024-01-31T05:21:47.688566+0000","last_became_active":"2024-01-31T05:20:40.724172+0000","last_became_peered":"2024-01-31T05:20:40.724172+0000","last_unstale":"2024-01-31T05:21:47.688566+0000","last_undegraded":"2024-01-31T05:21:47.688566+0000","last_fullsized":"2024-01-31T05:21:47.688566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:45:28.748481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895954+0000","last_change":"2024-01-31T05:20:40.721982+0000","last_active":"2024-01-31T05:21:47.895954+0000","last_peered":"2024-01-31T05:21:47.895954+0000","last_clean":"2024-01-31T05:21:47.895954+0000","last_became_active":"2024-01-31T05:20:40.721649+0000","last_became_peered":"2024-01-31T05:20:40.721649+0000","last_unstale":"2024-01-31T05:21:47.895954+0000","last_undegraded":"2024-01-31T05:21:47.895954+0000","last_fullsized":"2024-01-31T05:21:47.895954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:47:32.776711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686428+0000","last_change":"2024-01-31T05:20:40.718340+0000","last_active":"2024-01-31T05:21:47.686428+0000","last_peered":"2024-01-31T05:21:47.686428+0000","last_clean":"2024-01-31T05:21:47.686428+0000","last_became_active":"2024-01-31T05:20:40.718017+0000","last_became_peered":"2024-01-31T05:20:40.718017+0000","last_unstale":"2024-01-31T05:21:47.686428+0000","last_undegraded":"2024-01-31T05:21:47.686428+0000","last_fullsized":"2024-01-31T05:21:47.686428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:17:45.606831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695366+0000","last_change":"2024-01-31T05:20:40.720271+0000","last_active":"2024-01-31T05:21:47.695366+0000","last_peered":"2024-01-31T05:21:47.695366+0000","last_clean":"2024-01-31T05:21:47.695366+0000","last_became_active":"2024-01-31T05:20:40.719960+0000","last_became_peered":"2024-01-31T05:20:40.719960+0000","last_unstale":"2024-01-31T05:21:47.695366+0000","last_undegraded":"2024-01-31T05:21:47.695366+0000","last_fullsized":"2024-01-31T05:21:47.695366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:59:16.243268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689929+0000","last_change":"2024-01-31T05:20:40.725779+0000","last_active":"2024-01-31T05:21:47.689929+0000","last_peered":"2024-01-31T05:21:47.689929+0000","last_clean":"2024-01-31T05:21:47.689929+0000","last_became_active":"2024-01-31T05:20:40.725423+0000","last_became_peered":"2024-01-31T05:20:40.725423+0000","last_unstale":"2024-01-31T05:21:47.689929+0000","last_undegraded":"2024-01-31T05:21:47.689929+0000","last_fullsized":"2024-01-31T05:21:47.689929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:11:46.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.896186+0000","last_change":"2024-01-31T05:20:40.723144+0000","last_active":"2024-01-31T05:21:47.896186+0000","last_peered":"2024-01-31T05:21:47.896186+0000","last_clean":"2024-01-31T05:21:47.896186+0000","last_became_active":"2024-01-31T05:20:40.722806+0000","last_became_peered":"2024-01-31T05:20:40.722806+0000","last_unstale":"2024-01-31T05:21:47.896186+0000","last_undegraded":"2024-01-31T05:21:47.896186+0000","last_fullsized":"2024-01-31T05:21:47.896186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:33.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686816+0000","last_change":"2024-01-31T05:20:40.721083+0000","last_active":"2024-01-31T05:21:47.686816+0000","last_peered":"2024-01-31T05:21:47.686816+0000","last_clean":"2024-01-31T05:21:47.686816+0000","last_became_active":"2024-01-31T05:20:40.720773+0000","last_became_peered":"2024-01-31T05:20:40.720773+0000","last_unstale":"2024-01-31T05:21:47.686816+0000","last_undegraded":"2024-01-31T05:21:47.686816+0000","last_fullsized":"2024-01-31T05:21:47.686816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:39:08.114126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692642+0000","last_change":"2024-01-31T05:20:40.723630+0000","last_active":"2024-01-31T05:21:47.692642+0000","last_peered":"2024-01-31T05:21:47.692642+0000","last_clean":"2024-01-31T05:21:47.692642+0000","last_became_active":"2024-01-31T05:20:40.723340+0000","last_became_peered":"2024-01-31T05:20:40.723340+0000","last_unstale":"2024-01-31T05:21:47.692642+0000","last_undegraded":"2024-01-31T05:21:47.692642+0000","last_fullsized":"2024-01-31T05:21:47.692642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:53:21.199111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895922+0000","last_change":"2024-01-31T05:20:40.712602+0000","last_active":"2024-01-31T05:21:47.895922+0000","last_peered":"2024-01-31T05:21:47.895922+0000","last_clean":"2024-01-31T05:21:47.895922+0000","last_became_active":"2024-01-31T05:20:40.712261+0000","last_became_peered":"2024-01-31T05:20:40.712261+0000","last_unstale":"2024-01-31T05:21:47.895922+0000","last_undegraded":"2024-01-31T05:21:47.895922+0000","last_fullsized":"2024-01-31T05:21:47.895922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:30:05.638679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691375+0000","last_change":"2024-01-31T05:20:40.723759+0000","last_active":"2024-01-31T05:21:47.691375+0000","last_peered":"2024-01-31T05:21:47.691375+0000","last_clean":"2024-01-31T05:21:47.691375+0000","last_became_active":"2024-01-31T05:20:40.723421+0000","last_became_peered":"2024-01-31T05:20:40.723421+0000","last_unstale":"2024-01-31T05:21:47.691375+0000","last_undegraded":"2024-01-31T05:21:47.691375+0000","last_fullsized":"2024-01-31T05:21:47.691375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:02:27.348273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691456+0000","last_change":"2024-01-31T05:20:40.726482+0000","last_active":"2024-01-31T05:21:47.691456+0000","last_peered":"2024-01-31T05:21:47.691456+0000","last_clean":"2024-01-31T05:21:47.691456+0000","last_became_active":"2024-01-31T05:20:40.726086+0000","last_became_peered":"2024-01-31T05:20:40.726086+0000","last_unstale":"2024-01-31T05:21:47.691456+0000","last_undegraded":"2024-01-31T05:21:47.691456+0000","last_fullsized":"2024-01-31T05:21:47.691456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:02:28.142740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689383+0000","last_change":"2024-01-31T05:20:40.728363+0000","last_active":"2024-01-31T05:21:47.689383+0000","last_peered":"2024-01-31T05:21:47.689383+0000","last_clean":"2024-01-31T05:21:47.689383+0000","last_became_active":"2024-01-31T05:20:40.727795+0000","last_became_peered":"2024-01-31T05:20:40.727795+0000","last_unstale":"2024-01-31T05:21:47.689383+0000","last_undegraded":"2024-01-31T05:21:47.689383+0000","last_fullsized":"2024-01-31T05:21:47.689383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:43:08.123159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689646+0000","last_change":"2024-01-31T05:20:40.720902+0000","last_active":"2024-01-31T05:21:47.689646+0000","last_peered":"2024-01-31T05:21:47.689646+0000","last_clean":"2024-01-31T05:21:47.689646+0000","last_became_active":"2024-01-31T05:20:40.720554+0000","last_became_peered":"2024-01-31T05:20:40.720554+0000","last_unstale":"2024-01-31T05:21:47.689646+0000","last_undegraded":"2024-01-31T05:21:47.689646+0000","last_fullsized":"2024-01-31T05:21:47.689646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:56:49.441368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.685578+0000","last_change":"2024-01-31T05:20:40.710504+0000","last_active":"2024-01-31T05:21:47.685578+0000","last_peered":"2024-01-31T05:21:47.685578+0000","last_clean":"2024-01-31T05:21:47.685578+0000","last_became_active":"2024-01-31T05:20:40.710162+0000","last_became_peered":"2024-01-31T05:20:40.710162+0000","last_unstale":"2024-01-31T05:21:47.685578+0000","last_undegraded":"2024-01-31T05:21:47.685578+0000","last_fullsized":"2024-01-31T05:21:47.685578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:49:49.930573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688084+0000","last_change":"2024-01-31T05:20:40.729023+0000","last_active":"2024-01-31T05:21:47.688084+0000","last_peered":"2024-01-31T05:21:47.688084+0000","last_clean":"2024-01-31T05:21:47.688084+0000","last_became_active":"2024-01-31T05:20:40.728672+0000","last_became_peered":"2024-01-31T05:20:40.728672+0000","last_unstale":"2024-01-31T05:21:47.688084+0000","last_undegraded":"2024-01-31T05:21:47.688084+0000","last_fullsized":"2024-01-31T05:21:47.688084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:56:41.578440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.685542+0000","last_change":"2024-01-31T05:20:39.716771+0000","last_active":"2024-01-31T05:21:47.685542+0000","last_peered":"2024-01-31T05:21:47.685542+0000","last_clean":"2024-01-31T05:21:47.685542+0000","last_became_active":"2024-01-31T05:20:39.716439+0000","last_became_peered":"2024-01-31T05:20:39.716439+0000","last_unstale":"2024-01-31T05:21:47.685542+0000","last_undegraded":"2024-01-31T05:21:47.685542+0000","last_fullsized":"2024-01-31T05:21:47.685542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:14:24.317293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695802+0000","last_change":"2024-01-31T05:20:40.730695+0000","last_active":"2024-01-31T05:21:47.695802+0000","last_peered":"2024-01-31T05:21:47.695802+0000","last_clean":"2024-01-31T05:21:47.695802+0000","last_became_active":"2024-01-31T05:20:40.730343+0000","last_became_peered":"2024-01-31T05:20:40.730343+0000","last_unstale":"2024-01-31T05:21:47.695802+0000","last_undegraded":"2024-01-31T05:21:47.695802+0000","last_fullsized":"2024-01-31T05:21:47.695802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:32.308239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695968+0000","last_change":"2024-01-31T05:20:39.711082+0000","last_active":"2024-01-31T05:21:47.695968+0000","last_peered":"2024-01-31T05:21:47.695968+0000","last_clean":"2024-01-31T05:21:47.695968+0000","last_became_active":"2024-01-31T05:20:39.710741+0000","last_became_peered":"2024-01-31T05:20:39.710741+0000","last_unstale":"2024-01-31T05:21:47.695968+0000","last_undegraded":"2024-01-31T05:21:47.695968+0000","last_fullsized":"2024-01-31T05:21:47.695968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:20:01.934837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691726+0000","last_change":"2024-01-31T05:20:40.729269+0000","last_active":"2024-01-31T05:21:47.691726+0000","last_peered":"2024-01-31T05:21:47.691726+0000","last_clean":"2024-01-31T05:21:47.691726+0000","last_became_active":"2024-01-31T05:20:40.728910+0000","last_became_peered":"2024-01-31T05:20:40.728910+0000","last_unstale":"2024-01-31T05:21:47.691726+0000","last_undegraded":"2024-01-31T05:21:47.691726+0000","last_fullsized":"2024-01-31T05:21:47.691726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:00:24.711108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.703286+0000","last_change":"2024-01-31T05:21:47.702095+0000","last_active":"2024-01-31T05:21:47.703286+0000","last_peered":"2024-01-31T05:21:47.703286+0000","last_clean":"2024-01-31T05:21:47.703286+0000","last_became_active":"2024-01-31T05:21:47.692164+0000","last_became_peered":"2024-01-31T05:21:47.692164+0000","last_unstale":"2024-01-31T05:21:47.703286+0000","last_undegraded":"2024-01-31T05:21:47.703286+0000","last_fullsized":"2024-01-31T05:21:47.703286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:31.642391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689477+0000","last_change":"2024-01-31T05:20:40.727934+0000","last_active":"2024-01-31T05:21:47.689477+0000","last_peered":"2024-01-31T05:21:47.689477+0000","last_clean":"2024-01-31T05:21:47.689477+0000","last_became_active":"2024-01-31T05:20:40.727517+0000","last_became_peered":"2024-01-31T05:20:40.727517+0000","last_unstale":"2024-01-31T05:21:47.689477+0000","last_undegraded":"2024-01-31T05:21:47.689477+0000","last_fullsized":"2024-01-31T05:21:47.689477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:35:46.329596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691357+0000","last_change":"2024-01-31T05:20:39.709541+0000","last_active":"2024-01-31T05:21:47.691357+0000","last_peered":"2024-01-31T05:21:47.691357+0000","last_clean":"2024-01-31T05:21:47.691357+0000","last_became_active":"2024-01-31T05:20:39.708918+0000","last_became_peered":"2024-01-31T05:20:39.708918+0000","last_unstale":"2024-01-31T05:21:47.691357+0000","last_undegraded":"2024-01-31T05:21:47.691357+0000","last_fullsized":"2024-01-31T05:21:47.691357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:55:51.545144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.693894+0000","last_change":"2024-01-31T05:20:40.729677+0000","last_active":"2024-01-31T05:21:47.693894+0000","last_peered":"2024-01-31T05:21:47.693894+0000","last_clean":"2024-01-31T05:21:47.693894+0000","last_became_active":"2024-01-31T05:20:40.727880+0000","last_became_peered":"2024-01-31T05:20:40.727880+0000","last_unstale":"2024-01-31T05:21:47.693894+0000","last_undegraded":"2024-01-31T05:21:47.693894+0000","last_fullsized":"2024-01-31T05:21:47.693894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:49:55.935842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:50.574828+0000","last_change":"2024-01-31T05:20:39.711310+0000","last_active":"2024-01-31T05:21:50.574828+0000","last_peered":"2024-01-31T05:21:50.574828+0000","last_clean":"2024-01-31T05:21:50.574828+0000","last_became_active":"2024-01-31T05:20:39.710108+0000","last_became_peered":"2024-01-31T05:20:39.710108+0000","last_unstale":"2024-01-31T05:21:50.574828+0000","last_undegraded":"2024-01-31T05:21:50.574828+0000","last_fullsized":"2024-01-31T05:21:50.574828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:03:25.550777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689420+0000","last_change":"2024-01-31T05:20:40.722095+0000","last_active":"2024-01-31T05:21:47.689420+0000","last_peered":"2024-01-31T05:21:47.689420+0000","last_clean":"2024-01-31T05:21:47.689420+0000","last_became_active":"2024-01-31T05:20:40.721778+0000","last_became_peered":"2024-01-31T05:20:40.721778+0000","last_unstale":"2024-01-31T05:21:47.689420+0000","last_undegraded":"2024-01-31T05:21:47.689420+0000","last_fullsized":"2024-01-31T05:21:47.689420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:48:04.284745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895807+0000","last_change":"2024-01-31T05:20:39.705704+0000","last_active":"2024-01-31T05:21:47.895807+0000","last_peered":"2024-01-31T05:21:47.895807+0000","last_clean":"2024-01-31T05:21:47.895807+0000","last_became_active":"2024-01-31T05:20:39.705304+0000","last_became_peered":"2024-01-31T05:20:39.705304+0000","last_unstale":"2024-01-31T05:21:47.895807+0000","last_undegraded":"2024-01-31T05:21:47.895807+0000","last_fullsized":"2024-01-31T05:21:47.895807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:01:40.116085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.897314+0000","last_change":"2024-01-31T05:21:47.897314+0000","last_active":"2024-01-31T05:21:47.897314+0000","last_peered":"2024-01-31T05:21:47.897314+0000","last_clean":"2024-01-31T05:21:47.897314+0000","last_became_active":"2024-01-31T05:21:47.896873+0000","last_became_peered":"2024-01-31T05:21:47.896873+0000","last_unstale":"2024-01-31T05:21:47.897314+0000","last_undegraded":"2024-01-31T05:21:47.897314+0000","last_fullsized":"2024-01-31T05:21:47.897314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:56:25.953686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895551+0000","last_change":"2024-01-31T05:20:39.705157+0000","last_active":"2024-01-31T05:21:47.895551+0000","last_peered":"2024-01-31T05:21:47.895551+0000","last_clean":"2024-01-31T05:21:47.895551+0000","last_became_active":"2024-01-31T05:20:39.704794+0000","last_became_peered":"2024-01-31T05:20:39.704794+0000","last_unstale":"2024-01-31T05:21:47.895551+0000","last_undegraded":"2024-01-31T05:21:47.895551+0000","last_fullsized":"2024-01-31T05:21:47.895551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:21:20.314515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.696058+0000","last_change":"2024-01-31T05:20:40.723906+0000","last_active":"2024-01-31T05:21:47.696058+0000","last_peered":"2024-01-31T05:21:47.696058+0000","last_clean":"2024-01-31T05:21:47.696058+0000","last_became_active":"2024-01-31T05:20:40.723612+0000","last_became_peered":"2024-01-31T05:20:40.723612+0000","last_unstale":"2024-01-31T05:21:47.696058+0000","last_undegraded":"2024-01-31T05:21:47.696058+0000","last_fullsized":"2024-01-31T05:21:47.696058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:42:42.509983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691991+0000","last_change":"2024-01-31T05:20:39.709315+0000","last_active":"2024-01-31T05:21:47.691991+0000","last_peered":"2024-01-31T05:21:47.691991+0000","last_clean":"2024-01-31T05:21:47.691991+0000","last_became_active":"2024-01-31T05:20:39.709033+0000","last_became_peered":"2024-01-31T05:20:39.709033+0000","last_unstale":"2024-01-31T05:21:47.691991+0000","last_undegraded":"2024-01-31T05:21:47.691991+0000","last_fullsized":"2024-01-31T05:21:47.691991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:43:29.540029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.685448+0000","last_change":"2024-01-31T05:20:40.717396+0000","last_active":"2024-01-31T05:21:47.685448+0000","last_peered":"2024-01-31T05:21:47.685448+0000","last_clean":"2024-01-31T05:21:47.685448+0000","last_became_active":"2024-01-31T05:20:40.717076+0000","last_became_peered":"2024-01-31T05:20:40.717076+0000","last_unstale":"2024-01-31T05:21:47.685448+0000","last_undegraded":"2024-01-31T05:21:47.685448+0000","last_fullsized":"2024-01-31T05:21:47.685448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:42:51.667070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:54.579497+0000","last_change":"2024-01-31T05:20:39.716940+0000","last_active":"2024-01-31T05:21:54.579497+0000","last_peered":"2024-01-31T05:21:54.579497+0000","last_clean":"2024-01-31T05:21:54.579497+0000","last_became_active":"2024-01-31T05:20:39.716691+0000","last_became_peered":"2024-01-31T05:20:39.716691+0000","last_unstale":"2024-01-31T05:21:54.579497+0000","last_undegraded":"2024-01-31T05:21:54.579497+0000","last_fullsized":"2024-01-31T05:21:54.579497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:38:23.786807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689798+0000","last_change":"2024-01-31T05:20:40.727618+0000","last_active":"2024-01-31T05:21:47.689798+0000","last_peered":"2024-01-31T05:21:47.689798+0000","last_clean":"2024-01-31T05:21:47.689798+0000","last_became_active":"2024-01-31T05:20:40.727282+0000","last_became_peered":"2024-01-31T05:20:40.727282+0000","last_unstale":"2024-01-31T05:21:47.689798+0000","last_undegraded":"2024-01-31T05:21:47.689798+0000","last_fullsized":"2024-01-31T05:21:47.689798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:07:12.416575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686860+0000","last_change":"2024-01-31T05:20:39.709584+0000","last_active":"2024-01-31T05:21:47.686860+0000","last_peered":"2024-01-31T05:21:47.686860+0000","last_clean":"2024-01-31T05:21:47.686860+0000","last_became_active":"2024-01-31T05:20:39.708660+0000","last_became_peered":"2024-01-31T05:20:39.708660+0000","last_unstale":"2024-01-31T05:21:47.686860+0000","last_undegraded":"2024-01-31T05:21:47.686860+0000","last_fullsized":"2024-01-31T05:21:47.686860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:21:06.450913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691837+0000","last_change":"2024-01-31T05:20:40.734040+0000","last_active":"2024-01-31T05:21:47.691837+0000","last_peered":"2024-01-31T05:21:47.691837+0000","last_clean":"2024-01-31T05:21:47.691837+0000","last_became_active":"2024-01-31T05:20:40.733307+0000","last_became_peered":"2024-01-31T05:20:40.733307+0000","last_unstale":"2024-01-31T05:21:47.691837+0000","last_undegraded":"2024-01-31T05:21:47.691837+0000","last_fullsized":"2024-01-31T05:21:47.691837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:31:01.584900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.693932+0000","last_change":"2024-01-31T05:20:39.708004+0000","last_active":"2024-01-31T05:21:47.693932+0000","last_peered":"2024-01-31T05:21:47.693932+0000","last_clean":"2024-01-31T05:21:47.693932+0000","last_became_active":"2024-01-31T05:20:39.707681+0000","last_became_peered":"2024-01-31T05:20:39.707681+0000","last_unstale":"2024-01-31T05:21:47.693932+0000","last_undegraded":"2024-01-31T05:21:47.693932+0000","last_fullsized":"2024-01-31T05:21:47.693932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:24:10.003803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689601+0000","last_change":"2024-01-31T05:20:40.714449+0000","last_active":"2024-01-31T05:21:47.689601+0000","last_peered":"2024-01-31T05:21:47.689601+0000","last_clean":"2024-01-31T05:21:47.689601+0000","last_became_active":"2024-01-31T05:20:40.714064+0000","last_became_peered":"2024-01-31T05:20:40.714064+0000","last_unstale":"2024-01-31T05:21:47.689601+0000","last_undegraded":"2024-01-31T05:21:47.689601+0000","last_fullsized":"2024-01-31T05:21:47.689601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:17:04.803852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":200,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:55.580475+0000","last_change":"2024-01-31T05:20:39.709050+0000","last_active":"2024-01-31T05:21:55.580475+0000","last_peered":"2024-01-31T05:21:55.580475+0000","last_clean":"2024-01-31T05:21:55.580475+0000","last_became_active":"2024-01-31T05:20:39.708674+0000","last_became_peered":"2024-01-31T05:20:39.708674+0000","last_unstale":"2024-01-31T05:21:55.580475+0000","last_undegraded":"2024-01-31T05:21:55.580475+0000","last_fullsized":"2024-01-31T05:21:55.580475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:08:34.994683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59160,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":152,"num_read_kb":133,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688902+0000","last_change":"2024-01-31T05:20:40.727191+0000","last_active":"2024-01-31T05:21:47.688902+0000","last_peered":"2024-01-31T05:21:47.688902+0000","last_clean":"2024-01-31T05:21:47.688902+0000","last_became_active":"2024-01-31T05:20:40.725114+0000","last_became_peered":"2024-01-31T05:20:40.725114+0000","last_unstale":"2024-01-31T05:21:47.688902+0000","last_undegraded":"2024-01-31T05:21:47.688902+0000","last_fullsized":"2024-01-31T05:21:47.688902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:00.758702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688113+0000","last_change":"2024-01-31T05:20:39.719349+0000","last_active":"2024-01-31T05:21:47.688113+0000","last_peered":"2024-01-31T05:21:47.688113+0000","last_clean":"2024-01-31T05:21:47.688113+0000","last_became_active":"2024-01-31T05:20:39.718918+0000","last_became_peered":"2024-01-31T05:20:39.718918+0000","last_unstale":"2024-01-31T05:21:47.688113+0000","last_undegraded":"2024-01-31T05:21:47.688113+0000","last_fullsized":"2024-01-31T05:21:47.688113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:47:48.521638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895635+0000","last_change":"2024-01-31T05:20:40.716868+0000","last_active":"2024-01-31T05:21:47.895635+0000","last_peered":"2024-01-31T05:21:47.895635+0000","last_clean":"2024-01-31T05:21:47.895635+0000","last_became_active":"2024-01-31T05:20:40.716519+0000","last_became_peered":"2024-01-31T05:20:40.716519+0000","last_unstale":"2024-01-31T05:21:47.895635+0000","last_undegraded":"2024-01-31T05:21:47.895635+0000","last_fullsized":"2024-01-31T05:21:47.895635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:21:05.365719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692637+0000","last_change":"2024-01-31T05:20:39.716631+0000","last_active":"2024-01-31T05:21:47.692637+0000","last_peered":"2024-01-31T05:21:47.692637+0000","last_clean":"2024-01-31T05:21:47.692637+0000","last_became_active":"2024-01-31T05:20:39.716331+0000","last_became_peered":"2024-01-31T05:20:39.716331+0000","last_unstale":"2024-01-31T05:21:47.692637+0000","last_undegraded":"2024-01-31T05:21:47.692637+0000","last_fullsized":"2024-01-31T05:21:47.692637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:05:59.555000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686625+0000","last_change":"2024-01-31T05:20:40.721548+0000","last_active":"2024-01-31T05:21:47.686625+0000","last_peered":"2024-01-31T05:21:47.686625+0000","last_clean":"2024-01-31T05:21:47.686625+0000","last_became_active":"2024-01-31T05:20:40.720296+0000","last_became_peered":"2024-01-31T05:20:40.720296+0000","last_unstale":"2024-01-31T05:21:47.686625+0000","last_undegraded":"2024-01-31T05:21:47.686625+0000","last_fullsized":"2024-01-31T05:21:47.686625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:04:08.406863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:51.575524+0000","last_change":"2024-01-31T05:20:39.707644+0000","last_active":"2024-01-31T05:21:51.575524+0000","last_peered":"2024-01-31T05:21:51.575524+0000","last_clean":"2024-01-31T05:21:51.575524+0000","last_became_active":"2024-01-31T05:20:39.707334+0000","last_became_peered":"2024-01-31T05:20:39.707334+0000","last_unstale":"2024-01-31T05:21:51.575524+0000","last_undegraded":"2024-01-31T05:21:51.575524+0000","last_fullsized":"2024-01-31T05:21:51.575524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:13:25.630415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686277+0000","last_change":"2024-01-31T05:20:40.715835+0000","last_active":"2024-01-31T05:21:47.686277+0000","last_peered":"2024-01-31T05:21:47.686277+0000","last_clean":"2024-01-31T05:21:47.686277+0000","last_became_active":"2024-01-31T05:20:40.715467+0000","last_became_peered":"2024-01-31T05:20:40.715467+0000","last_unstale":"2024-01-31T05:21:47.686277+0000","last_undegraded":"2024-01-31T05:21:47.686277+0000","last_fullsized":"2024-01-31T05:21:47.686277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:40:33.583238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688427+0000","last_change":"2024-01-31T05:20:39.717929+0000","last_active":"2024-01-31T05:21:47.688427+0000","last_peered":"2024-01-31T05:21:47.688427+0000","last_clean":"2024-01-31T05:21:47.688427+0000","last_became_active":"2024-01-31T05:20:39.717644+0000","last_became_peered":"2024-01-31T05:20:39.717644+0000","last_unstale":"2024-01-31T05:21:47.688427+0000","last_undegraded":"2024-01-31T05:21:47.688427+0000","last_fullsized":"2024-01-31T05:21:47.688427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:24:41.296706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692360+0000","last_change":"2024-01-31T05:20:40.726182+0000","last_active":"2024-01-31T05:21:47.692360+0000","last_peered":"2024-01-31T05:21:47.692360+0000","last_clean":"2024-01-31T05:21:47.692360+0000","last_became_active":"2024-01-31T05:20:40.725858+0000","last_became_peered":"2024-01-31T05:20:40.725858+0000","last_unstale":"2024-01-31T05:21:47.692360+0000","last_undegraded":"2024-01-31T05:21:47.692360+0000","last_fullsized":"2024-01-31T05:21:47.692360+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:27.209744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689293+0000","last_change":"2024-01-31T05:20:39.706132+0000","last_active":"2024-01-31T05:21:47.689293+0000","last_peered":"2024-01-31T05:21:47.689293+0000","last_clean":"2024-01-31T05:21:47.689293+0000","last_became_active":"2024-01-31T05:20:39.705791+0000","last_became_peered":"2024-01-31T05:20:39.705791+0000","last_unstale":"2024-01-31T05:21:47.689293+0000","last_undegraded":"2024-01-31T05:21:47.689293+0000","last_fullsized":"2024-01-31T05:21:47.689293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:54:57.857011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.696014+0000","last_change":"2024-01-31T05:21:47.696014+0000","last_active":"2024-01-31T05:21:47.696014+0000","last_peered":"2024-01-31T05:21:47.696014+0000","last_clean":"2024-01-31T05:21:47.696014+0000","last_became_active":"2024-01-31T05:21:47.695012+0000","last_became_peered":"2024-01-31T05:21:47.695012+0000","last_unstale":"2024-01-31T05:21:47.696014+0000","last_undegraded":"2024-01-31T05:21:47.696014+0000","last_fullsized":"2024-01-31T05:21:47.696014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:58:49.034777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.685513+0000","last_change":"2024-01-31T05:20:39.710195+0000","last_active":"2024-01-31T05:21:47.685513+0000","last_peered":"2024-01-31T05:21:47.685513+0000","last_clean":"2024-01-31T05:21:47.685513+0000","last_became_active":"2024-01-31T05:20:39.709787+0000","last_became_peered":"2024-01-31T05:20:39.709787+0000","last_unstale":"2024-01-31T05:21:47.685513+0000","last_undegraded":"2024-01-31T05:21:47.685513+0000","last_fullsized":"2024-01-31T05:21:47.685513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:28:36.283468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.896149+0000","last_change":"2024-01-31T05:20:40.716243+0000","last_active":"2024-01-31T05:21:47.896149+0000","last_peered":"2024-01-31T05:21:47.896149+0000","last_clean":"2024-01-31T05:21:47.896149+0000","last_became_active":"2024-01-31T05:20:40.715927+0000","last_became_peered":"2024-01-31T05:20:40.715927+0000","last_unstale":"2024-01-31T05:21:47.896149+0000","last_undegraded":"2024-01-31T05:21:47.896149+0000","last_fullsized":"2024-01-31T05:21:47.896149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:20:19.289014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686547+0000","last_change":"2024-01-31T05:20:39.712333+0000","last_active":"2024-01-31T05:21:47.686547+0000","last_peered":"2024-01-31T05:21:47.686547+0000","last_clean":"2024-01-31T05:21:47.686547+0000","last_became_active":"2024-01-31T05:20:39.712008+0000","last_became_peered":"2024-01-31T05:20:39.712008+0000","last_unstale":"2024-01-31T05:21:47.686547+0000","last_undegraded":"2024-01-31T05:21:47.686547+0000","last_fullsized":"2024-01-31T05:21:47.686547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:04:52.962130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.896096+0000","last_change":"2024-01-31T05:20:40.725131+0000","last_active":"2024-01-31T05:21:47.896096+0000","last_peered":"2024-01-31T05:21:47.896096+0000","last_clean":"2024-01-31T05:21:47.896096+0000","last_became_active":"2024-01-31T05:20:40.724797+0000","last_became_peered":"2024-01-31T05:20:40.724797+0000","last_unstale":"2024-01-31T05:21:47.896096+0000","last_undegraded":"2024-01-31T05:21:47.896096+0000","last_fullsized":"2024-01-31T05:21:47.896096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:23:26.535023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692221+0000","last_change":"2024-01-31T05:20:40.726857+0000","last_active":"2024-01-31T05:21:47.692221+0000","last_peered":"2024-01-31T05:21:47.692221+0000","last_clean":"2024-01-31T05:21:47.692221+0000","last_became_active":"2024-01-31T05:20:40.726545+0000","last_became_peered":"2024-01-31T05:20:40.726545+0000","last_unstale":"2024-01-31T05:21:47.692221+0000","last_undegraded":"2024-01-31T05:21:47.692221+0000","last_fullsized":"2024-01-31T05:21:47.692221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:43:55.852474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:50.573500+0000","last_change":"2024-01-31T05:20:39.713920+0000","last_active":"2024-01-31T05:21:50.573500+0000","last_peered":"2024-01-31T05:21:50.573500+0000","last_clean":"2024-01-31T05:21:50.573500+0000","last_became_active":"2024-01-31T05:20:39.713582+0000","last_became_peered":"2024-01-31T05:20:39.713582+0000","last_unstale":"2024-01-31T05:21:50.573500+0000","last_undegraded":"2024-01-31T05:21:50.573500+0000","last_fullsized":"2024-01-31T05:21:50.573500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:20:18.932993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695600+0000","last_change":"2024-01-31T05:20:40.720442+0000","last_active":"2024-01-31T05:21:47.695600+0000","last_peered":"2024-01-31T05:21:47.695600+0000","last_clean":"2024-01-31T05:21:47.695600+0000","last_became_active":"2024-01-31T05:20:40.720114+0000","last_became_peered":"2024-01-31T05:20:40.720114+0000","last_unstale":"2024-01-31T05:21:47.695600+0000","last_undegraded":"2024-01-31T05:21:47.695600+0000","last_fullsized":"2024-01-31T05:21:47.695600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:11:01.536820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692617+0000","last_change":"2024-01-31T05:20:39.706791+0000","last_active":"2024-01-31T05:21:47.692617+0000","last_peered":"2024-01-31T05:21:47.692617+0000","last_clean":"2024-01-31T05:21:47.692617+0000","last_became_active":"2024-01-31T05:20:39.706447+0000","last_became_peered":"2024-01-31T05:20:39.706447+0000","last_unstale":"2024-01-31T05:21:47.692617+0000","last_undegraded":"2024-01-31T05:21:47.692617+0000","last_fullsized":"2024-01-31T05:21:47.692617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:41:05.118138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686420+0000","last_change":"2024-01-31T05:19:28.446661+0000","last_active":"2024-01-31T05:21:47.686420+0000","last_peered":"2024-01-31T05:21:47.686420+0000","last_clean":"2024-01-31T05:21:47.686420+0000","last_became_active":"2024-01-31T05:19:28.446238+0000","last_became_peered":"2024-01-31T05:19:28.446238+0000","last_unstale":"2024-01-31T05:21:47.686420+0000","last_undegraded":"2024-01-31T05:21:47.686420+0000","last_fullsized":"2024-01-31T05:21:47.686420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:16:11.415995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:16:11.415995+0000","last_clean_scrub_stamp":"2024-01-31T05:16:11.415995+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:56:53.665942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895436+0000","last_change":"2024-01-31T05:20:40.714343+0000","last_active":"2024-01-31T05:21:47.895436+0000","last_peered":"2024-01-31T05:21:47.895436+0000","last_clean":"2024-01-31T05:21:47.895436+0000","last_became_active":"2024-01-31T05:20:40.713820+0000","last_became_peered":"2024-01-31T05:20:40.713820+0000","last_unstale":"2024-01-31T05:21:47.895436+0000","last_undegraded":"2024-01-31T05:21:47.895436+0000","last_fullsized":"2024-01-31T05:21:47.895436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:44:16.525005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686643+0000","last_change":"2024-01-31T05:20:39.712068+0000","last_active":"2024-01-31T05:21:47.686643+0000","last_peered":"2024-01-31T05:21:47.686643+0000","last_clean":"2024-01-31T05:21:47.686643+0000","last_became_active":"2024-01-31T05:20:39.711641+0000","last_became_peered":"2024-01-31T05:20:39.711641+0000","last_unstale":"2024-01-31T05:21:47.686643+0000","last_undegraded":"2024-01-31T05:21:47.686643+0000","last_fullsized":"2024-01-31T05:21:47.686643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:27.199351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689032+0000","last_change":"2024-01-31T05:20:39.716003+0000","last_active":"2024-01-31T05:21:47.689032+0000","last_peered":"2024-01-31T05:21:47.689032+0000","last_clean":"2024-01-31T05:21:47.689032+0000","last_became_active":"2024-01-31T05:20:39.715653+0000","last_became_peered":"2024-01-31T05:20:39.715653+0000","last_unstale":"2024-01-31T05:21:47.689032+0000","last_undegraded":"2024-01-31T05:21:47.689032+0000","last_fullsized":"2024-01-31T05:21:47.689032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:45:56.886077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895762+0000","last_change":"2024-01-31T05:20:40.712298+0000","last_active":"2024-01-31T05:21:47.895762+0000","last_peered":"2024-01-31T05:21:47.895762+0000","last_clean":"2024-01-31T05:21:47.895762+0000","last_became_active":"2024-01-31T05:20:40.711907+0000","last_became_peered":"2024-01-31T05:20:40.711907+0000","last_unstale":"2024-01-31T05:21:47.895762+0000","last_undegraded":"2024-01-31T05:21:47.895762+0000","last_fullsized":"2024-01-31T05:21:47.895762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:04:13.482003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.685696+0000","last_change":"2024-01-31T05:20:39.720639+0000","last_active":"2024-01-31T05:21:47.685696+0000","last_peered":"2024-01-31T05:21:47.685696+0000","last_clean":"2024-01-31T05:21:47.685696+0000","last_became_active":"2024-01-31T05:20:39.720302+0000","last_became_peered":"2024-01-31T05:20:39.720302+0000","last_unstale":"2024-01-31T05:21:47.685696+0000","last_undegraded":"2024-01-31T05:21:47.685696+0000","last_fullsized":"2024-01-31T05:21:47.685696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:11:31.507908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695719+0000","last_change":"2024-01-31T05:20:40.725324+0000","last_active":"2024-01-31T05:21:47.695719+0000","last_peered":"2024-01-31T05:21:47.695719+0000","last_clean":"2024-01-31T05:21:47.695719+0000","last_became_active":"2024-01-31T05:20:40.725027+0000","last_became_peered":"2024-01-31T05:20:40.725027+0000","last_unstale":"2024-01-31T05:21:47.695719+0000","last_undegraded":"2024-01-31T05:21:47.695719+0000","last_fullsized":"2024-01-31T05:21:47.695719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:06:24.129276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692814+0000","last_change":"2024-01-31T05:20:39.705805+0000","last_active":"2024-01-31T05:21:47.692814+0000","last_peered":"2024-01-31T05:21:47.692814+0000","last_clean":"2024-01-31T05:21:47.692814+0000","last_became_active":"2024-01-31T05:20:39.705447+0000","last_became_peered":"2024-01-31T05:20:39.705447+0000","last_unstale":"2024-01-31T05:21:47.692814+0000","last_undegraded":"2024-01-31T05:21:47.692814+0000","last_fullsized":"2024-01-31T05:21:47.692814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:24:47.856773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691840+0000","last_change":"2024-01-31T05:20:40.733818+0000","last_active":"2024-01-31T05:21:47.691840+0000","last_peered":"2024-01-31T05:21:47.691840+0000","last_clean":"2024-01-31T05:21:47.691840+0000","last_became_active":"2024-01-31T05:20:40.733272+0000","last_became_peered":"2024-01-31T05:20:40.733272+0000","last_unstale":"2024-01-31T05:21:47.691840+0000","last_undegraded":"2024-01-31T05:21:47.691840+0000","last_fullsized":"2024-01-31T05:21:47.691840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:19:14.144194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.699674+0000","last_change":"2024-01-31T05:21:47.699674+0000","last_active":"2024-01-31T05:21:47.699674+0000","last_peered":"2024-01-31T05:21:47.699674+0000","last_clean":"2024-01-31T05:21:47.699674+0000","last_became_active":"2024-01-31T05:21:47.693473+0000","last_became_peered":"2024-01-31T05:21:47.693473+0000","last_unstale":"2024-01-31T05:21:47.699674+0000","last_undegraded":"2024-01-31T05:21:47.699674+0000","last_fullsized":"2024-01-31T05:21:47.699674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:57:20.931061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688519+0000","last_change":"2024-01-31T05:20:40.722376+0000","last_active":"2024-01-31T05:21:47.688519+0000","last_peered":"2024-01-31T05:21:47.688519+0000","last_clean":"2024-01-31T05:21:47.688519+0000","last_became_active":"2024-01-31T05:20:40.722055+0000","last_became_peered":"2024-01-31T05:20:40.722055+0000","last_unstale":"2024-01-31T05:21:47.688519+0000","last_undegraded":"2024-01-31T05:21:47.688519+0000","last_fullsized":"2024-01-31T05:21:47.688519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:11:47.198866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695502+0000","last_change":"2024-01-31T05:20:39.714498+0000","last_active":"2024-01-31T05:21:47.695502+0000","last_peered":"2024-01-31T05:21:47.695502+0000","last_clean":"2024-01-31T05:21:47.695502+0000","last_became_active":"2024-01-31T05:20:39.714179+0000","last_became_peered":"2024-01-31T05:20:39.714179+0000","last_unstale":"2024-01-31T05:21:47.695502+0000","last_undegraded":"2024-01-31T05:21:47.695502+0000","last_fullsized":"2024-01-31T05:21:47.695502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:42:55.913696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689706+0000","last_change":"2024-01-31T05:20:40.716878+0000","last_active":"2024-01-31T05:21:47.689706+0000","last_peered":"2024-01-31T05:21:47.689706+0000","last_clean":"2024-01-31T05:21:47.689706+0000","last_became_active":"2024-01-31T05:20:40.715998+0000","last_became_peered":"2024-01-31T05:20:40.715998+0000","last_unstale":"2024-01-31T05:21:47.689706+0000","last_undegraded":"2024-01-31T05:21:47.689706+0000","last_fullsized":"2024-01-31T05:21:47.689706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:59:31.030147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.687838+0000","last_change":"2024-01-31T05:20:39.719267+0000","last_active":"2024-01-31T05:21:47.687838+0000","last_peered":"2024-01-31T05:21:47.687838+0000","last_clean":"2024-01-31T05:21:47.687838+0000","last_became_active":"2024-01-31T05:20:39.718964+0000","last_became_peered":"2024-01-31T05:20:39.718964+0000","last_unstale":"2024-01-31T05:21:47.687838+0000","last_undegraded":"2024-01-31T05:21:47.687838+0000","last_fullsized":"2024-01-31T05:21:47.687838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:31.150796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695812+0000","last_change":"2024-01-31T05:20:40.727530+0000","last_active":"2024-01-31T05:21:47.695812+0000","last_peered":"2024-01-31T05:21:47.695812+0000","last_clean":"2024-01-31T05:21:47.695812+0000","last_became_active":"2024-01-31T05:20:40.727016+0000","last_became_peered":"2024-01-31T05:20:40.727016+0000","last_unstale":"2024-01-31T05:21:47.695812+0000","last_undegraded":"2024-01-31T05:21:47.695812+0000","last_fullsized":"2024-01-31T05:21:47.695812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:15:25.851713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688698+0000","last_change":"2024-01-31T05:20:39.711083+0000","last_active":"2024-01-31T05:21:47.688698+0000","last_peered":"2024-01-31T05:21:47.688698+0000","last_clean":"2024-01-31T05:21:47.688698+0000","last_became_active":"2024-01-31T05:20:39.710732+0000","last_became_peered":"2024-01-31T05:20:39.710732+0000","last_unstale":"2024-01-31T05:21:47.688698+0000","last_undegraded":"2024-01-31T05:21:47.688698+0000","last_fullsized":"2024-01-31T05:21:47.688698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:43.846910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686565+0000","last_change":"2024-01-31T05:20:40.723667+0000","last_active":"2024-01-31T05:21:47.686565+0000","last_peered":"2024-01-31T05:21:47.686565+0000","last_clean":"2024-01-31T05:21:47.686565+0000","last_became_active":"2024-01-31T05:20:40.723343+0000","last_became_peered":"2024-01-31T05:20:40.723343+0000","last_unstale":"2024-01-31T05:21:47.686565+0000","last_undegraded":"2024-01-31T05:21:47.686565+0000","last_fullsized":"2024-01-31T05:21:47.686565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:12.983137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688823+0000","last_change":"2024-01-31T05:20:39.717263+0000","last_active":"2024-01-31T05:21:47.688823+0000","last_peered":"2024-01-31T05:21:47.688823+0000","last_clean":"2024-01-31T05:21:47.688823+0000","last_became_active":"2024-01-31T05:20:39.716666+0000","last_became_peered":"2024-01-31T05:20:39.716666+0000","last_unstale":"2024-01-31T05:21:47.688823+0000","last_undegraded":"2024-01-31T05:21:47.688823+0000","last_fullsized":"2024-01-31T05:21:47.688823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:10:41.789503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686865+0000","last_change":"2024-01-31T05:20:40.721728+0000","last_active":"2024-01-31T05:21:47.686865+0000","last_peered":"2024-01-31T05:21:47.686865+0000","last_clean":"2024-01-31T05:21:47.686865+0000","last_became_active":"2024-01-31T05:20:40.717082+0000","last_became_peered":"2024-01-31T05:20:40.717082+0000","last_unstale":"2024-01-31T05:21:47.686865+0000","last_undegraded":"2024-01-31T05:21:47.686865+0000","last_fullsized":"2024-01-31T05:21:47.686865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:15:55.823060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.693723+0000","last_change":"2024-01-31T05:20:39.715450+0000","last_active":"2024-01-31T05:21:47.693723+0000","last_peered":"2024-01-31T05:21:47.693723+0000","last_clean":"2024-01-31T05:21:47.693723+0000","last_became_active":"2024-01-31T05:20:39.715140+0000","last_became_peered":"2024-01-31T05:20:39.715140+0000","last_unstale":"2024-01-31T05:21:47.693723+0000","last_undegraded":"2024-01-31T05:21:47.693723+0000","last_fullsized":"2024-01-31T05:21:47.693723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:34:01.299228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686541+0000","last_change":"2024-01-31T05:20:40.723855+0000","last_active":"2024-01-31T05:21:47.686541+0000","last_peered":"2024-01-31T05:21:47.686541+0000","last_clean":"2024-01-31T05:21:47.686541+0000","last_became_active":"2024-01-31T05:20:40.723532+0000","last_became_peered":"2024-01-31T05:20:40.723532+0000","last_unstale":"2024-01-31T05:21:47.686541+0000","last_undegraded":"2024-01-31T05:21:47.686541+0000","last_fullsized":"2024-01-31T05:21:47.686541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:48:59.700145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895790+0000","last_change":"2024-01-31T05:20:39.713941+0000","last_active":"2024-01-31T05:21:47.895790+0000","last_peered":"2024-01-31T05:21:47.895790+0000","last_clean":"2024-01-31T05:21:47.895790+0000","last_became_active":"2024-01-31T05:20:39.713610+0000","last_became_peered":"2024-01-31T05:20:39.713610+0000","last_unstale":"2024-01-31T05:21:47.895790+0000","last_undegraded":"2024-01-31T05:21:47.895790+0000","last_fullsized":"2024-01-31T05:21:47.895790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:38.680050+0000","last_clean_scrub_stamp":"2024-01-31T05:20:38.680050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:48:13.722639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.686946+0000","last_change":"2024-01-31T05:20:40.716485+0000","last_active":"2024-01-31T05:21:47.686946+0000","last_peered":"2024-01-31T05:21:47.686946+0000","last_clean":"2024-01-31T05:21:47.686946+0000","last_became_active":"2024-01-31T05:20:40.716177+0000","last_became_peered":"2024-01-31T05:20:40.716177+0000","last_unstale":"2024-01-31T05:21:47.686946+0000","last_undegraded":"2024-01-31T05:21:47.686946+0000","last_fullsized":"2024-01-31T05:21:47.686946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:25:37.359617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689813+0000","last_change":"2024-01-31T05:20:40.710376+0000","last_active":"2024-01-31T05:21:47.689813+0000","last_peered":"2024-01-31T05:21:47.689813+0000","last_clean":"2024-01-31T05:21:47.689813+0000","last_became_active":"2024-01-31T05:20:40.710017+0000","last_became_peered":"2024-01-31T05:20:40.710017+0000","last_unstale":"2024-01-31T05:21:47.689813+0000","last_undegraded":"2024-01-31T05:21:47.689813+0000","last_fullsized":"2024-01-31T05:21:47.689813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:48:37.519561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695196+0000","last_change":"2024-01-31T05:21:47.695196+0000","last_active":"2024-01-31T05:21:47.695196+0000","last_peered":"2024-01-31T05:21:47.695196+0000","last_clean":"2024-01-31T05:21:47.695196+0000","last_became_active":"2024-01-31T05:21:47.694661+0000","last_became_peered":"2024-01-31T05:21:47.694661+0000","last_unstale":"2024-01-31T05:21:47.695196+0000","last_undegraded":"2024-01-31T05:21:47.695196+0000","last_fullsized":"2024-01-31T05:21:47.695196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:15:56.062684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.687683+0000","last_change":"2024-01-31T05:20:40.725974+0000","last_active":"2024-01-31T05:21:47.687683+0000","last_peered":"2024-01-31T05:21:47.687683+0000","last_clean":"2024-01-31T05:21:47.687683+0000","last_became_active":"2024-01-31T05:20:40.725586+0000","last_became_peered":"2024-01-31T05:20:40.725586+0000","last_unstale":"2024-01-31T05:21:47.687683+0000","last_undegraded":"2024-01-31T05:21:47.687683+0000","last_fullsized":"2024-01-31T05:21:47.687683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:03:06.413502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691589+0000","last_change":"2024-01-31T05:20:40.733524+0000","last_active":"2024-01-31T05:21:47.691589+0000","last_peered":"2024-01-31T05:21:47.691589+0000","last_clean":"2024-01-31T05:21:47.691589+0000","last_became_active":"2024-01-31T05:20:40.733162+0000","last_became_peered":"2024-01-31T05:20:40.733162+0000","last_unstale":"2024-01-31T05:21:47.691589+0000","last_undegraded":"2024-01-31T05:21:47.691589+0000","last_fullsized":"2024-01-31T05:21:47.691589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:12:03.146533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895685+0000","last_change":"2024-01-31T05:20:40.715460+0000","last_active":"2024-01-31T05:21:47.895685+0000","last_peered":"2024-01-31T05:21:47.895685+0000","last_clean":"2024-01-31T05:21:47.895685+0000","last_became_active":"2024-01-31T05:20:40.715103+0000","last_became_peered":"2024-01-31T05:20:40.715103+0000","last_unstale":"2024-01-31T05:21:47.895685+0000","last_undegraded":"2024-01-31T05:21:47.895685+0000","last_fullsized":"2024-01-31T05:21:47.895685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:41:43.178835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688467+0000","last_change":"2024-01-31T05:20:40.714317+0000","last_active":"2024-01-31T05:21:47.688467+0000","last_peered":"2024-01-31T05:21:47.688467+0000","last_clean":"2024-01-31T05:21:47.688467+0000","last_became_active":"2024-01-31T05:20:40.714008+0000","last_became_peered":"2024-01-31T05:20:40.714008+0000","last_unstale":"2024-01-31T05:21:47.688467+0000","last_undegraded":"2024-01-31T05:21:47.688467+0000","last_fullsized":"2024-01-31T05:21:47.688467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:36:39.740278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688019+0000","last_change":"2024-01-31T05:20:40.710901+0000","last_active":"2024-01-31T05:21:47.688019+0000","last_peered":"2024-01-31T05:21:47.688019+0000","last_clean":"2024-01-31T05:21:47.688019+0000","last_became_active":"2024-01-31T05:20:40.710412+0000","last_became_peered":"2024-01-31T05:20:40.710412+0000","last_unstale":"2024-01-31T05:21:47.688019+0000","last_undegraded":"2024-01-31T05:21:47.688019+0000","last_fullsized":"2024-01-31T05:21:47.688019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:39.966723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.695601+0000","last_change":"2024-01-31T05:20:40.732753+0000","last_active":"2024-01-31T05:21:47.695601+0000","last_peered":"2024-01-31T05:21:47.695601+0000","last_clean":"2024-01-31T05:21:47.695601+0000","last_became_active":"2024-01-31T05:20:40.732330+0000","last_became_peered":"2024-01-31T05:20:40.732330+0000","last_unstale":"2024-01-31T05:21:47.695601+0000","last_undegraded":"2024-01-31T05:21:47.695601+0000","last_fullsized":"2024-01-31T05:21:47.695601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:54:01.933393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.692108+0000","last_change":"2024-01-31T05:20:40.723129+0000","last_active":"2024-01-31T05:21:47.692108+0000","last_peered":"2024-01-31T05:21:47.692108+0000","last_clean":"2024-01-31T05:21:47.692108+0000","last_became_active":"2024-01-31T05:20:40.722796+0000","last_became_peered":"2024-01-31T05:20:40.722796+0000","last_unstale":"2024-01-31T05:21:47.692108+0000","last_undegraded":"2024-01-31T05:21:47.692108+0000","last_fullsized":"2024-01-31T05:21:47.692108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:18.120495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.691136+0000","last_change":"2024-01-31T05:21:47.690719+0000","last_active":"2024-01-31T05:21:47.691136+0000","last_peered":"2024-01-31T05:21:47.691136+0000","last_clean":"2024-01-31T05:21:47.691136+0000","last_became_active":"2024-01-31T05:21:47.690347+0000","last_became_peered":"2024-01-31T05:21:47.690347+0000","last_unstale":"2024-01-31T05:21:47.691136+0000","last_undegraded":"2024-01-31T05:21:47.691136+0000","last_fullsized":"2024-01-31T05:21:47.691136+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:05.607970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.689591+0000","last_change":"2024-01-31T05:20:40.720798+0000","last_active":"2024-01-31T05:21:47.689591+0000","last_peered":"2024-01-31T05:21:47.689591+0000","last_clean":"2024-01-31T05:21:47.689591+0000","last_became_active":"2024-01-31T05:20:40.720393+0000","last_became_peered":"2024-01-31T05:20:40.720393+0000","last_unstale":"2024-01-31T05:21:47.689591+0000","last_undegraded":"2024-01-31T05:21:47.689591+0000","last_fullsized":"2024-01-31T05:21:47.689591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:05:51.442874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.895940+0000","last_change":"2024-01-31T05:20:40.715469+0000","last_active":"2024-01-31T05:21:47.895940+0000","last_peered":"2024-01-31T05:21:47.895940+0000","last_clean":"2024-01-31T05:21:47.895940+0000","last_became_active":"2024-01-31T05:20:40.715124+0000","last_became_peered":"2024-01-31T05:20:40.715124+0000","last_unstale":"2024-01-31T05:21:47.895940+0000","last_undegraded":"2024-01-31T05:21:47.895940+0000","last_fullsized":"2024-01-31T05:21:47.895940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:54:32.258584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-31T05:21:47.688626+0000","last_change":"2024-01-31T05:20:40.719815+0000","last_active":"2024-01-31T05:21:47.688626+0000","last_peered":"2024-01-31T05:21:47.688626+0000","last_clean":"2024-01-31T05:21:47.688626+0000","last_became_active":"2024-01-31T05:20:40.719438+0000","last_became_peered":"2024-01-31T05:20:40.719438+0000","last_unstale":"2024-01-31T05:21:47.688626+0000","last_undegraded":"2024-01-31T05:21:47.688626+0000","last_fullsized":"2024-01-31T05:21:47.688626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:20:39.686754+0000","last_clean_scrub_stamp":"2024-01-31T05:20:39.686754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:24:44.814291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62770,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":447,"num_read_kb":291,"num_write":156,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":188310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":458961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.79100000000000004,"5min":0.877,"15min":0.877},"last":0.749}]},{"osd":1,"last update":"Wed Jan 31 05:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60399999999999998,"15min":0.59299999999999997},"min":{"1min":0.55800000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82099999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.39600000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.61299999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48499999999999999,"15min":0.47599999999999998},"min":{"1min":0.40400000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.33300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.79800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79900000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.77500000000000002}]},{"osd":5,"last update":"Wed Jan 31 05:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.875,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.92200000000000004}]},{"osd":6,"last update":"Wed Jan 31 05:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.63300000000000001,"15min":0.64600000000000002},"min":{"1min":0.434,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83999999999999997,"5min":1.226,"15min":1.226},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.55500000000000005,"5min":0.371,"15min":0.371},"max":{"1min":0.93600000000000005,"5min":1.345,"15min":1.345},"last":0.89800000000000002}]},{"osd":7,"last update":"Wed Jan 31 05:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.1759999999999999,"15min":1.242},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.036,"5min":11.733000000000001,"15min":11.733000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92400000000000004,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.88400000000000001}]},{"osd":8,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.98999999999999999}]},{"osd":9,"last update":"Wed Jan 31 05:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77800000000000002,"15min":0.77100000000000002},"min":{"1min":0.63600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.052,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.79100000000000004,"15min":0.78100000000000003},"min":{"1min":0.69699999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.119,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.0189999999999999}]},{"osd":10,"last update":"Wed Jan 31 05:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.76600000000000001,"15min":0.76500000000000001},"min":{"1min":0.59799999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.95199999999999996,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.80300000000000005,"15min":0.79500000000000004},"min":{"1min":0.66500000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.006,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.002}]}]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41112,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704040,"statfs":{"total":95995035648,"available":95952936960,"internally_reserved":0,"allocated":1646592,"data_stored":1109602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.92600000000000005}]},{"osd":1,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.32400000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69699999999999995,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.504}]},{"osd":2,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.47999999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.46100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.873}]},{"osd":3,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.752,"5min":0.878,"15min":0.878},"last":0.80900000000000005}]},{"osd":4,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.314,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.627},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.78400000000000003,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66200000000000003}]},{"osd":5,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89000000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.82999999999999996}]},{"osd":6,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.498,"5min":0.374,"15min":0.374},"max":{"1min":0.86299999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.74399999999999999,"15min":0.753},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85899999999999999,"5min":1.044,"15min":1.044},"last":0.89900000000000002}]},{"osd":7,"last update":"Wed Jan 31 05:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.439},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.67200000000000004}]},{"osd":8,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91800000000000004,"5min":0.995,"15min":0.995},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.81599999999999995,"15min":0.83099999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.92800000000000005,"5min":1.028,"15min":1.028},"last":0.88800000000000001}]},{"osd":9,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81699999999999995,"15min":0.82799999999999996},"min":{"1min":0.54100000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.88900000000000001,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.84199999999999997,"15min":0.86199999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.90500000000000003,"5min":1.073,"15min":1.073},"last":0.91300000000000003}]},{"osd":11,"last update":"Wed Jan 31 05:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.58899999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.95199999999999996,"5min":1.103,"15min":1.103},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.64300000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.97699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.97699999999999998}]}]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40444,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704708,"statfs":{"total":95995035648,"available":95953620992,"internally_reserved":0,"allocated":966656,"data_stored":459197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.45800000000000002,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.51900000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.29399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81899999999999995,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.373}]},{"osd":1,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93200000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.39000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83199999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.39500000000000002}]},{"osd":2,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0569999999999999,"5min":1.089,"15min":1.089},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.59599999999999997}]},{"osd":3,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.56499999999999995,"15min":0.58299999999999996},"min":{"1min":0.313,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69199999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.318},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75700000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.433}]},{"osd":4,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68400000000000005,"15min":0.69199999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80800000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70499999999999996,"15min":0.71999999999999997},"min":{"1min":0.443,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77700000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.45000000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.71999999999999997,"15min":0.73999999999999999},"min":{"1min":0.48299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77400000000000002,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.78900000000000003,"15min":0.81499999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81499999999999995,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.76000000000000001,"15min":0.78400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94499999999999995,"5min":1.298,"15min":1.298},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.69999999999999996,"15min":0.72099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80000000000000004,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.29499999999999998}]},{"osd":7,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.80800000000000005,"15min":0.82899999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.85399999999999998,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.61399999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.86499999999999999,"5min":1.619,"15min":1.619},"last":0.496}]},{"osd":8,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.80600000000000005,"15min":0.83199999999999996},"min":{"1min":0.48099999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83499999999999996,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1120000000000001,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.624}]},{"osd":10,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.78400000000000003,"15min":0.79700000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.93799999999999994,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79600000000000004,"15min":0.80700000000000005},"min":{"1min":0.57099999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.996,"5min":1.083,"15min":1.083},"last":0.50700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":459117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.78300000000000003,"15min":0.66500000000000004},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83599999999999997,"5min":17.460999999999999,"15min":17.460999999999999},"last":0.89800000000000002}]},{"osd":1,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.59499999999999997,"15min":0.57299999999999995},"min":{"1min":0.47799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.56899999999999995},"min":{"1min":0.48899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.69599999999999995}]},{"osd":2,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72599999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48299999999999998},"min":{"1min":0.38500000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80200000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.42199999999999999}]},{"osd":3,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46700000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.44700000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.80300000000000005}]},{"osd":4,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.504,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.64600000000000002,"15min":0.63100000000000001},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55100000000000005}]},{"osd":5,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57199999999999995,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.85499999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.84599999999999997,"15min":0.72899999999999998},"min":{"1min":0.47699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.093,"5min":17.439,"15min":17.439},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.53300000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.69999999999999996,"15min":0.69099999999999995},"min":{"1min":0.59699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.93500000000000005,"15min":0.81799999999999995},"min":{"1min":0.623,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0349999999999999,"5min":17.420000000000002,"15min":17.420000000000002},"last":0.81799999999999995}]},{"osd":9,"last update":"Wed Jan 31 05:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":1.355,"15min":1.4510000000000001},"min":{"1min":0.58599999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2470000000000001,"5min":17.396999999999998,"15min":17.396999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":1.4079999999999999,"15min":1.508},"min":{"1min":0.60999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.296,"5min":17.489999999999998,"15min":17.489999999999998},"last":0.999}]},{"osd":10,"last update":"Wed Jan 31 05:21:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.68500000000000005,"15min":0.66100000000000003},"min":{"1min":0.60699999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.71699999999999997,"15min":0.69599999999999995},"min":{"1min":0.65500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.91400000000000003}]},{"osd":11,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.86299999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1560576,"data_stored":1050656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.49299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.59199999999999997}]},{"osd":1,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40600000000000003},"min":{"1min":0.30099999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.44,"15min":0.436},"min":{"1min":0.32800000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.69299999999999995}]},{"osd":2,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83899999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60599999999999998,"15min":0.57999999999999996},"min":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85399999999999998,"5min":1.131,"15min":1.131},"last":0.72199999999999998}]},{"osd":3,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.70099999999999996,"15min":0.64200000000000002},"min":{"1min":0.39100000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92300000000000004,"5min":7.883,"15min":7.883},"last":0.61499999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.442},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.34899999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.36499999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60799999999999998,"15min":0.58599999999999997},"min":{"1min":0.52300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84299999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.47599999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95899999999999996,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.75700000000000001,"15min":0.70099999999999996},"min":{"1min":0.49399999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0780000000000001,"5min":7.9299999999999997,"15min":7.9299999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.499}]},{"osd":8,"last update":"Wed Jan 31 05:21:39 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70999999999999996,"15min":0.69899999999999995},"min":{"1min":0.61699999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.95899999999999996,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.70499999999999996,"15min":0.69599999999999995},"min":{"1min":0.61899999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.92900000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.628}]},{"osd":9,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.73199999999999998,"15min":0.747},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":1.02,"15min":1.079},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.044,"5min":7.9020000000000001,"15min":7.9020000000000001},"last":0.63600000000000001}]},{"osd":10,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.751,"15min":0.755},"min":{"1min":0.58599999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91400000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76200000000000001,"15min":0.77100000000000002},"min":{"1min":0.499,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0369999999999999,"5min":1.163,"15min":1.163},"last":0.67900000000000005}]},{"osd":11,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.70999999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40436,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704716,"statfs":{"total":95995035648,"available":95953629184,"internally_reserved":0,"allocated":954368,"data_stored":458586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.29599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.41999999999999998,"15min":0.434},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.438,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.438}]},{"osd":1,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.41999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60999999999999999,"15min":0.59199999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65400000000000003}]},{"osd":3,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.46600000000000003,"15min":0.48599999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.52400000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.5},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.51500000000000001,"15min":0.52200000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.59199999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88300000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66500000000000004,"15min":0.66900000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81299999999999994,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.63900000000000001,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72999999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed Jan 31 05:21:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67500000000000004,"15min":0.65200000000000002},"min":{"1min":0.52500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98099999999999998,"5min":1.169,"15min":1.169},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96699999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.70599999999999996}]},{"osd":8,"last update":"Wed Jan 31 05:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.70399999999999996,"15min":0.72699999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91700000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71799999999999997,"15min":0.72899999999999998},"min":{"1min":0.53300000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76200000000000001}]},{"osd":9,"last update":"Wed Jan 31 05:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72799999999999998,"15min":0.74299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94399999999999995,"5min":1.091,"15min":1.091},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.71499999999999997,"15min":0.73299999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72999999999999998}]},{"osd":10,"last update":"Wed Jan 31 05:21:26 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81499999999999995,"15min":0.82499999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0580000000000001,"5min":1.131,"15min":1.131},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.81699999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.99299999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.747}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":517539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70899999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.63},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.30499999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.433}]},{"osd":2,"last update":"Wed Jan 31 05:21:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.56200000000000006,"15min":0.51700000000000002},"min":{"1min":0.42199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.53500000000000003,"15min":0.50700000000000001},"min":{"1min":0.38700000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61299999999999999}]},{"osd":3,"last update":"Wed Jan 31 05:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.52500000000000002},"min":{"1min":0.497,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.61899999999999999,"15min":0.56100000000000005},"min":{"1min":0.52600000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Wed Jan 31 05:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.44400000000000001,"15min":0.39900000000000002},"min":{"1min":0.30399999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.94999999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.44600000000000001,"15min":0.40400000000000003},"min":{"1min":0.28799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76200000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.40500000000000003}]},{"osd":5,"last update":"Wed Jan 31 05:21:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.63100000000000001,"15min":0.57499999999999996},"min":{"1min":0.58399999999999996,"5min":0.38,"15min":0.38},"max":{"1min":1.014,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.61699999999999999,"15min":0.57299999999999995},"min":{"1min":0.51300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97599999999999998,"5min":1.028,"15min":1.028},"last":0.65500000000000003}]},{"osd":6,"last update":"Wed Jan 31 05:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.60699999999999998,"15min":0.58499999999999996},"min":{"1min":0.46800000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.67500000000000004,"15min":0.64100000000000001},"min":{"1min":0.47399999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Jan 31 05:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56599999999999995,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.41799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55900000000000005}]},{"osd":8,"last update":"Wed Jan 31 05:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.67200000000000004,"15min":0.65100000000000002},"min":{"1min":0.51700000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99099999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.73699999999999999,"15min":0.70599999999999996},"min":{"1min":0.69399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.83999999999999997}]},{"osd":9,"last update":"Wed Jan 31 05:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.73699999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.64300000000000002}]},{"osd":10,"last update":"Wed Jan 31 05:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.75700000000000001,"15min":0.73899999999999999},"min":{"1min":0.60299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72899999999999998}]},{"osd":11,"last update":"Wed Jan 31 05:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78400000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40452,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704700,"statfs":{"total":95995035648,"available":95953612800,"internally_reserved":0,"allocated":974848,"data_stored":459857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.56000000000000005,"15min":0.53000000000000003},"min":{"1min":0.32000000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":0.441},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.60199999999999998,"15min":0.53300000000000003},"min":{"1min":0.377,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":7.835,"5min":7.835,"15min":7.835},"last":0.72199999999999998}]},{"osd":2,"last update":"Wed Jan 31 05:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77000000000000002,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":3.4529999999999998,"5min":18.466999999999999,"15min":18.466999999999999},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Jan 31 05:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.56499999999999995}]},{"osd":4,"last update":"Wed Jan 31 05:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58999999999999997,"15min":0.52300000000000002},"min":{"1min":0.42799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.972,"5min":2.972,"15min":2.972},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.55300000000000005,"15min":0.501},"min":{"1min":0.39800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:21:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.63600000000000001,"15min":0.55200000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":7.9130000000000003,"5min":7.9130000000000003,"15min":7.9130000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81299999999999994,"15min":0.63600000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.359,"5min":18.541,"15min":18.541},"last":0.58999999999999997}]},{"osd":6,"last update":"Wed Jan 31 05:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48799999999999999,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71099999999999997,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.60799999999999998,"15min":0.53200000000000003},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.81799999999999995,"5min":8.6560000000000006,"15min":8.6560000000000006},"last":0.61599999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.56799999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":3.1789999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.63700000000000001,"15min":0.622},"min":{"1min":0.502,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":2.105}]},{"osd":8,"last update":"Wed Jan 31 05:21:39 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.76600000000000001,"15min":0.73299999999999998},"min":{"1min":0.51200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":8.6039999999999992,"5min":8.6039999999999992,"15min":8.6039999999999992},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.67000000000000004}]},{"osd":9,"last update":"Wed Jan 31 05:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66200000000000003,"15min":0.66100000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.77400000000000002,"15min":0.74399999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.7050000000000001,"5min":8.7050000000000001,"15min":8.7050000000000001},"last":0.68799999999999994}]},{"osd":10,"last update":"Wed Jan 31 05:21:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.73299999999999998}]},{"osd":11,"last update":"Wed Jan 31 05:20:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.626},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.6749999999999998,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":0.753}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":518889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54000000000000004,"15min":0.51200000000000001},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.76600000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.504},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.504},"min":{"1min":0.39200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82199999999999995,"5min":1.617,"15min":1.617},"last":0.39900000000000002}]},{"osd":1,"last update":"Wed Jan 31 05:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57499999999999996,"15min":0.53500000000000003},"min":{"1min":0.45000000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76300000000000001,"5min":1.04,"15min":1.04},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.57899999999999996,"15min":0.53400000000000003},"min":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.70399999999999996}]},{"osd":3,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88300000000000001,"5min":1.661,"15min":1.661},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.58199999999999996,"15min":0.56799999999999995},"min":{"1min":0.42599999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72199999999999998}]},{"osd":4,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.85899999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.44800000000000001,"15min":0.42399999999999999},"min":{"1min":0.39600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.48099999999999998,"15min":0.45900000000000002},"min":{"1min":0.40000000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.748}]},{"osd":6,"last update":"Wed Jan 31 05:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.60299999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.45800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.60699999999999998}]},{"osd":7,"last update":"Wed Jan 31 05:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.746,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.878,"15min":0.878},"last":0.66800000000000004}]},{"osd":8,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57399999999999995,"15min":0.57699999999999996},"min":{"1min":0.40600000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.63400000000000001}]},{"osd":9,"last update":"Wed Jan 31 05:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.52200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58899999999999997}]},{"osd":10,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.60699999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.59299999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.88100000000000001}]},{"osd":11,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.79900000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":458897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.48699999999999999,"15min":0.373},"min":{"1min":0.28399999999999997,"5min":0.189,"15min":0.189},"max":{"1min":10.337999999999999,"5min":10.337999999999999,"15min":10.337999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.40200000000000002,"15min":0.34399999999999997},"min":{"1min":0.27900000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.0430000000000001,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Wed Jan 31 05:21:46 2024","interfaces":[{"interface":"back","average":{"1min":2.0449999999999999,"5min":0.871,"15min":0.64500000000000002},"min":{"1min":0.45400000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":23.84,"5min":23.84,"15min":23.84},"last":0.46100000000000002},{"interface":"front","average":{"1min":1.651,"5min":0.77800000000000002,"15min":0.60999999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":17.474,"5min":17.474,"15min":17.474},"last":0.501}]},{"osd":2,"last update":"Wed Jan 31 05:21:46 2024","interfaces":[{"interface":"back","average":{"1min":2.0089999999999999,"5min":0.86699999999999999,"15min":0.64400000000000002},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":23.823,"5min":23.823,"15min":23.823},"last":0.57799999999999996},{"interface":"front","average":{"1min":1.6779999999999999,"5min":0.78700000000000003,"15min":0.60899999999999999},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":18.242000000000001,"5min":18.242000000000001,"15min":18.242000000000001},"last":0.40899999999999997}]},{"osd":4,"last update":"Wed Jan 31 05:21:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.35099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.80900000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.876,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.372}]},{"osd":5,"last update":"Wed Jan 31 05:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.38900000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.76100000000000001,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.875,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.41999999999999998}]},{"osd":6,"last update":"Wed Jan 31 05:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48299999999999998},"min":{"1min":0.29599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70899999999999996,"5min":0.872,"15min":0.872},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.434}]},{"osd":7,"last update":"Wed Jan 31 05:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82499999999999996,"5min":0.88,"15min":0.88},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.55000000000000004,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85199999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.53500000000000003}]},{"osd":8,"last update":"Wed Jan 31 05:20:35 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":0.92300000000000004,"15min":0.77700000000000002},"min":{"1min":0.504,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":17.98,"5min":17.98,"15min":17.98},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.82699999999999996}]},{"osd":9,"last update":"Wed Jan 31 05:21:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66800000000000004,"15min":0.67300000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92300000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56299999999999994}]},{"osd":10,"last update":"Wed Jan 31 05:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.77800000000000002,"15min":0.79400000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.82199999999999995,"5min":1.137,"15min":1.137},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.78000000000000003,"15min":0.79400000000000004},"min":{"1min":0.58499999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90400000000000003,"5min":1.014,"15min":1.014},"last":0.64400000000000002}]},{"osd":11,"last update":"Wed Jan 31 05:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.622},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.60799999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888127,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40432,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704720,"statfs":{"total":95995035648,"available":95953633280,"internally_reserved":0,"allocated":958464,"data_stored":459023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53400000000000003,"15min":0.50800000000000001},"min":{"1min":0.38400000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78900000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.443},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.49199999999999999},"min":{"1min":0.45200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.753,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.43099999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.41199999999999998,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.72899999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Jan 31 05:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.48299999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78900000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61799999999999999}]},{"osd":3,"last update":"Wed Jan 31 05:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72899999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.496}]},{"osd":5,"last update":"Wed Jan 31 05:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.45900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed Jan 31 05:20:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.52200000000000002}]},{"osd":7,"last update":"Wed Jan 31 05:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.45000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.879,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.36599999999999999}]},{"osd":8,"last update":"Wed Jan 31 05:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.54700000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93799999999999994,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.57699999999999996,"5min":0.495,"15min":0.495},"max":{"1min":0.92500000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68000000000000005}]},{"osd":9,"last update":"Wed Jan 31 05:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.76400000000000001},"min":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.57699999999999996}]},{"osd":10,"last update":"Wed Jan 31 05:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.58499999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.438},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.57399999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.59099999999999997}]},{"osd":11,"last update":"Wed Jan 31 05:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70999999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1572864,"data_stored":1050340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.53300000000000003,"15min":0.498},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.38600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82299999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.58399999999999996},"min":{"1min":0.53500000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.80400000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57399999999999995,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69699999999999995}]},{"osd":2,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.41299999999999998,"15min":0.39600000000000002},"min":{"1min":0.32400000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.67700000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.41199999999999998,"15min":0.39000000000000001},"min":{"1min":0.34499999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.65900000000000003,"5min":0.75,"15min":0.75},"last":0.621}]},{"osd":3,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86599999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.65400000000000003}]},{"osd":4,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.60699999999999998,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82099999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.5,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.78700000000000003}]},{"osd":6,"last update":"Wed Jan 31 05:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60799999999999998,"15min":0.59499999999999997},"min":{"1min":0.47499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63100000000000001,"15min":0.61699999999999999},"min":{"1min":0.49199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.79900000000000004}]},{"osd":7,"last update":"Wed Jan 31 05:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65500000000000003,"15min":0.65000000000000002},"min":{"1min":0.47399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.77200000000000002}]},{"osd":8,"last update":"Wed Jan 31 05:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.495,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.60699999999999998,"15min":0.59399999999999997},"min":{"1min":0.52000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.67000000000000004}]},{"osd":9,"last update":"Wed Jan 31 05:21:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69499999999999995,"15min":0.69199999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.81499999999999995}]},{"osd":10,"last update":"Wed Jan 31 05:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.745,"15min":0.73299999999999998},"min":{"1min":0.627,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.65700000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.83799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:21:57.355 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-01-31T05:21:57.355 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-31T05:21:57.355 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-01-31T05:21:57.355 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph health --format=json 2024-01-31T05:21:57.600 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:21:57.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:57 smithi156 ceph-mon[35720]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 309 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-01-31T05:21:57.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:57 smithi156 ceph-mon[35720]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:21:57.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:57 smithi045 ceph-mon[31585]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 309 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-01-31T05:21:57.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:57 smithi045 ceph-mon[31585]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:21:58.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:57 smithi186 ceph-mon[35659]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 309 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-01-31T05:21:58.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:57 smithi186 ceph-mon[35659]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-31T05:21:59.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.427+0000 7f68348dd700 1 -- 172.21.15.45:0/193134716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830101080 msgr2=0x7f683010d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:59.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.427+0000 7f68348dd700 1 --2- 172.21.15.45:0/193134716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830101080 0x7f683010d520 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6828009990 tx=0x7f6828009ca0 comp rx=0 tx=0).stop 2024-01-31T05:21:59.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.427+0000 7f68348dd700 1 -- 172.21.15.45:0/193134716 shutdown_connections 2024-01-31T05:21:59.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.427+0000 7f68348dd700 1 --2- 172.21.15.45:0/193134716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830101080 0x7f683010d520 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:59.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.427+0000 7f68348dd700 1 --2- 172.21.15.45:0/193134716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830105590 0x7f6830105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:59.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.427+0000 7f68348dd700 1 --2- 172.21.15.45:0/193134716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68301006c0 0x7f6830100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:59.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.427+0000 7f68348dd700 1 -- 172.21.15.45:0/193134716 >> 172.21.15.45:0/193134716 conn(0x7f68300776f0 msgr2=0x7f6830077b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:59.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.427+0000 7f68348dd700 1 -- 172.21.15.45:0/193134716 shutdown_connections 2024-01-31T05:21:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.427+0000 7f68348dd700 1 -- 172.21.15.45:0/193134716 wait complete. 2024-01-31T05:21:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f68348dd700 1 Processor -- start 2024-01-31T05:21:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f68348dd700 1 -- start start 2024-01-31T05:21:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f68348dd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68301006c0 0x7f683019d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f68348dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830101080 0x7f683019dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f68348dd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830105590 0x7f6830197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f68348dd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301980c0 con 0x7f6830101080 2024-01-31T05:21:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f68348dd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830198200 con 0x7f6830105590 2024-01-31T05:21:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f68348dd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301983c0 con 0x7f68301006c0 2024-01-31T05:21:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f682dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830101080 0x7f683019dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f682dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830101080 0x7f683019dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38992/0 (socket says 172.21.15.45:38992) 2024-01-31T05:21:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f682dd9b700 1 -- 172.21.15.45:0/3718908533 learned_addr learned my addr 172.21.15.45:0/3718908533 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f682ed9d700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830105590 0x7f6830197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f682e59c700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68301006c0 0x7f683019d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f682dd9b700 1 -- 172.21.15.45:0/3718908533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68301006c0 msgr2=0x7f683019d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f682dd9b700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68301006c0 0x7f683019d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f682dd9b700 1 -- 172.21.15.45:0/3718908533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830105590 msgr2=0x7f6830197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f682dd9b700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830105590 0x7f6830197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f682dd9b700 1 -- 172.21.15.45:0/3718908533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68280095f0 con 0x7f6830101080 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.428+0000 7f682e59c700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68301006c0 0x7f683019d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.429+0000 7f682dd9b700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830101080 0x7f683019dde0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f681800b530 tx=0x7f681800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.429+0000 7f68277fe700 1 -- 172.21.15.45:0/3718908533 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818011830 con 0x7f6830101080 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.429+0000 7f68277fe700 1 -- 172.21.15.45:0/3718908533 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6818011e50 con 0x7f6830101080 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.429+0000 7f68277fe700 1 -- 172.21.15.45:0/3718908533 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681800f2e0 con 0x7f6830101080 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.429+0000 7f68348dd700 1 -- 172.21.15.45:0/3718908533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68301986a0 con 0x7f6830101080 2024-01-31T05:21:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.429+0000 7f68348dd700 1 -- 172.21.15.45:0/3718908533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68301a4940 con 0x7f6830101080 2024-01-31T05:21:59.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.430+0000 7f68348dd700 1 -- 172.21.15.45:0/3718908533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6830109c10 con 0x7f6830101080 2024-01-31T05:21:59.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.431+0000 7f68277fe700 1 -- 172.21.15.45:0/3718908533 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f6818010380 con 0x7f6830101080 2024-01-31T05:21:59.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.431+0000 7f68277fe700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f681c0a13f0 0x7f681c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:21:59.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.432+0000 7f682e59c700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f681c0a13f0 0x7f681c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:21:59.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.432+0000 7f68277fe700 1 -- 172.21.15.45:0/3718908533 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f68180ccbd0 con 0x7f6830101080 2024-01-31T05:21:59.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.432+0000 7f682e59c700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f681c0a13f0 0x7f681c0a38b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f68200060b0 tx=0x7f68200076c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:21:59.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.435+0000 7f68277fe700 1 -- 172.21.15.45:0/3718908533 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f6818096080 con 0x7f6830101080 2024-01-31T05:21:59.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.682+0000 7f68348dd700 1 -- 172.21.15.45:0/3718908533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f68300684a0 con 0x7f6830101080 2024-01-31T05:21:59.681 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:21:59 smithi045 ceph-mon[31585]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 281 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-01-31T05:21:59.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.682+0000 7f68277fe700 1 -- 172.21.15.45:0/3718908533 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6818015080 con 0x7f6830101080 2024-01-31T05:21:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:21:59.682 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-31T05:21:59.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.684+0000 7f68348dd700 1 -- 172.21.15.45:0/3718908533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f681c0a13f0 msgr2=0x7f681c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:59.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.684+0000 7f68348dd700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f681c0a13f0 0x7f681c0a38b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f68200060b0 tx=0x7f68200076c0 comp rx=0 tx=0).stop 2024-01-31T05:21:59.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.684+0000 7f68348dd700 1 -- 172.21.15.45:0/3718908533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830101080 msgr2=0x7f683019dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:21:59.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.684+0000 7f68348dd700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830101080 0x7f683019dde0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f681800b530 tx=0x7f681800b840 comp rx=0 tx=0).stop 2024-01-31T05:21:59.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.684+0000 7f68348dd700 1 -- 172.21.15.45:0/3718908533 shutdown_connections 2024-01-31T05:21:59.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.684+0000 7f68348dd700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830105590 0x7f6830197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:59.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.684+0000 7f68348dd700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68301006c0 0x7f683019d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:59.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.684+0000 7f68348dd700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f681c0a13f0 0x7f681c0a38b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:59.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.684+0000 7f68348dd700 1 --2- 172.21.15.45:0/3718908533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830101080 0x7f683019dde0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:21:59.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.684+0000 7f68348dd700 1 -- 172.21.15.45:0/3718908533 >> 172.21.15.45:0/3718908533 conn(0x7f68300776f0 msgr2=0x7f683006a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:21:59.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.685+0000 7f68348dd700 1 -- 172.21.15.45:0/3718908533 shutdown_connections 2024-01-31T05:21:59.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:21:59.685+0000 7f68348dd700 1 -- 172.21.15.45:0/3718908533 wait complete. 2024-01-31T05:21:59.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:21:59 smithi156 ceph-mon[35720]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 281 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-01-31T05:22:00.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:21:59 smithi186 ceph-mon[35659]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 281 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-01-31T05:22:00.393 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-01-31T05:22:00.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:22:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.502+0000 7f1ee71f1640 1 -- 172.21.15.45:0/3737959231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee0104140 msgr2=0x7f1ee0106530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.502+0000 7f1ee71f1640 1 --2- 172.21.15.45:0/3737959231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee0104140 0x7f1ee0106530 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009900 tx=0x7f1ed002bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.502+0000 7f1ee71f1640 1 -- 172.21.15.45:0/3737959231 shutdown_connections 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.502+0000 7f1ee71f1640 1 --2- 172.21.15.45:0/3737959231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee0104140 0x7f1ee0106530 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.502+0000 7f1ee71f1640 1 --2- 172.21.15.45:0/3737959231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee0106a70 0x7f1ee0108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.502+0000 7f1ee71f1640 1 --2- 172.21.15.45:0/3737959231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ee0101810 0x7f1ee0103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.502+0000 7f1ee71f1640 1 -- 172.21.15.45:0/3737959231 >> 172.21.15.45:0/3737959231 conn(0x7f1ee00fb220 msgr2=0x7f1ee00fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.502+0000 7f1ee71f1640 1 -- 172.21.15.45:0/3737959231 shutdown_connections 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.502+0000 7f1ee71f1640 1 -- 172.21.15.45:0/3737959231 wait complete. 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee71f1640 1 Processor -- start 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee71f1640 1 -- start start 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee71f1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee0101810 0x7f1ee0196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee71f1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ee0104140 0x7f1ee0197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee71f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee0106a70 0x7f1ee019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee71f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee019e930 con 0x7f1ee0106a70 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee71f1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee019eaa0 con 0x7f1ee0101810 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee71f1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee019ec60 con 0x7f1ee0104140 2024-01-31T05:22:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee578e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee0106a70 0x7f1ee019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee578e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee0106a70 0x7f1ee019e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39000/0 (socket says 172.21.15.45:39000) 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee578e640 1 -- 172.21.15.45:0/2292820898 learned_addr learned my addr 172.21.15.45:0/2292820898 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee4f8d640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee0101810 0x7f1ee0196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee578e640 1 -- 172.21.15.45:0/2292820898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ee0104140 msgr2=0x7f1ee0197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee578e640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ee0104140 0x7f1ee0197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee578e640 1 -- 172.21.15.45:0/2292820898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee0101810 msgr2=0x7f1ee0196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee578e640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee0101810 0x7f1ee0196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee578e640 1 -- 172.21.15.45:0/2292820898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed0009560 con 0x7f1ee0106a70 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.503+0000 7f1ee4f8d640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee0101810 0x7f1ee0196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.504+0000 7f1ee578e640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee0106a70 0x7f1ee019e3f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009fd0 tx=0x7f1ed800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.504+0000 7f1ed5ffb640 1 -- 172.21.15.45:0/2292820898 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8011830 con 0x7f1ee0106a70 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.504+0000 7f1ed5ffb640 1 -- 172.21.15.45:0/2292820898 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ed8011e50 con 0x7f1ee0106a70 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.504+0000 7f1ee71f1640 1 -- 172.21.15.45:0/2292820898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee019ef40 con 0x7f1ee0106a70 2024-01-31T05:22:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.504+0000 7f1ed5ffb640 1 -- 172.21.15.45:0/2292820898 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed800f2e0 con 0x7f1ee0106a70 2024-01-31T05:22:00.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.504+0000 7f1ee71f1640 1 -- 172.21.15.45:0/2292820898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee019f450 con 0x7f1ee0106a70 2024-01-31T05:22:00.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.505+0000 7f1ee71f1640 1 -- 172.21.15.45:0/2292820898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee00fcd10 con 0x7f1ee0106a70 2024-01-31T05:22:00.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.506+0000 7f1ed5ffb640 1 -- 172.21.15.45:0/2292820898 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f1ed8010580 con 0x7f1ee0106a70 2024-01-31T05:22:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.507+0000 7f1ed5ffb640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ec80a12e0 0x7f1ec80a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.507+0000 7f1ee4f8d640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ec80a12e0 0x7f1ec80a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.507+0000 7f1ed5ffb640 1 -- 172.21.15.45:0/2292820898 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1ed80cc3d0 con 0x7f1ee0106a70 2024-01-31T05:22:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.507+0000 7f1ee4f8d640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ec80a12e0 0x7f1ec80a37a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1edc007eb0 tx=0x7f1edc0073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.511+0000 7f1ed5ffb640 1 -- 172.21.15.45:0/2292820898 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1ed8095880 con 0x7f1ee0106a70 2024-01-31T05:22:00.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.694+0000 7f1ee71f1640 1 -- 172.21.15.45:0/2292820898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ee002cb70 con 0x7f1ee0106a70 2024-01-31T05:22:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.695+0000 7f1ed5ffb640 1 -- 172.21.15.45:0/2292820898 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22520 (secure 0 0 0) 0x7f1ed8098ea0 con 0x7f1ee0106a70 2024-01-31T05:22:00.695 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:00.696 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":88,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:21:47.682334+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:22:00.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.697+0000 7f1ee71f1640 1 -- 172.21.15.45:0/2292820898 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ec80a12e0 msgr2=0x7f1ec80a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.697+0000 7f1ee71f1640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ec80a12e0 0x7f1ec80a37a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1edc007eb0 tx=0x7f1edc0073c0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.697+0000 7f1ee71f1640 1 -- 172.21.15.45:0/2292820898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee0106a70 msgr2=0x7f1ee019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.697+0000 7f1ee71f1640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee0106a70 0x7f1ee019e3f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009fd0 tx=0x7f1ed800b8e0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.697+0000 7f1ee71f1640 1 -- 172.21.15.45:0/2292820898 shutdown_connections 2024-01-31T05:22:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.697+0000 7f1ee71f1640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1ec80a12e0 0x7f1ec80a37a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.697+0000 7f1ee71f1640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee0106a70 0x7f1ee019e3f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.697+0000 7f1ee71f1640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ee0104140 0x7f1ee0197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.697+0000 7f1ee71f1640 1 --2- 172.21.15.45:0/2292820898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee0101810 0x7f1ee0196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.697+0000 7f1ee71f1640 1 -- 172.21.15.45:0/2292820898 >> 172.21.15.45:0/2292820898 conn(0x7f1ee00fb220 msgr2=0x7f1ee0102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.697+0000 7f1ee71f1640 1 -- 172.21.15.45:0/2292820898 shutdown_connections 2024-01-31T05:22:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.697+0000 7f1ee71f1640 1 -- 172.21.15.45:0/2292820898 wait complete. 2024-01-31T05:22:00.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:16:10.856581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:20:38.347599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:20:39.270522+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:22:00.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:22:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.816+0000 7f424c6c5640 1 -- 172.21.15.45:0/3119299971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4244101f00 msgr2=0x7f4244102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.816+0000 7f424c6c5640 1 --2- 172.21.15.45:0/3119299971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4244101f00 0x7f4244102380 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4234009900 tx=0x7f423402bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.816+0000 7f424c6c5640 1 -- 172.21.15.45:0/3119299971 shutdown_connections 2024-01-31T05:22:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.816+0000 7f424c6c5640 1 --2- 172.21.15.45:0/3119299971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4244101f00 0x7f4244102380 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.816+0000 7f424c6c5640 1 --2- 172.21.15.45:0/3119299971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42441028c0 0x7f4244109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.816+0000 7f424c6c5640 1 --2- 172.21.15.45:0/3119299971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4244100d10 0x7f4244101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.816+0000 7f424c6c5640 1 -- 172.21.15.45:0/3119299971 >> 172.21.15.45:0/3119299971 conn(0x7f42440fc2a0 msgr2=0x7f42440fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.816+0000 7f424c6c5640 1 -- 172.21.15.45:0/3119299971 shutdown_connections 2024-01-31T05:22:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.816+0000 7f424c6c5640 1 -- 172.21.15.45:0/3119299971 wait complete. 2024-01-31T05:22:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f424c6c5640 1 Processor -- start 2024-01-31T05:22:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f424c6c5640 1 -- start start 2024-01-31T05:22:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f424c6c5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4244100d10 0x7f4244196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f424c6c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4244101f00 0x7f42441973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f424c6c5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42441028c0 0x7f424419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f424c6c5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424419e970 con 0x7f4244101f00 2024-01-31T05:22:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f424c6c5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424419eae0 con 0x7f42441028c0 2024-01-31T05:22:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f424c6c5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424419eca0 con 0x7f4244100d10 2024-01-31T05:22:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f4249c60640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4244101f00 0x7f42441973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f4249c60640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4244101f00 0x7f42441973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39014/0 (socket says 172.21.15.45:39014) 2024-01-31T05:22:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f4249c60640 1 -- 172.21.15.45:0/1312601828 learned_addr learned my addr 172.21.15.45:0/1312601828 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f424ac62640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42441028c0 0x7f424419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f424a461640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4244100d10 0x7f4244196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f4249c60640 1 -- 172.21.15.45:0/1312601828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4244100d10 msgr2=0x7f4244196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f4249c60640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4244100d10 0x7f4244196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f4249c60640 1 -- 172.21.15.45:0/1312601828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42441028c0 msgr2=0x7f424419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f4249c60640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42441028c0 0x7f424419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f4249c60640 1 -- 172.21.15.45:0/1312601828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4234009560 con 0x7f4244101f00 2024-01-31T05:22:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.817+0000 7f4249c60640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4244101f00 0x7f42441973c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4234009230 tx=0x7f42340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.818+0000 7f423b7fe640 1 -- 172.21.15.45:0/1312601828 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42340044e0 con 0x7f4244101f00 2024-01-31T05:22:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.818+0000 7f424c6c5640 1 -- 172.21.15.45:0/1312601828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424419ef20 con 0x7f4244101f00 2024-01-31T05:22:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.818+0000 7f423b7fe640 1 -- 172.21.15.45:0/1312601828 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42340035f0 con 0x7f4244101f00 2024-01-31T05:22:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.818+0000 7f423b7fe640 1 -- 172.21.15.45:0/1312601828 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423403e6b0 con 0x7f4244101f00 2024-01-31T05:22:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.818+0000 7f424c6c5640 1 -- 172.21.15.45:0/1312601828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424419f420 con 0x7f4244101f00 2024-01-31T05:22:00.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.819+0000 7f424c6c5640 1 -- 172.21.15.45:0/1312601828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42441081a0 con 0x7f4244101f00 2024-01-31T05:22:00.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.820+0000 7f423b7fe640 1 -- 172.21.15.45:0/1312601828 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f4234003960 con 0x7f4244101f00 2024-01-31T05:22:00.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.820+0000 7f423b7fe640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f42300a12e0 0x7f42300a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:00.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.820+0000 7f424a461640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f42300a12e0 0x7f42300a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:00.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.820+0000 7f423b7fe640 1 -- 172.21.15.45:0/1312601828 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f42340efa00 con 0x7f4244101f00 2024-01-31T05:22:00.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.821+0000 7f424a461640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f42300a12e0 0x7f42300a37a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4244101d60 tx=0x7f4240008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:00.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:00.824+0000 7f423b7fe640 1 -- 172.21.15.45:0/1312601828 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f42340b8eb0 con 0x7f4244101f00 2024-01-31T05:22:00.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:00 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3718908533' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-01-31T05:22:01.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.003+0000 7f424c6c5640 1 -- 172.21.15.45:0/1312601828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f424402cb70 con 0x7f4244101f00 2024-01-31T05:22:01.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.004+0000 7f423b7fe640 1 -- 172.21.15.45:0/1312601828 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42340430a0 con 0x7f4244101f00 2024-01-31T05:22:01.004 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-01-31T05:22:01.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.006+0000 7f424c6c5640 1 -- 172.21.15.45:0/1312601828 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f42300a12e0 msgr2=0x7f42300a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.006+0000 7f424c6c5640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f42300a12e0 0x7f42300a37a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4244101d60 tx=0x7f4240008040 comp rx=0 tx=0).stop 2024-01-31T05:22:01.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.006+0000 7f424c6c5640 1 -- 172.21.15.45:0/1312601828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4244101f00 msgr2=0x7f42441973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.006+0000 7f424c6c5640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4244101f00 0x7f42441973c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4234009230 tx=0x7f42340042f0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.006+0000 7f424c6c5640 1 -- 172.21.15.45:0/1312601828 shutdown_connections 2024-01-31T05:22:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.006+0000 7f424c6c5640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f42300a12e0 0x7f42300a37a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.006+0000 7f424c6c5640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4244101f00 0x7f42441973c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.006+0000 7f424c6c5640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42441028c0 0x7f424419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.006+0000 7f424c6c5640 1 --2- 172.21.15.45:0/1312601828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4244100d10 0x7f4244196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.006+0000 7f424c6c5640 1 -- 172.21.15.45:0/1312601828 >> 172.21.15.45:0/1312601828 conn(0x7f42440fc2a0 msgr2=0x7f4244105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.006+0000 7f424c6c5640 1 -- 172.21.15.45:0/1312601828 shutdown_connections 2024-01-31T05:22:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.006+0000 7f424c6c5640 1 -- 172.21.15.45:0/1312601828 wait complete. 2024-01-31T05:22:01.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:00 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3718908533' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-01-31T05:22:01.017 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:22:01.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:00 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3718908533' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-01-31T05:22:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.122+0000 7fb7a81ae640 1 -- 172.21.15.45:0/16286311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7a01040c0 msgr2=0x7fb7a01064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.122+0000 7fb7a81ae640 1 --2- 172.21.15.45:0/16286311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7a01040c0 0x7fb7a01064b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb790009900 tx=0x7fb79002bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.122+0000 7fb7a81ae640 1 -- 172.21.15.45:0/16286311 shutdown_connections 2024-01-31T05:22:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.122+0000 7fb7a81ae640 1 --2- 172.21.15.45:0/16286311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7a01040c0 0x7fb7a01064b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.122+0000 7fb7a81ae640 1 --2- 172.21.15.45:0/16286311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a01069f0 0x7fb7a0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.122+0000 7fb7a81ae640 1 --2- 172.21.15.45:0/16286311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a0101790 0x7fb7a0103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.122+0000 7fb7a81ae640 1 -- 172.21.15.45:0/16286311 >> 172.21.15.45:0/16286311 conn(0x7fb7a00fb1a0 msgr2=0x7fb7a00fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.122+0000 7fb7a81ae640 1 -- 172.21.15.45:0/16286311 shutdown_connections 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.122+0000 7fb7a81ae640 1 -- 172.21.15.45:0/16286311 wait complete. 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.122+0000 7fb7a81ae640 1 Processor -- start 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a81ae640 1 -- start start 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a81ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7a0101790 0x7fb7a0196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a81ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a01040c0 0x7fb7a0197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a81ae640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a01069f0 0x7fb7a019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a81ae640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a019e820 con 0x7fb7a0101790 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a81ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a019e990 con 0x7fb7a01040c0 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a81ae640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a019eb50 con 0x7fb7a01069f0 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a5f4a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7a0101790 0x7fb7a0196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a5f4a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7a0101790 0x7fb7a0196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39030/0 (socket says 172.21.15.45:39030) 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a5f4a640 1 -- 172.21.15.45:0/3729946278 learned_addr learned my addr 172.21.15.45:0/3729946278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a674b640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a01069f0 0x7fb7a019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a5f4a640 1 -- 172.21.15.45:0/3729946278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a01069f0 msgr2=0x7fb7a019e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a5f4a640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a01069f0 0x7fb7a019e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a5f4a640 1 -- 172.21.15.45:0/3729946278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a01040c0 msgr2=0x7fb7a0197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a5749640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a01040c0 0x7fb7a0197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a5f4a640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a01040c0 0x7fb7a0197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a5f4a640 1 -- 172.21.15.45:0/3729946278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb790009560 con 0x7fb7a0101790 2024-01-31T05:22:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a5f4a640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7a0101790 0x7fb7a0196d40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00d950 tx=0x7fb79c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb796ffd640 1 -- 172.21.15.45:0/3729946278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c009c30 con 0x7fb7a0101790 2024-01-31T05:22:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb796ffd640 1 -- 172.21.15.45:0/3729946278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79c009d90 con 0x7fb7a0101790 2024-01-31T05:22:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a81ae640 1 -- 172.21.15.45:0/3729946278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a019ee30 con 0x7fb7a0101790 2024-01-31T05:22:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb796ffd640 1 -- 172.21.15.45:0/3729946278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c00f740 con 0x7fb7a0101790 2024-01-31T05:22:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.123+0000 7fb7a81ae640 1 -- 172.21.15.45:0/3729946278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a019f250 con 0x7fb7a0101790 2024-01-31T05:22:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.124+0000 7fb7a81ae640 1 -- 172.21.15.45:0/3729946278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a00fcc90 con 0x7fb7a0101790 2024-01-31T05:22:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.125+0000 7fb796ffd640 1 -- 172.21.15.45:0/3729946278 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fb79c010a20 con 0x7fb7a0101790 2024-01-31T05:22:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.126+0000 7fb796ffd640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb78c0a12e0 0x7fb78c0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.126+0000 7fb7a5749640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb78c0a12e0 0x7fb78c0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.126+0000 7fb796ffd640 1 -- 172.21.15.45:0/3729946278 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb79c0cc880 con 0x7fb7a0101790 2024-01-31T05:22:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.126+0000 7fb7a5749640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb78c0a12e0 0x7fb78c0a37a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb79002c7e0 tx=0x7fb790000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.130+0000 7fb796ffd640 1 -- 172.21.15.45:0/3729946278 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb79c095ca0 con 0x7fb7a0101790 2024-01-31T05:22:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.309+0000 7fb7a81ae640 1 -- 172.21.15.45:0/3729946278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7a002cc30 con 0x7fb7a0101790 2024-01-31T05:22:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.309+0000 7fb796ffd640 1 -- 172.21.15.45:0/3729946278 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb79c097bd0 con 0x7fb7a0101790 2024-01-31T05:22:01.309 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-01-31T05:22:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.311+0000 7fb7a81ae640 1 -- 172.21.15.45:0/3729946278 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb78c0a12e0 msgr2=0x7fb78c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.311+0000 7fb7a81ae640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb78c0a12e0 0x7fb78c0a37a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb79002c7e0 tx=0x7fb790000bc0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.311+0000 7fb7a81ae640 1 -- 172.21.15.45:0/3729946278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7a0101790 msgr2=0x7fb7a0196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.311+0000 7fb7a81ae640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7a0101790 0x7fb7a0196d40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00d950 tx=0x7fb79c00de10 comp rx=0 tx=0).stop 2024-01-31T05:22:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.311+0000 7fb7a81ae640 1 -- 172.21.15.45:0/3729946278 shutdown_connections 2024-01-31T05:22:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.311+0000 7fb7a81ae640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fb78c0a12e0 0x7fb78c0a37a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.311+0000 7fb7a81ae640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a01069f0 0x7fb7a019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.311+0000 7fb7a81ae640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a01040c0 0x7fb7a0197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.311+0000 7fb7a81ae640 1 --2- 172.21.15.45:0/3729946278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7a0101790 0x7fb7a0196d40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.311+0000 7fb7a81ae640 1 -- 172.21.15.45:0/3729946278 >> 172.21.15.45:0/3729946278 conn(0x7fb7a00fb1a0 msgr2=0x7fb7a0102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.311+0000 7fb7a81ae640 1 -- 172.21.15.45:0/3729946278 shutdown_connections 2024-01-31T05:22:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.311+0000 7fb7a81ae640 1 -- 172.21.15.45:0/3729946278 wait complete. 2024-01-31T05:22:01.322 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:22:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.427+0000 7fe79cdfb640 1 -- 172.21.15.45:0/4233329975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe798102950 msgr2=0x7fe7981091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.427+0000 7fe79cdfb640 1 --2- 172.21.15.45:0/4233329975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe798102950 0x7fe7981091e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe788009ca0 tx=0x7fe78802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.427+0000 7fe79cdfb640 1 -- 172.21.15.45:0/4233329975 shutdown_connections 2024-01-31T05:22:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.427+0000 7fe79cdfb640 1 --2- 172.21.15.45:0/4233329975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe798102950 0x7fe7981091e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.427+0000 7fe79cdfb640 1 --2- 172.21.15.45:0/4233329975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe798101f90 0x7fe798102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.427+0000 7fe79cdfb640 1 --2- 172.21.15.45:0/4233329975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7980fefc0 0x7fe7980ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.427+0000 7fe79cdfb640 1 -- 172.21.15.45:0/4233329975 >> 172.21.15.45:0/4233329975 conn(0x7fe7980fa910 msgr2=0x7fe7980fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.427+0000 7fe79cdfb640 1 -- 172.21.15.45:0/4233329975 shutdown_connections 2024-01-31T05:22:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.427+0000 7fe79cdfb640 1 -- 172.21.15.45:0/4233329975 wait complete. 2024-01-31T05:22:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.428+0000 7fe79cdfb640 1 Processor -- start 2024-01-31T05:22:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.428+0000 7fe79cdfb640 1 -- start start 2024-01-31T05:22:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.428+0000 7fe79cdfb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7980fefc0 0x7fe798101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.428+0000 7fe79cdfb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe798101f90 0x7fe7980fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.428+0000 7fe79cdfb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe798102950 0x7fe798100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.428+0000 7fe79cdfb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe798100be0 con 0x7fe7980fefc0 2024-01-31T05:22:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.428+0000 7fe79cdfb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe798100d50 con 0x7fe798101f90 2024-01-31T05:22:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.428+0000 7fe79cdfb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7981a0cd0 con 0x7fe798102950 2024-01-31T05:22:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.428+0000 7fe79659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7980fefc0 0x7fe798101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.428+0000 7fe79659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7980fefc0 0x7fe798101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39044/0 (socket says 172.21.15.45:39044) 2024-01-31T05:22:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.428+0000 7fe79659c640 1 -- 172.21.15.45:0/1229948706 learned_addr learned my addr 172.21.15.45:0/1229948706 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe795d9b640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe798101f90 0x7fe7980fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe796d9d640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe798102950 0x7fe798100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe79659c640 1 -- 172.21.15.45:0/1229948706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe798102950 msgr2=0x7fe798100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe79659c640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe798102950 0x7fe798100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe79659c640 1 -- 172.21.15.45:0/1229948706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe798101f90 msgr2=0x7fe7980fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe79659c640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe798101f90 0x7fe7980fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe79659c640 1 -- 172.21.15.45:0/1229948706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788009900 con 0x7fe7980fefc0 2024-01-31T05:22:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe796d9d640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe798102950 0x7fe798100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe795d9b640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe798101f90 0x7fe7980fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe79659c640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7980fefc0 0x7fe798101900 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe79000e9f0 tx=0x7fe79000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe78f7fe640 1 -- 172.21.15.45:0/1229948706 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790009700 con 0x7fe7980fefc0 2024-01-31T05:22:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe78f7fe640 1 -- 172.21.15.45:0/1229948706 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7900044e0 con 0x7fe7980fefc0 2024-01-31T05:22:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe79cdfb640 1 -- 172.21.15.45:0/1229948706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7981a0f80 con 0x7fe7980fefc0 2024-01-31T05:22:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe78f7fe640 1 -- 172.21.15.45:0/1229948706 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790010430 con 0x7fe7980fefc0 2024-01-31T05:22:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.429+0000 7fe79cdfb640 1 -- 172.21.15.45:0/1229948706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7981a13a0 con 0x7fe7980fefc0 2024-01-31T05:22:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.430+0000 7fe79cdfb640 1 -- 172.21.15.45:0/1229948706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7981080c0 con 0x7fe7980fefc0 2024-01-31T05:22:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.431+0000 7fe78f7fe640 1 -- 172.21.15.45:0/1229948706 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe79000cd80 con 0x7fe7980fefc0 2024-01-31T05:22:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.432+0000 7fe78f7fe640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe7840a12e0 0x7fe7840a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.432+0000 7fe795d9b640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe7840a12e0 0x7fe7840a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.432+0000 7fe78f7fe640 1 -- 172.21.15.45:0/1229948706 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe790014070 con 0x7fe7980fefc0 2024-01-31T05:22:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.432+0000 7fe795d9b640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe7840a12e0 0x7fe7840a37a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe798101670 tx=0x7fe780005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.436+0000 7fe78f7fe640 1 -- 172.21.15.45:0/1229948706 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe790096400 con 0x7fe7980fefc0 2024-01-31T05:22:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.615+0000 7fe79cdfb640 1 -- 172.21.15.45:0/1229948706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe79810c5d0 con 0x7fe7980fefc0 2024-01-31T05:22:01.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.616+0000 7fe78f7fe640 1 -- 172.21.15.45:0/1229948706 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe790099a20 con 0x7fe7980fefc0 2024-01-31T05:22:01.616 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-01-31T05:22:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.617+0000 7fe79cdfb640 1 -- 172.21.15.45:0/1229948706 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe7840a12e0 msgr2=0x7fe7840a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.617+0000 7fe79cdfb640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe7840a12e0 0x7fe7840a37a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe798101670 tx=0x7fe780005f90 comp rx=0 tx=0).stop 2024-01-31T05:22:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.617+0000 7fe79cdfb640 1 -- 172.21.15.45:0/1229948706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7980fefc0 msgr2=0x7fe798101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.617+0000 7fe79cdfb640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7980fefc0 0x7fe798101900 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe79000e9f0 tx=0x7fe79000eeb0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.617+0000 7fe79cdfb640 1 -- 172.21.15.45:0/1229948706 shutdown_connections 2024-01-31T05:22:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.617+0000 7fe79cdfb640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe7840a12e0 0x7fe7840a37a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.617+0000 7fe79cdfb640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe798102950 0x7fe798100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.618+0000 7fe79cdfb640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe798101f90 0x7fe7980fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.618+0000 7fe79cdfb640 1 --2- 172.21.15.45:0/1229948706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7980fefc0 0x7fe798101900 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.618+0000 7fe79cdfb640 1 -- 172.21.15.45:0/1229948706 >> 172.21.15.45:0/1229948706 conn(0x7fe7980fa910 msgr2=0x7fe798105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.618+0000 7fe79cdfb640 1 -- 172.21.15.45:0/1229948706 shutdown_connections 2024-01-31T05:22:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.618+0000 7fe79cdfb640 1 -- 172.21.15.45:0/1229948706 wait complete. 2024-01-31T05:22:01.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:22:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.731+0000 7efdc3e99640 1 -- 172.21.15.45:0/1258231122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdbc074cb0 msgr2=0x7efdbc073100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.731+0000 7efdc3e99640 1 --2- 172.21.15.45:0/1258231122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdbc074cb0 0x7efdbc073100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009900 tx=0x7efdb402bcd0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.731+0000 7efdc3e99640 1 -- 172.21.15.45:0/1258231122 shutdown_connections 2024-01-31T05:22:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.731+0000 7efdc3e99640 1 --2- 172.21.15.45:0/1258231122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdbc108bf0 0x7efdbc10b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.731+0000 7efdc3e99640 1 --2- 172.21.15.45:0/1258231122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc0736d0 0x7efdbc073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.731+0000 7efdc3e99640 1 --2- 172.21.15.45:0/1258231122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdbc074cb0 0x7efdbc073100 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.731+0000 7efdc3e99640 1 -- 172.21.15.45:0/1258231122 >> 172.21.15.45:0/1258231122 conn(0x7efdbc0fc290 msgr2=0x7efdbc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.731+0000 7efdc3e99640 1 -- 172.21.15.45:0/1258231122 shutdown_connections 2024-01-31T05:22:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc3e99640 1 -- 172.21.15.45:0/1258231122 wait complete. 2024-01-31T05:22:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc3e99640 1 Processor -- start 2024-01-31T05:22:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc3e99640 1 -- start start 2024-01-31T05:22:01.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc3e99640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdbc0736d0 0x7efdbc198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc3e99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdbc074cb0 0x7efdbc1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc3e99640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc108bf0 0x7efdbc1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc3e99640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc1a0950 con 0x7efdbc074cb0 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc3e99640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc1a0ac0 con 0x7efdbc108bf0 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc3e99640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc1a0c80 con 0x7efdbc0736d0 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc1434640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdbc074cb0 0x7efdbc1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc1434640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdbc074cb0 0x7efdbc1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39062/0 (socket says 172.21.15.45:39062) 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc1434640 1 -- 172.21.15.45:0/3867609957 learned_addr learned my addr 172.21.15.45:0/3867609957 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc1c35640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdbc0736d0 0x7efdbc198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.732+0000 7efdc2436640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc108bf0 0x7efdbc1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.733+0000 7efdc1434640 1 -- 172.21.15.45:0/3867609957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdbc0736d0 msgr2=0x7efdbc198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.733+0000 7efdc1434640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdbc0736d0 0x7efdbc198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.733+0000 7efdc1434640 1 -- 172.21.15.45:0/3867609957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc108bf0 msgr2=0x7efdbc1a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.733+0000 7efdc1434640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc108bf0 0x7efdbc1a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.733+0000 7efdc1434640 1 -- 172.21.15.45:0/3867609957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb4009560 con 0x7efdbc074cb0 2024-01-31T05:22:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.733+0000 7efdc1c35640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdbc0736d0 0x7efdbc198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.733+0000 7efdc1434640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdbc074cb0 0x7efdbc1993a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efdb000d950 tx=0x7efdb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.733+0000 7efdbaffd640 1 -- 172.21.15.45:0/3867609957 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0009c30 con 0x7efdbc074cb0 2024-01-31T05:22:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.733+0000 7efdbaffd640 1 -- 172.21.15.45:0/3867609957 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb0009d90 con 0x7efdbc074cb0 2024-01-31T05:22:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.733+0000 7efdc3e99640 1 -- 172.21.15.45:0/3867609957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc1a0f60 con 0x7efdbc074cb0 2024-01-31T05:22:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.733+0000 7efdbaffd640 1 -- 172.21.15.45:0/3867609957 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb000f740 con 0x7efdbc074cb0 2024-01-31T05:22:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.733+0000 7efdc3e99640 1 -- 172.21.15.45:0/3867609957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc1a13e0 con 0x7efdbc074cb0 2024-01-31T05:22:01.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.734+0000 7efdc3e99640 1 -- 172.21.15.45:0/3867609957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc1079b0 con 0x7efdbc074cb0 2024-01-31T05:22:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.735+0000 7efdbaffd640 1 -- 172.21.15.45:0/3867609957 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7efdb0010a20 con 0x7efdbc074cb0 2024-01-31T05:22:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.736+0000 7efdbaffd640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7efda40a12e0 0x7efda40a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.736+0000 7efdc1c35640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7efda40a12e0 0x7efda40a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.736+0000 7efdbaffd640 1 -- 172.21.15.45:0/3867609957 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7efdb00cc780 con 0x7efdbc074cb0 2024-01-31T05:22:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.736+0000 7efdc1c35640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7efda40a12e0 0x7efda40a37a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efdb4006010 tx=0x7efdb402c670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:01.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.739+0000 7efdbaffd640 1 -- 172.21.15.45:0/3867609957 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7efdb0095c30 con 0x7efdbc074cb0 2024-01-31T05:22:01.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:01 smithi045 ceph-mon[31585]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-01-31T05:22:01.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:01 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2292820898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:01.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:01 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1312601828' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:22:01.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:01 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3729946278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:22:01.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:01 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1229948706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:22:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.954+0000 7efdc3e99640 1 -- 172.21.15.45:0/3867609957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdbc10e7e0 con 0x7efdbc074cb0 2024-01-31T05:22:01.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.955+0000 7efdbaffd640 1 -- 172.21.15.45:0/3867609957 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11346 (secure 0 0 0) 0x7efdb0020090 con 0x7efdbc074cb0 2024-01-31T05:22:01.955 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:01.955 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1801709710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1801709710},{"type":"v1","addr":"172.21.15.45:6839","nonce":1801709710}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/1148586810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1148586810},{"type":"v1","addr":"172.21.15.45:6841","nonce":1148586810}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-31T05:20:40.057668+0000","modified":"2024-01-31T05:20:44.476398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3782584640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3782584640},{"type":"v1","addr":"172.21.15.156:6837","nonce":3782584640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/3821255404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3821255404},{"type":"v1","addr":"172.21.15.186:6839","nonce":3821255404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/3137398384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3137398384},{"type":"v1","addr":"172.21.15.156:6839","nonce":3137398384}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:22:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 20 2024-01-31T05:22:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.957+0000 7efdc3e99640 1 -- 172.21.15.45:0/3867609957 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7efda40a12e0 msgr2=0x7efda40a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.957+0000 7efdc3e99640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7efda40a12e0 0x7efda40a37a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efdb4006010 tx=0x7efdb402c670 comp rx=0 tx=0).stop 2024-01-31T05:22:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.957+0000 7efdc3e99640 1 -- 172.21.15.45:0/3867609957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdbc074cb0 msgr2=0x7efdbc1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.957+0000 7efdc3e99640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdbc074cb0 0x7efdbc1993a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efdb000d950 tx=0x7efdb000de10 comp rx=0 tx=0).stop 2024-01-31T05:22:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.957+0000 7efdc3e99640 1 -- 172.21.15.45:0/3867609957 shutdown_connections 2024-01-31T05:22:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.957+0000 7efdc3e99640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7efda40a12e0 0x7efda40a37a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.957+0000 7efdc3e99640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdbc074cb0 0x7efdbc1993a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.957+0000 7efdc3e99640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc108bf0 0x7efdbc1a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.957+0000 7efdc3e99640 1 --2- 172.21.15.45:0/3867609957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdbc0736d0 0x7efdbc198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.957+0000 7efdc3e99640 1 -- 172.21.15.45:0/3867609957 >> 172.21.15.45:0/3867609957 conn(0x7efdbc0fc290 msgr2=0x7efdbc104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.957+0000 7efdc3e99640 1 -- 172.21.15.45:0/3867609957 shutdown_connections 2024-01-31T05:22:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:01.957+0000 7efdc3e99640 1 -- 172.21.15.45:0/3867609957 wait complete. 2024-01-31T05:22:01.968 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:22:01.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:01 smithi156 ceph-mon[35720]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-01-31T05:22:01.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:01 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2292820898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:01.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:01 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1312601828' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:22:01.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:01 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3729946278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:22:01.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:01 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1229948706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:22:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.070+0000 7f1f3ba09640 1 -- 172.21.15.45:0/498658881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f341017c0 msgr2=0x7f1f34108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.070+0000 7f1f3ba09640 1 --2- 172.21.15.45:0/498658881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f341017c0 0x7f1f34108050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009ca0 tx=0x7f1f2c02c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.070+0000 7f1f3ba09640 1 -- 172.21.15.45:0/498658881 shutdown_connections 2024-01-31T05:22:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.070+0000 7f1f3ba09640 1 --2- 172.21.15.45:0/498658881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f341017c0 0x7f1f34108050 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.070+0000 7f1f3ba09640 1 --2- 172.21.15.45:0/498658881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f34100e00 0x7f1f34101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.070+0000 7f1f3ba09640 1 --2- 172.21.15.45:0/498658881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f340ffc10 0x7f1f34100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.070+0000 7f1f3ba09640 1 -- 172.21.15.45:0/498658881 >> 172.21.15.45:0/498658881 conn(0x7f1f340fb180 msgr2=0x7f1f340fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.070+0000 7f1f3ba09640 1 -- 172.21.15.45:0/498658881 shutdown_connections 2024-01-31T05:22:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.070+0000 7f1f3ba09640 1 -- 172.21.15.45:0/498658881 wait complete. 2024-01-31T05:22:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f3ba09640 1 Processor -- start 2024-01-31T05:22:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f3ba09640 1 -- start start 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f3ba09640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f340ffc10 0x7f1f34103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f3ba09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f34100e00 0x7f1f341021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f3ba09640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f341017c0 0x7f1f34102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f3ba09640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34102e80 con 0x7f1f34100e00 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f3ba09640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34102ff0 con 0x7f1f340ffc10 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f3ba09640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f341a0d20 con 0x7f1f341017c0 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f38fa4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f34100e00 0x7f1f341021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f38fa4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f34100e00 0x7f1f341021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39084/0 (socket says 172.21.15.45:39084) 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f38fa4640 1 -- 172.21.15.45:0/4101238080 learned_addr learned my addr 172.21.15.45:0/4101238080 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f397a5640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f340ffc10 0x7f1f34103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f38fa4640 1 -- 172.21.15.45:0/4101238080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f341017c0 msgr2=0x7f1f34102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f38fa4640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f341017c0 0x7f1f34102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f38fa4640 1 -- 172.21.15.45:0/4101238080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f340ffc10 msgr2=0x7f1f34103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f38fa4640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f340ffc10 0x7f1f34103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f38fa4640 1 -- 172.21.15.45:0/4101238080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c009900 con 0x7f1f34100e00 2024-01-31T05:22:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.071+0000 7f1f397a5640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f340ffc10 0x7f1f34103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.072+0000 7f1f38fa4640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f34100e00 0x7f1f341021e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400d980 tx=0x7f1f2400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.072+0000 7f1f2a7fc640 1 -- 172.21.15.45:0/4101238080 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24009c30 con 0x7f1f34100e00 2024-01-31T05:22:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.072+0000 7f1f2a7fc640 1 -- 172.21.15.45:0/4101238080 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f2400b660 con 0x7f1f34100e00 2024-01-31T05:22:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.072+0000 7f1f3ba09640 1 -- 172.21.15.45:0/4101238080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f341a0fd0 con 0x7f1f34100e00 2024-01-31T05:22:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.072+0000 7f1f2a7fc640 1 -- 172.21.15.45:0/4101238080 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2400f620 con 0x7f1f34100e00 2024-01-31T05:22:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.072+0000 7f1f3ba09640 1 -- 172.21.15.45:0/4101238080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f341a1450 con 0x7f1f34100e00 2024-01-31T05:22:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.073+0000 7f1f3ba09640 1 -- 172.21.15.45:0/4101238080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3410b4d0 con 0x7f1f34100e00 2024-01-31T05:22:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.074+0000 7f1f2a7fc640 1 -- 172.21.15.45:0/4101238080 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f1f24005030 con 0x7f1f34100e00 2024-01-31T05:22:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.074+0000 7f1f2a7fc640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1f200a12e0 0x7f1f200a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.075+0000 7f1f397a5640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1f200a12e0 0x7f1f200a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.075+0000 7f1f2a7fc640 1 -- 172.21.15.45:0/4101238080 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1f240cc840 con 0x7f1f34100e00 2024-01-31T05:22:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.075+0000 7f1f397a5640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1f200a12e0 0x7f1f200a37a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f34100c60 tx=0x7f1f30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.078+0000 7f1f2a7fc640 1 -- 172.21.15.45:0/4101238080 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1f24095c90 con 0x7f1f34100e00 2024-01-31T05:22:02.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:01 smithi186 ceph-mon[35659]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-01-31T05:22:02.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:01 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2292820898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:02.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:01 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1312601828' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:22:02.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:01 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3729946278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:22:02.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:01 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1229948706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:22:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.257+0000 7f1f3ba09640 1 -- 172.21.15.45:0/4101238080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f341a1670 con 0x7f1f34100e00 2024-01-31T05:22:02.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.259+0000 7f1f2a7fc640 1 -- 172.21.15.45:0/4101238080 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22520 (secure 0 0 0) 0x7f1f24020090 con 0x7f1f34100e00 2024-01-31T05:22:02.259 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:02.259 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":88,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:21:47.682334+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:22:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.261+0000 7f1f3ba09640 1 -- 172.21.15.45:0/4101238080 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1f200a12e0 msgr2=0x7f1f200a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.261+0000 7f1f3ba09640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1f200a12e0 0x7f1f200a37a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f34100c60 tx=0x7f1f30005f90 comp rx=0 tx=0).stop 2024-01-31T05:22:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.261+0000 7f1f3ba09640 1 -- 172.21.15.45:0/4101238080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f34100e00 msgr2=0x7f1f341021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.261+0000 7f1f3ba09640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f34100e00 0x7f1f341021e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400d980 tx=0x7f1f2400de40 comp rx=0 tx=0).stop 2024-01-31T05:22:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.261+0000 7f1f3ba09640 1 -- 172.21.15.45:0/4101238080 shutdown_connections 2024-01-31T05:22:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.261+0000 7f1f3ba09640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1f200a12e0 0x7f1f200a37a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.261+0000 7f1f3ba09640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f34100e00 0x7f1f341021e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.261+0000 7f1f3ba09640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f341017c0 0x7f1f34102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.261+0000 7f1f3ba09640 1 --2- 172.21.15.45:0/4101238080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f340ffc10 0x7f1f34103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.261+0000 7f1f3ba09640 1 -- 172.21.15.45:0/4101238080 >> 172.21.15.45:0/4101238080 conn(0x7f1f340fb180 msgr2=0x7f1f340fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:02.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.261+0000 7f1f3ba09640 1 -- 172.21.15.45:0/4101238080 shutdown_connections 2024-01-31T05:22:02.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.261+0000 7f1f3ba09640 1 -- 172.21.15.45:0/4101238080 wait complete. 2024-01-31T05:22:02.272 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:16:10.856581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:20:38.347599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:20:39.270522+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:22:02.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:22:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.377+0000 7f00bce1d640 1 -- 172.21.15.45:0/3941055775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00b81017a0 msgr2=0x7f00b8108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.377+0000 7f00bce1d640 1 --2- 172.21.15.45:0/3941055775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00b81017a0 0x7f00b8108030 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009ca0 tx=0x7f00b002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.377+0000 7f00bce1d640 1 -- 172.21.15.45:0/3941055775 shutdown_connections 2024-01-31T05:22:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.377+0000 7f00bce1d640 1 --2- 172.21.15.45:0/3941055775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00b81017a0 0x7f00b8108030 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.377+0000 7f00bce1d640 1 --2- 172.21.15.45:0/3941055775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b8100de0 0x7f00b8101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.377+0000 7f00bce1d640 1 --2- 172.21.15.45:0/3941055775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b80ffbf0 0x7f00b8100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.377+0000 7f00bce1d640 1 -- 172.21.15.45:0/3941055775 >> 172.21.15.45:0/3941055775 conn(0x7f00b80fb180 msgr2=0x7f00b80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.377+0000 7f00bce1d640 1 -- 172.21.15.45:0/3941055775 shutdown_connections 2024-01-31T05:22:02.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.377+0000 7f00bce1d640 1 -- 172.21.15.45:0/3941055775 wait complete. 2024-01-31T05:22:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00bce1d640 1 Processor -- start 2024-01-31T05:22:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00bce1d640 1 -- start start 2024-01-31T05:22:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00bce1d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b80ffbf0 0x7f00b810a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00bce1d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00b8100de0 0x7f00b8108870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00bce1d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b81017a0 0x7f00b8108db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00bce1d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b8109510 con 0x7f00b8100de0 2024-01-31T05:22:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00bce1d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b8109680 con 0x7f00b80ffbf0 2024-01-31T05:22:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00bce1d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b81a0cc0 con 0x7f00b81017a0 2024-01-31T05:22:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00b5d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00b8100de0 0x7f00b8108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00b5d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00b8100de0 0x7f00b8108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39102/0 (socket says 172.21.15.45:39102) 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00b5d9b640 1 -- 172.21.15.45:0/2986409761 learned_addr learned my addr 172.21.15.45:0/2986409761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00b6d9d640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b81017a0 0x7f00b8108db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00b659c640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b80ffbf0 0x7f00b810a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00b5d9b640 1 -- 172.21.15.45:0/2986409761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b81017a0 msgr2=0x7f00b8108db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00b5d9b640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b81017a0 0x7f00b8108db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00b5d9b640 1 -- 172.21.15.45:0/2986409761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b80ffbf0 msgr2=0x7f00b810a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00b5d9b640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b80ffbf0 0x7f00b810a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00b5d9b640 1 -- 172.21.15.45:0/2986409761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b0009900 con 0x7f00b8100de0 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.378+0000 7f00b6d9d640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b81017a0 0x7f00b8108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.379+0000 7f00b5d9b640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00b8100de0 0x7f00b8108870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00c8c0 tx=0x7f00ac00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.379+0000 7f00a77fe640 1 -- 172.21.15.45:0/2986409761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac004220 con 0x7f00b8100de0 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.379+0000 7f00a77fe640 1 -- 172.21.15.45:0/2986409761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00ac007e70 con 0x7f00b8100de0 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.379+0000 7f00bce1d640 1 -- 172.21.15.45:0/2986409761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b81a0f70 con 0x7f00b8100de0 2024-01-31T05:22:02.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.379+0000 7f00a77fe640 1 -- 172.21.15.45:0/2986409761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac004a00 con 0x7f00b8100de0 2024-01-31T05:22:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.379+0000 7f00bce1d640 1 -- 172.21.15.45:0/2986409761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b81a13f0 con 0x7f00b8100de0 2024-01-31T05:22:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.380+0000 7f00bce1d640 1 -- 172.21.15.45:0/2986409761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b8106f10 con 0x7f00b8100de0 2024-01-31T05:22:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.381+0000 7f00a77fe640 1 -- 172.21.15.45:0/2986409761 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f00ac0079b0 con 0x7f00b8100de0 2024-01-31T05:22:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.381+0000 7f00a77fe640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f00a00a12e0 0x7f00a00a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.382+0000 7f00b659c640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f00a00a12e0 0x7f00a00a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.382+0000 7f00a77fe640 1 -- 172.21.15.45:0/2986409761 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f00ac0cc830 con 0x7f00b8100de0 2024-01-31T05:22:02.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.382+0000 7f00b659c640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f00a00a12e0 0x7f00a00a37a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f00b8100c40 tx=0x7f00a800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.386+0000 7f00a77fe640 1 -- 172.21.15.45:0/2986409761 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f00ac095c80 con 0x7f00b8100de0 2024-01-31T05:22:02.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.567+0000 7f00bce1d640 1 -- 172.21.15.45:0/2986409761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00b810c5b0 con 0x7f00b8100de0 2024-01-31T05:22:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.568+0000 7f00a77fe640 1 -- 172.21.15.45:0/2986409761 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00ac0992a0 con 0x7f00b8100de0 2024-01-31T05:22:02.568 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-01-31T05:22:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.570+0000 7f00bce1d640 1 -- 172.21.15.45:0/2986409761 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f00a00a12e0 msgr2=0x7f00a00a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.570+0000 7f00bce1d640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f00a00a12e0 0x7f00a00a37a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f00b8100c40 tx=0x7f00a800a2d0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.570+0000 7f00bce1d640 1 -- 172.21.15.45:0/2986409761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00b8100de0 msgr2=0x7f00b8108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.570+0000 7f00bce1d640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00b8100de0 0x7f00b8108870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00c8c0 tx=0x7f00ac00cd80 comp rx=0 tx=0).stop 2024-01-31T05:22:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.570+0000 7f00bce1d640 1 -- 172.21.15.45:0/2986409761 shutdown_connections 2024-01-31T05:22:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.570+0000 7f00bce1d640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f00a00a12e0 0x7f00a00a37a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.570+0000 7f00bce1d640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00b8100de0 0x7f00b8108870 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.570+0000 7f00bce1d640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b81017a0 0x7f00b8108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.570+0000 7f00bce1d640 1 --2- 172.21.15.45:0/2986409761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b80ffbf0 0x7f00b810a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.570+0000 7f00bce1d640 1 -- 172.21.15.45:0/2986409761 >> 172.21.15.45:0/2986409761 conn(0x7f00b80fb180 msgr2=0x7f00b81040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.570+0000 7f00bce1d640 1 -- 172.21.15.45:0/2986409761 shutdown_connections 2024-01-31T05:22:02.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.570+0000 7f00bce1d640 1 -- 172.21.15.45:0/2986409761 wait complete. 2024-01-31T05:22:02.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:22:02.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.687+0000 7fc98a91f640 1 -- 172.21.15.45:0/3221830430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841069f0 msgr2=0x7fc984108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.687+0000 7fc98a91f640 1 --2- 172.21.15.45:0/3221830430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841069f0 0x7fc984108e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009ca0 tx=0x7fc97c02c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.687+0000 7fc98a91f640 1 -- 172.21.15.45:0/3221830430 shutdown_connections 2024-01-31T05:22:02.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.687+0000 7fc98a91f640 1 --2- 172.21.15.45:0/3221830430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841069f0 0x7fc984108e40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.687+0000 7fc98a91f640 1 --2- 172.21.15.45:0/3221830430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9841040c0 0x7fc9841064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.687+0000 7fc98a91f640 1 --2- 172.21.15.45:0/3221830430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc984101790 0x7fc984103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.687+0000 7fc98a91f640 1 -- 172.21.15.45:0/3221830430 >> 172.21.15.45:0/3221830430 conn(0x7fc9840fb1a0 msgr2=0x7fc9840fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:02.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.687+0000 7fc98a91f640 1 -- 172.21.15.45:0/3221830430 shutdown_connections 2024-01-31T05:22:02.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.687+0000 7fc98a91f640 1 -- 172.21.15.45:0/3221830430 wait complete. 2024-01-31T05:22:02.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc98a91f640 1 Processor -- start 2024-01-31T05:22:02.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc98a91f640 1 -- start start 2024-01-31T05:22:02.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc98a91f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc984101790 0x7fc984196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc98a91f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9841040c0 0x7fc9841971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc98a91f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841069f0 0x7fc98419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc98a91f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98419e790 con 0x7fc9841069f0 2024-01-31T05:22:02.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc98a91f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98419e900 con 0x7fc984101790 2024-01-31T05:22:02.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc98a91f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98419eac0 con 0x7fc9841040c0 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc988ebc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841069f0 0x7fc98419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc988ebc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841069f0 0x7fc98419e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39130/0 (socket says 172.21.15.45:39130) 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc988ebc640 1 -- 172.21.15.45:0/1779566432 learned_addr learned my addr 172.21.15.45:0/1779566432 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc983fff640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc984101790 0x7fc984196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc9837fe640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9841040c0 0x7fc9841971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc988ebc640 1 -- 172.21.15.45:0/1779566432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9841040c0 msgr2=0x7fc9841971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc988ebc640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9841040c0 0x7fc9841971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc988ebc640 1 -- 172.21.15.45:0/1779566432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc984101790 msgr2=0x7fc984196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc988ebc640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc984101790 0x7fc984196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc988ebc640 1 -- 172.21.15.45:0/1779566432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c009900 con 0x7fc9841069f0 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc983fff640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc984101790 0x7fc984196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.688+0000 7fc9837fe640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9841040c0 0x7fc9841971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.689+0000 7fc988ebc640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841069f0 0x7fc98419e220 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc97c037040 tx=0x7fc97c032810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.689+0000 7fc9817fa640 1 -- 172.21.15.45:0/1779566432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c034e80 con 0x7fc9841069f0 2024-01-31T05:22:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.689+0000 7fc9817fa640 1 -- 172.21.15.45:0/1779566432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc97c003e30 con 0x7fc9841069f0 2024-01-31T05:22:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.689+0000 7fc98a91f640 1 -- 172.21.15.45:0/1779566432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98419ed40 con 0x7fc9841069f0 2024-01-31T05:22:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.689+0000 7fc9817fa640 1 -- 172.21.15.45:0/1779566432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c03d3f0 con 0x7fc9841069f0 2024-01-31T05:22:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.689+0000 7fc98a91f640 1 -- 172.21.15.45:0/1779566432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98419f1a0 con 0x7fc9841069f0 2024-01-31T05:22:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.690+0000 7fc98a91f640 1 -- 172.21.15.45:0/1779566432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9840fcc90 con 0x7fc9841069f0 2024-01-31T05:22:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.691+0000 7fc9817fa640 1 -- 172.21.15.45:0/1779566432 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fc97c03b040 con 0x7fc9841069f0 2024-01-31T05:22:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.691+0000 7fc9817fa640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc9700a12e0 0x7fc9700a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.691+0000 7fc983fff640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc9700a12e0 0x7fc9700a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.692+0000 7fc9817fa640 1 -- 172.21.15.45:0/1779566432 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc97c0ef3c0 con 0x7fc9841069f0 2024-01-31T05:22:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.692+0000 7fc983fff640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc9700a12e0 0x7fc9700a37a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc974005fd0 tx=0x7fc974005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.696+0000 7fc9817fa640 1 -- 172.21.15.45:0/1779566432 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fc97c0b8810 con 0x7fc9841069f0 2024-01-31T05:22:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.874+0000 7fc98a91f640 1 -- 172.21.15.45:0/1779566432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc98402cb40 con 0x7fc9841069f0 2024-01-31T05:22:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.874+0000 7fc9817fa640 1 -- 172.21.15.45:0/1779566432 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc97c0bbe30 con 0x7fc9841069f0 2024-01-31T05:22:02.874 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-01-31T05:22:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.876+0000 7fc98a91f640 1 -- 172.21.15.45:0/1779566432 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc9700a12e0 msgr2=0x7fc9700a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.876+0000 7fc98a91f640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc9700a12e0 0x7fc9700a37a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc974005fd0 tx=0x7fc974005ec0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.876+0000 7fc98a91f640 1 -- 172.21.15.45:0/1779566432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841069f0 msgr2=0x7fc98419e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.876+0000 7fc98a91f640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841069f0 0x7fc98419e220 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc97c037040 tx=0x7fc97c032810 comp rx=0 tx=0).stop 2024-01-31T05:22:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.876+0000 7fc98a91f640 1 -- 172.21.15.45:0/1779566432 shutdown_connections 2024-01-31T05:22:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.876+0000 7fc98a91f640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fc9700a12e0 0x7fc9700a37a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.876+0000 7fc98a91f640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841069f0 0x7fc98419e220 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.876+0000 7fc98a91f640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9841040c0 0x7fc9841971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.876+0000 7fc98a91f640 1 --2- 172.21.15.45:0/1779566432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc984101790 0x7fc984196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.876+0000 7fc98a91f640 1 -- 172.21.15.45:0/1779566432 >> 172.21.15.45:0/1779566432 conn(0x7fc9840fb1a0 msgr2=0x7fc984102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.876+0000 7fc98a91f640 1 -- 172.21.15.45:0/1779566432 shutdown_connections 2024-01-31T05:22:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.876+0000 7fc98a91f640 1 -- 172.21.15.45:0/1779566432 wait complete. 2024-01-31T05:22:02.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:02 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3867609957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:02.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:02 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/4101238080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:02.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:02 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2986409761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:22:02.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:22:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:02 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3867609957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:02 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/4101238080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:02 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2986409761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:22:02.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.993+0000 7f45cc4f1640 1 -- 172.21.15.45:0/283097657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c40ffc10 msgr2=0x7f45c4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.993+0000 7f45cc4f1640 1 --2- 172.21.15.45:0/283097657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c40ffc10 0x7f45c4100030 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009900 tx=0x7f45c002bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:02.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.993+0000 7f45cc4f1640 1 -- 172.21.15.45:0/283097657 shutdown_connections 2024-01-31T05:22:02.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.993+0000 7f45cc4f1640 1 --2- 172.21.15.45:0/283097657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c41017c0 0x7f45c4108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.993+0000 7f45cc4f1640 1 --2- 172.21.15.45:0/283097657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c4100e00 0x7f45c4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.993+0000 7f45cc4f1640 1 --2- 172.21.15.45:0/283097657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c40ffc10 0x7f45c4100030 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.993+0000 7f45cc4f1640 1 -- 172.21.15.45:0/283097657 >> 172.21.15.45:0/283097657 conn(0x7f45c40fb1a0 msgr2=0x7f45c40fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.993+0000 7f45cc4f1640 1 -- 172.21.15.45:0/283097657 shutdown_connections 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.993+0000 7f45cc4f1640 1 -- 172.21.15.45:0/283097657 wait complete. 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.993+0000 7f45cc4f1640 1 Processor -- start 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.993+0000 7f45cc4f1640 1 -- start start 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45cc4f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c40ffc10 0x7f45c410a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45cc4f1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c4100e00 0x7f45c4108b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45cc4f1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c41017c0 0x7f45c4109050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45cc4f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c41097b0 con 0x7f45c40ffc10 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45cc4f1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c4109920 con 0x7f45c4100e00 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45cc4f1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c41a0d60 con 0x7f45c41017c0 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45ca28d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c40ffc10 0x7f45c410a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45ca28d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c40ffc10 0x7f45c410a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39146/0 (socket says 172.21.15.45:39146) 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45ca28d640 1 -- 172.21.15.45:0/3906129652 learned_addr learned my addr 172.21.15.45:0/3906129652 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45c9a8c640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c4100e00 0x7f45c4108b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45ca28d640 1 -- 172.21.15.45:0/3906129652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c41017c0 msgr2=0x7f45c4109050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45ca28d640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c41017c0 0x7f45c4109050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45ca28d640 1 -- 172.21.15.45:0/3906129652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c4100e00 msgr2=0x7f45c4108b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45ca28d640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c4100e00 0x7f45c4108b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45ca28d640 1 -- 172.21.15.45:0/3906129652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0009560 con 0x7f45c40ffc10 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45ca28d640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c40ffc10 0x7f45c410a4d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f45c0036040 tx=0x7f45c0004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45bb7fe640 1 -- 172.21.15.45:0/3906129652 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c00043d0 con 0x7f45c40ffc10 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45bb7fe640 1 -- 172.21.15.45:0/3906129652 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45c00054c0 con 0x7f45c40ffc10 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45cc4f1640 1 -- 172.21.15.45:0/3906129652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c41a0fb0 con 0x7f45c40ffc10 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45bb7fe640 1 -- 172.21.15.45:0/3906129652 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0003e10 con 0x7f45c40ffc10 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.994+0000 7f45cc4f1640 1 -- 172.21.15.45:0/3906129652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c41a13e0 con 0x7f45c40ffc10 2024-01-31T05:22:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.995+0000 7f45cc4f1640 1 -- 172.21.15.45:0/3906129652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c4106ff0 con 0x7f45c40ffc10 2024-01-31T05:22:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.996+0000 7f45bb7fe640 1 -- 172.21.15.45:0/3906129652 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f45c0005000 con 0x7f45c40ffc10 2024-01-31T05:22:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.997+0000 7f45bb7fe640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f45b00a12e0 0x7f45b00a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.997+0000 7f45c9a8c640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f45b00a12e0 0x7f45b00a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.997+0000 7f45bb7fe640 1 -- 172.21.15.45:0/3906129652 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f45c00ef460 con 0x7f45c40ffc10 2024-01-31T05:22:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:02.997+0000 7f45c9a8c640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f45b00a12e0 0x7f45b00a37a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f45c410a240 tx=0x7f45b400a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:03.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.000+0000 7f45bb7fe640 1 -- 172.21.15.45:0/3906129652 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f45c00b8910 con 0x7f45c40ffc10 2024-01-31T05:22:03.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:02 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3867609957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:03.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:02 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/4101238080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:03.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:02 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2986409761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:22:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.185+0000 7f45cc4f1640 1 -- 172.21.15.45:0/3906129652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45c410c790 con 0x7f45c40ffc10 2024-01-31T05:22:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.185+0000 7f45bb7fe640 1 -- 172.21.15.45:0/3906129652 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45c0043020 con 0x7f45c40ffc10 2024-01-31T05:22:03.185 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-01-31T05:22:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.187+0000 7f45cc4f1640 1 -- 172.21.15.45:0/3906129652 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f45b00a12e0 msgr2=0x7f45b00a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.187+0000 7f45cc4f1640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f45b00a12e0 0x7f45b00a37a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f45c410a240 tx=0x7f45b400a4c0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.187+0000 7f45cc4f1640 1 -- 172.21.15.45:0/3906129652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c40ffc10 msgr2=0x7f45c410a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.187+0000 7f45cc4f1640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c40ffc10 0x7f45c410a4d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f45c0036040 tx=0x7f45c0004ed0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.187+0000 7f45cc4f1640 1 -- 172.21.15.45:0/3906129652 shutdown_connections 2024-01-31T05:22:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.187+0000 7f45cc4f1640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f45b00a12e0 0x7f45b00a37a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.187+0000 7f45cc4f1640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c41017c0 0x7f45c4109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.187+0000 7f45cc4f1640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c4100e00 0x7f45c4108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.187+0000 7f45cc4f1640 1 --2- 172.21.15.45:0/3906129652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c40ffc10 0x7f45c410a4d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.187+0000 7f45cc4f1640 1 -- 172.21.15.45:0/3906129652 >> 172.21.15.45:0/3906129652 conn(0x7f45c40fb1a0 msgr2=0x7f45c41041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.188+0000 7f45cc4f1640 1 -- 172.21.15.45:0/3906129652 shutdown_connections 2024-01-31T05:22:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.188+0000 7f45cc4f1640 1 -- 172.21.15.45:0/3906129652 wait complete. 2024-01-31T05:22:03.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:22:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.301+0000 7fd55c2b8640 1 -- 172.21.15.45:0/2675584056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5540ff000 msgr2=0x7fd5540ff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.301+0000 7fd55c2b8640 1 --2- 172.21.15.45:0/2675584056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5540ff000 0x7fd5540ff420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd550009900 tx=0x7fd55002bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.301+0000 7fd55c2b8640 1 -- 172.21.15.45:0/2675584056 shutdown_connections 2024-01-31T05:22:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.301+0000 7fd55c2b8640 1 --2- 172.21.15.45:0/2675584056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd554102990 0x7fd554109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.301+0000 7fd55c2b8640 1 --2- 172.21.15.45:0/2675584056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd554101fd0 0x7fd554102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.301+0000 7fd55c2b8640 1 --2- 172.21.15.45:0/2675584056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5540ff000 0x7fd5540ff420 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.301+0000 7fd55c2b8640 1 -- 172.21.15.45:0/2675584056 >> 172.21.15.45:0/2675584056 conn(0x7fd5540fa930 msgr2=0x7fd5540fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.301+0000 7fd55c2b8640 1 -- 172.21.15.45:0/2675584056 shutdown_connections 2024-01-31T05:22:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.301+0000 7fd55c2b8640 1 -- 172.21.15.45:0/2675584056 wait complete. 2024-01-31T05:22:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.301+0000 7fd55c2b8640 1 Processor -- start 2024-01-31T05:22:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.301+0000 7fd55c2b8640 1 -- start start 2024-01-31T05:22:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55c2b8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5540ff000 0x7fd554196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55c2b8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd554101fd0 0x7fd554197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55c2b8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd554102990 0x7fd55419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55c2b8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55419e7d0 con 0x7fd554102990 2024-01-31T05:22:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55c2b8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55419e940 con 0x7fd554101fd0 2024-01-31T05:22:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55c2b8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55419eb00 con 0x7fd5540ff000 2024-01-31T05:22:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55a855640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd554102990 0x7fd55419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55a855640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd554102990 0x7fd55419e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39150/0 (socket says 172.21.15.45:39150) 2024-01-31T05:22:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55a855640 1 -- 172.21.15.45:0/1441718602 learned_addr learned my addr 172.21.15.45:0/1441718602 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd559853640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd554101fd0 0x7fd554197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55a054640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5540ff000 0x7fd554196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55a855640 1 -- 172.21.15.45:0/1441718602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5540ff000 msgr2=0x7fd554196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55a855640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5540ff000 0x7fd554196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55a855640 1 -- 172.21.15.45:0/1441718602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd554101fd0 msgr2=0x7fd554197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55a855640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd554101fd0 0x7fd554197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55a855640 1 -- 172.21.15.45:0/1441718602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd550009560 con 0x7fd554102990 2024-01-31T05:22:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55a054640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5540ff000 0x7fd554196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd559853640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd554101fd0 0x7fd554197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55a855640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd554102990 0x7fd55419e260 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00eaa0 tx=0x7fd54c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd54b7fe640 1 -- 172.21.15.45:0/1441718602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c00cdb0 con 0x7fd554102990 2024-01-31T05:22:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd54b7fe640 1 -- 172.21.15.45:0/1441718602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd54c005020 con 0x7fd554102990 2024-01-31T05:22:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55c2b8640 1 -- 172.21.15.45:0/1441718602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55419ede0 con 0x7fd554102990 2024-01-31T05:22:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd54b7fe640 1 -- 172.21.15.45:0/1441718602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c0116d0 con 0x7fd554102990 2024-01-31T05:22:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.302+0000 7fd55c2b8640 1 -- 172.21.15.45:0/1441718602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55419f260 con 0x7fd554102990 2024-01-31T05:22:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.303+0000 7fd55c2b8640 1 -- 172.21.15.45:0/1441718602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd554108150 con 0x7fd554102990 2024-01-31T05:22:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.304+0000 7fd54b7fe640 1 -- 172.21.15.45:0/1441718602 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fd54c005190 con 0x7fd554102990 2024-01-31T05:22:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.305+0000 7fd54b7fe640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd5400a1360 0x7fd5400a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.305+0000 7fd55a054640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd5400a1360 0x7fd5400a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.305+0000 7fd54b7fe640 1 -- 172.21.15.45:0/1441718602 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd54c0cce40 con 0x7fd554102990 2024-01-31T05:22:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.305+0000 7fd55a054640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd5400a1360 0x7fd5400a3820 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd550006010 tx=0x7fd550000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.309+0000 7fd54b7fe640 1 -- 172.21.15.45:0/1441718602 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd54c096260 con 0x7fd554102990 2024-01-31T05:22:03.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.526+0000 7fd55c2b8640 1 -- 172.21.15.45:0/1441718602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd55402cbc0 con 0x7fd554102990 2024-01-31T05:22:03.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.526+0000 7fd54b7fe640 1 -- 172.21.15.45:0/1441718602 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11346 (secure 0 0 0) 0x7fd54c01f020 con 0x7fd554102990 2024-01-31T05:22:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:03.527 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1801709710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1801709710},{"type":"v1","addr":"172.21.15.45:6839","nonce":1801709710}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/1148586810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1148586810},{"type":"v1","addr":"172.21.15.45:6841","nonce":1148586810}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-31T05:20:40.057668+0000","modified":"2024-01-31T05:20:44.476398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3782584640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3782584640},{"type":"v1","addr":"172.21.15.156:6837","nonce":3782584640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/3821255404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3821255404},{"type":"v1","addr":"172.21.15.186:6839","nonce":3821255404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/3137398384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3137398384},{"type":"v1","addr":"172.21.15.156:6839","nonce":3137398384}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:22:03.527 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 20 2024-01-31T05:22:03.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.528+0000 7fd55c2b8640 1 -- 172.21.15.45:0/1441718602 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd5400a1360 msgr2=0x7fd5400a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.528+0000 7fd55c2b8640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd5400a1360 0x7fd5400a3820 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd550006010 tx=0x7fd550000bc0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.528+0000 7fd55c2b8640 1 -- 172.21.15.45:0/1441718602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd554102990 msgr2=0x7fd55419e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.528+0000 7fd55c2b8640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd554102990 0x7fd55419e260 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00eaa0 tx=0x7fd54c00ef60 comp rx=0 tx=0).stop 2024-01-31T05:22:03.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.528+0000 7fd55c2b8640 1 -- 172.21.15.45:0/1441718602 shutdown_connections 2024-01-31T05:22:03.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.528+0000 7fd55c2b8640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd5400a1360 0x7fd5400a3820 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.528+0000 7fd55c2b8640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd554102990 0x7fd55419e260 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.528+0000 7fd55c2b8640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd554101fd0 0x7fd554197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.528+0000 7fd55c2b8640 1 --2- 172.21.15.45:0/1441718602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5540ff000 0x7fd554196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.528+0000 7fd55c2b8640 1 -- 172.21.15.45:0/1441718602 >> 172.21.15.45:0/1441718602 conn(0x7fd5540fa930 msgr2=0x7fd554105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:03.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.528+0000 7fd55c2b8640 1 -- 172.21.15.45:0/1441718602 shutdown_connections 2024-01-31T05:22:03.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.528+0000 7fd55c2b8640 1 -- 172.21.15.45:0/1441718602 wait complete. 2024-01-31T05:22:03.540 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:22:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.642+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/1205723613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4101780 msgr2=0x7f3ec4103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.642+0000 7f3ecc0a3640 1 --2- 172.21.15.45:0/1205723613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4101780 0x7f3ec4103b70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009900 tx=0x7f3ec002bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.642+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/1205723613 shutdown_connections 2024-01-31T05:22:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.642+0000 7f3ecc0a3640 1 --2- 172.21.15.45:0/1205723613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ec41069e0 0x7f3ec4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.642+0000 7f3ecc0a3640 1 --2- 172.21.15.45:0/1205723613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ec41040b0 0x7f3ec41064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.642+0000 7f3ecc0a3640 1 --2- 172.21.15.45:0/1205723613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4101780 0x7f3ec4103b70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.642+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/1205723613 >> 172.21.15.45:0/1205723613 conn(0x7f3ec40fb190 msgr2=0x7f3ec40fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.642+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/1205723613 shutdown_connections 2024-01-31T05:22:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.642+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/1205723613 wait complete. 2024-01-31T05:22:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ecc0a3640 1 Processor -- start 2024-01-31T05:22:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ecc0a3640 1 -- start start 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ecc0a3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ec4101780 0x7f3ec4196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ecc0a3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec41040b0 0x7f3ec41971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ecc0a3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ec41069e0 0x7f3ec419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ecc0a3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec419e7a0 con 0x7f3ec41040b0 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ecc0a3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec419e910 con 0x7f3ec41069e0 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ecc0a3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec419ead0 con 0x7f3ec4101780 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ec963e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec41040b0 0x7f3ec41971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ec963e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec41040b0 0x7f3ec41971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39174/0 (socket says 172.21.15.45:39174) 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ec963e640 1 -- 172.21.15.45:0/3366854746 learned_addr learned my addr 172.21.15.45:0/3366854746 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ec9e3f640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ec4101780 0x7f3ec4196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3eca640640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ec41069e0 0x7f3ec419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ec963e640 1 -- 172.21.15.45:0/3366854746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ec4101780 msgr2=0x7f3ec4196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ec963e640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ec4101780 0x7f3ec4196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ec963e640 1 -- 172.21.15.45:0/3366854746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ec41069e0 msgr2=0x7f3ec419e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ec963e640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ec41069e0 0x7f3ec419e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ec963e640 1 -- 172.21.15.45:0/3366854746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec0009560 con 0x7f3ec41040b0 2024-01-31T05:22:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3eca640640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ec41069e0 0x7f3ec419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ec963e640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec41040b0 0x7f3ec41971d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400d740 tx=0x7f3eb400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ebaffd640 1 -- 172.21.15.45:0/3366854746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4009c30 con 0x7f3ec41040b0 2024-01-31T05:22:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ebaffd640 1 -- 172.21.15.45:0/3366854746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb400de50 con 0x7f3ec41040b0 2024-01-31T05:22:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.643+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/3366854746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec419edb0 con 0x7f3ec41040b0 2024-01-31T05:22:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.644+0000 7f3ebaffd640 1 -- 172.21.15.45:0/3366854746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4004ea0 con 0x7f3ec41040b0 2024-01-31T05:22:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.644+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/3366854746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec419f1e0 con 0x7f3ec41040b0 2024-01-31T05:22:03.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.644+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/3366854746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec40fcc80 con 0x7f3ec41040b0 2024-01-31T05:22:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.645+0000 7f3ebaffd640 1 -- 172.21.15.45:0/3366854746 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f3eb4005020 con 0x7f3ec41040b0 2024-01-31T05:22:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.646+0000 7f3ebaffd640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3eb00a12e0 0x7f3eb00a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:03.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.646+0000 7f3ec9e3f640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3eb00a12e0 0x7f3eb00a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.646+0000 7f3ebaffd640 1 -- 172.21.15.45:0/3366854746 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3eb40cc550 con 0x7f3ec41040b0 2024-01-31T05:22:03.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.647+0000 7f3ec9e3f640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3eb00a12e0 0x7f3eb00a37a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0005ab0 tx=0x7f3ec002c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:03.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.651+0000 7f3ebaffd640 1 -- 172.21.15.45:0/3366854746 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f3eb40959a0 con 0x7f3ec41040b0 2024-01-31T05:22:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.833+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/3366854746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ec402cbb0 con 0x7f3ec41040b0 2024-01-31T05:22:03.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.835+0000 7f3ebaffd640 1 -- 172.21.15.45:0/3366854746 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22520 (secure 0 0 0) 0x7f3eb4098fc0 con 0x7f3ec41040b0 2024-01-31T05:22:03.835 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:03.835 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":88,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:21:47.682334+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:22:03.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.837+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/3366854746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3eb00a12e0 msgr2=0x7f3eb00a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.837+0000 7f3ecc0a3640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3eb00a12e0 0x7f3eb00a37a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0005ab0 tx=0x7f3ec002c5d0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.837+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/3366854746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec41040b0 msgr2=0x7f3ec41971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.837+0000 7f3ecc0a3640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec41040b0 0x7f3ec41971d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400d740 tx=0x7f3eb400dc00 comp rx=0 tx=0).stop 2024-01-31T05:22:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.837+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/3366854746 shutdown_connections 2024-01-31T05:22:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.837+0000 7f3ecc0a3640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3eb00a12e0 0x7f3eb00a37a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.837+0000 7f3ecc0a3640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec41040b0 0x7f3ec41971d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.837+0000 7f3ecc0a3640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ec41069e0 0x7f3ec419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.837+0000 7f3ecc0a3640 1 --2- 172.21.15.45:0/3366854746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ec4101780 0x7f3ec4196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.837+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/3366854746 >> 172.21.15.45:0/3366854746 conn(0x7f3ec40fb190 msgr2=0x7f3ec41020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.837+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/3366854746 shutdown_connections 2024-01-31T05:22:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.837+0000 7f3ecc0a3640 1 -- 172.21.15.45:0/3366854746 wait complete. 2024-01-31T05:22:03.847 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:03 smithi045 ceph-mon[31585]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-01-31T05:22:03.847 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:03 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1779566432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:22:03.847 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:03 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3906129652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:22:03.847 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:03 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1441718602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:03.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:22:03.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.950+0000 7f62bd11f640 1 -- 172.21.15.45:0/712028578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b8100e00 msgr2=0x7f62b8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.950+0000 7f62bd11f640 1 --2- 172.21.15.45:0/712028578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b8100e00 0x7f62b8101280 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009900 tx=0x7f62a002bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:03.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.950+0000 7f62bd11f640 1 -- 172.21.15.45:0/712028578 shutdown_connections 2024-01-31T05:22:03.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.950+0000 7f62bd11f640 1 --2- 172.21.15.45:0/712028578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b8100e00 0x7f62b8101280 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.950+0000 7f62bd11f640 1 --2- 172.21.15.45:0/712028578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62b81017c0 0x7f62b8108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.950+0000 7f62bd11f640 1 --2- 172.21.15.45:0/712028578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62b80ffc10 0x7f62b8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.950+0000 7f62bd11f640 1 -- 172.21.15.45:0/712028578 >> 172.21.15.45:0/712028578 conn(0x7f62b80fb180 msgr2=0x7f62b80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:03.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.950+0000 7f62bd11f640 1 -- 172.21.15.45:0/712028578 shutdown_connections 2024-01-31T05:22:03.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.950+0000 7f62bd11f640 1 -- 172.21.15.45:0/712028578 wait complete. 2024-01-31T05:22:03.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62bd11f640 1 Processor -- start 2024-01-31T05:22:03.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62bd11f640 1 -- start start 2024-01-31T05:22:03.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62bd11f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62b80ffc10 0x7f62b8103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:03.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62bd11f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62b8100e00 0x7f62b81021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:03.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62bd11f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b81017c0 0x7f62b8102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:03.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62bd11f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b8102e80 con 0x7f62b81017c0 2024-01-31T05:22:03.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62bd11f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b8102ff0 con 0x7f62b8100e00 2024-01-31T05:22:03.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62bd11f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b81a0ed0 con 0x7f62b80ffc10 2024-01-31T05:22:03.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62b759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b81017c0 0x7f62b8102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:03.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62b759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b81017c0 0x7f62b8102720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39180/0 (socket says 172.21.15.45:39180) 2024-01-31T05:22:03.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62b759e640 1 -- 172.21.15.45:0/577441528 learned_addr learned my addr 172.21.15.45:0/577441528 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:03.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62b6d9d640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62b80ffc10 0x7f62b8103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:03.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62affff640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62b8100e00 0x7f62b81021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:03.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62b759e640 1 -- 172.21.15.45:0/577441528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62b80ffc10 msgr2=0x7f62b8103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62b759e640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62b80ffc10 0x7f62b8103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62b759e640 1 -- 172.21.15.45:0/577441528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62b8100e00 msgr2=0x7f62b81021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62b759e640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62b8100e00 0x7f62b81021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.951+0000 7f62b759e640 1 -- 172.21.15.45:0/577441528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a0009560 con 0x7f62b81017c0 2024-01-31T05:22:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.952+0000 7f62b759e640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b81017c0 0x7f62b8102720 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f62b000b9a0 tx=0x7f62b000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.952+0000 7f62b4d99640 1 -- 172.21.15.45:0/577441528 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b000cba0 con 0x7f62b81017c0 2024-01-31T05:22:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.952+0000 7f62b4d99640 1 -- 172.21.15.45:0/577441528 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62b000cd00 con 0x7f62b81017c0 2024-01-31T05:22:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.952+0000 7f62bd11f640 1 -- 172.21.15.45:0/577441528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b81a1180 con 0x7f62b81017c0 2024-01-31T05:22:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.952+0000 7f62b4d99640 1 -- 172.21.15.45:0/577441528 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0007640 con 0x7f62b81017c0 2024-01-31T05:22:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.952+0000 7f62bd11f640 1 -- 172.21.15.45:0/577441528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b81a1710 con 0x7f62b81017c0 2024-01-31T05:22:03.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.953+0000 7f62bd11f640 1 -- 172.21.15.45:0/577441528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b810b5c0 con 0x7f62b81017c0 2024-01-31T05:22:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.954+0000 7f62b4d99640 1 -- 172.21.15.45:0/577441528 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f62b0014440 con 0x7f62b81017c0 2024-01-31T05:22:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.954+0000 7f62b4d99640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f62a40a12e0 0x7f62a40a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.954+0000 7f62b6d9d640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f62a40a12e0 0x7f62a40a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.954+0000 7f62b4d99640 1 -- 172.21.15.45:0/577441528 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f62b00cc3c0 con 0x7f62b81017c0 2024-01-31T05:22:03.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.955+0000 7f62b6d9d640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f62a40a12e0 0x7f62a40a37a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62b8100c60 tx=0x7f62a8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:03.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:03.959+0000 7f62b4d99640 1 -- 172.21.15.45:0/577441528 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f62b0095870 con 0x7f62b81017c0 2024-01-31T05:22:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:03 smithi156 ceph-mon[35720]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-01-31T05:22:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:03 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1779566432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:22:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:03 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3906129652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:22:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:03 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1441718602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:04.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:03 smithi186 ceph-mon[35659]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-01-31T05:22:04.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:03 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1779566432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:22:04.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:03 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3906129652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:22:04.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:03 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1441718602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:04.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.178+0000 7f62bd11f640 1 -- 172.21.15.45:0/577441528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62b81a1c10 con 0x7f62b81017c0 2024-01-31T05:22:04.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.178+0000 7f62b4d99640 1 -- 172.21.15.45:0/577441528 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f62b00b0c70 con 0x7f62b81017c0 2024-01-31T05:22:04.179 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:04.179 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1801709710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1801709710},{"type":"v1","addr":"172.21.15.45:6839","nonce":1801709710}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/1148586810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1148586810},{"type":"v1","addr":"172.21.15.45:6841","nonce":1148586810}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-31T05:20:40.057668+0000","modified":"2024-01-31T05:20:44.476398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3782584640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3782584640},{"type":"v1","addr":"172.21.15.156:6837","nonce":3782584640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/3821255404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3821255404},{"type":"v1","addr":"172.21.15.186:6839","nonce":3821255404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/3137398384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3137398384},{"type":"v1","addr":"172.21.15.156:6839","nonce":3137398384}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:22:04.179 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 20 2024-01-31T05:22:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.181+0000 7f62bd11f640 1 -- 172.21.15.45:0/577441528 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f62a40a12e0 msgr2=0x7f62a40a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.181+0000 7f62bd11f640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f62a40a12e0 0x7f62a40a37a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62b8100c60 tx=0x7f62a8008040 comp rx=0 tx=0).stop 2024-01-31T05:22:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.181+0000 7f62bd11f640 1 -- 172.21.15.45:0/577441528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b81017c0 msgr2=0x7f62b8102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.181+0000 7f62bd11f640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b81017c0 0x7f62b8102720 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f62b000b9a0 tx=0x7f62b000be60 comp rx=0 tx=0).stop 2024-01-31T05:22:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.181+0000 7f62bd11f640 1 -- 172.21.15.45:0/577441528 shutdown_connections 2024-01-31T05:22:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.181+0000 7f62bd11f640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f62a40a12e0 0x7f62a40a37a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.181+0000 7f62bd11f640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b81017c0 0x7f62b8102720 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.181+0000 7f62bd11f640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62b8100e00 0x7f62b81021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.181+0000 7f62bd11f640 1 --2- 172.21.15.45:0/577441528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62b80ffc10 0x7f62b8103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.181+0000 7f62bd11f640 1 -- 172.21.15.45:0/577441528 >> 172.21.15.45:0/577441528 conn(0x7f62b80fb180 msgr2=0x7f62b8069410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.181+0000 7f62bd11f640 1 -- 172.21.15.45:0/577441528 shutdown_connections 2024-01-31T05:22:04.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.181+0000 7f62bd11f640 1 -- 172.21.15.45:0/577441528 wait complete. 2024-01-31T05:22:04.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-01-31T05:20:40.057668+0000', 'modified': '2024-01-31T05:20:44.476398+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24503, 'mds_1': 24490, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/4045803513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 4045803513}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 4045803513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/696958495', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 696958495}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 696958495}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6837/3782584640', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 3782584640}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 3782584640}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.186:6837/2559285393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 2559285393}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 2559285393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 16, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.186:6839/3821255404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 3821255404}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 3821255404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24503': {'gid': 24503, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6839/3137398384', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 3137398384}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 3137398384}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-31T05:22:04.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-01-31T05:22:04.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-31T05:22:04.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.296+0000 7f71de621640 1 -- 172.21.15.45:0/4161809842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d80ffbf0 msgr2=0x7f71d8100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.296+0000 7f71de621640 1 --2- 172.21.15.45:0/4161809842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d80ffbf0 0x7f71d8100010 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009980 tx=0x7f71c802de70 comp rx=0 tx=0).stop 2024-01-31T05:22:04.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.296+0000 7f71de621640 1 -- 172.21.15.45:0/4161809842 shutdown_connections 2024-01-31T05:22:04.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.296+0000 7f71de621640 1 --2- 172.21.15.45:0/4161809842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71d81017a0 0x7f71d8108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.296+0000 7f71de621640 1 --2- 172.21.15.45:0/4161809842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71d8100de0 0x7f71d8101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.296+0000 7f71de621640 1 --2- 172.21.15.45:0/4161809842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d80ffbf0 0x7f71d8100010 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.296+0000 7f71de621640 1 -- 172.21.15.45:0/4161809842 >> 172.21.15.45:0/4161809842 conn(0x7f71d80fb180 msgr2=0x7f71d80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.296+0000 7f71de621640 1 -- 172.21.15.45:0/4161809842 shutdown_connections 2024-01-31T05:22:04.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.296+0000 7f71de621640 1 -- 172.21.15.45:0/4161809842 wait complete. 2024-01-31T05:22:04.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71de621640 1 Processor -- start 2024-01-31T05:22:04.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71de621640 1 -- start start 2024-01-31T05:22:04.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71de621640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71d80ffbf0 0x7f71d8194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71de621640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71d8100de0 0x7f71d8194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71de621640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81017a0 0x7f71d819bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71de621640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d819c520 con 0x7f71d81017a0 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71de621640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d819c690 con 0x7f71d80ffbf0 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71de621640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d819c850 con 0x7f71d8100de0 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71dcbbe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81017a0 0x7f71d819bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71dcbbe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81017a0 0x7f71d819bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39200/0 (socket says 172.21.15.45:39200) 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71dcbbe640 1 -- 172.21.15.45:0/2721316588 learned_addr learned my addr 172.21.15.45:0/2721316588 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71d7fff640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71d80ffbf0 0x7f71d8194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71d77fe640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71d8100de0 0x7f71d8194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71dcbbe640 1 -- 172.21.15.45:0/2721316588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71d8100de0 msgr2=0x7f71d8194f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71dcbbe640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71d8100de0 0x7f71d8194f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71dcbbe640 1 -- 172.21.15.45:0/2721316588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71d80ffbf0 msgr2=0x7f71d8194a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71dcbbe640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71d80ffbf0 0x7f71d8194a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.297+0000 7f71dcbbe640 1 -- 172.21.15.45:0/2721316588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c8009560 con 0x7f71d81017a0 2024-01-31T05:22:04.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.298+0000 7f71dcbbe640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81017a0 0x7f71d819bfe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f71d000eaa0 tx=0x7f71d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.298+0000 7f71d57fa640 1 -- 172.21.15.45:0/2721316588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d000cdb0 con 0x7f71d81017a0 2024-01-31T05:22:04.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.298+0000 7f71d57fa640 1 -- 172.21.15.45:0/2721316588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71d0005020 con 0x7f71d81017a0 2024-01-31T05:22:04.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.298+0000 7f71d57fa640 1 -- 172.21.15.45:0/2721316588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d00116d0 con 0x7f71d81017a0 2024-01-31T05:22:04.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.298+0000 7f71de621640 1 -- 172.21.15.45:0/2721316588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d819cb30 con 0x7f71d81017a0 2024-01-31T05:22:04.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.298+0000 7f71de621640 1 -- 172.21.15.45:0/2721316588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d819cff0 con 0x7f71d81017a0 2024-01-31T05:22:04.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.300+0000 7f71d57fa640 1 -- 172.21.15.45:0/2721316588 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f71d0005190 con 0x7f71d81017a0 2024-01-31T05:22:04.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.300+0000 7f71e13df740 1 Processor -- start 2024-01-31T05:22:04.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.300+0000 7f71d57fa640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f71c00a12e0 0x7f71c00a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.300+0000 7f71e13df740 1 -- start start 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.300+0000 7f71d77fe640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f71c00a12e0 0x7f71c00a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.300+0000 7f71d57fa640 1 -- 172.21.15.45:0/2721316588 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f71d00ccd40 con 0x7f71d81017a0 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.300+0000 7f71e13df740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a972670 0x56514a972a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.300+0000 7f71e13df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a973060 0x56514a977b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.300+0000 7f71e13df740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a9781d0 0x56514a97a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.300+0000 7f71e13df740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514a8e1000 con 0x56514a9781d0 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71e13df740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514a8de6f0 con 0x56514a972670 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71e13df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514a8d9fe0 con 0x56514a973060 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71dcbbe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a9781d0 0x56514a97a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71dcbbe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a9781d0 0x56514a97a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39212/0 (socket says 172.21.15.45:39212) 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71dcbbe640 1 -- 172.21.15.45:0/1112607656 learned_addr learned my addr 172.21.15.45:0/1112607656 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71d7fff640 1 --2- 172.21.15.45:0/1112607656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a972670 0x56514a972a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71d77fe640 1 --2- 172.21.15.45:0/1112607656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a973060 0x56514a977b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71dcbbe640 1 -- 172.21.15.45:0/1112607656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a973060 msgr2=0x56514a977b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71dcbbe640 1 --2- 172.21.15.45:0/1112607656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a973060 0x56514a977b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71dcbbe640 1 -- 172.21.15.45:0/1112607656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a972670 msgr2=0x56514a972a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71dcbbe640 1 --2- 172.21.15.45:0/1112607656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a972670 0x56514a972a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71d77fe640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f71c00a12e0 0x7f71c00a37a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f71d8195f40 tx=0x7f71cc00be20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71dcbbe640 1 -- 172.21.15.45:0/1112607656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56514a89c650 con 0x56514a9781d0 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71d7fff640 1 --2- 172.21.15.45:0/1112607656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a972670 0x56514a972a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71dcbbe640 1 --2- 172.21.15.45:0/1112607656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a9781d0 0x56514a97a660 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f71d0096920 tx=0x7f71d00a1c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e7b55b3acc0f60 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71be7fc640 1 -- 172.21.15.45:0/1112607656 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d009e000 con 0x56514a9781d0 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71be7fc640 1 -- 172.21.15.45:0/1112607656 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71d00a0d00 con 0x56514a9781d0 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.301+0000 7f71be7fc640 1 -- 172.21.15.45:0/1112607656 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0005470 con 0x56514a9781d0 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.302+0000 7f71e13df740 1 -- 172.21.15.45:0/1112607656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a9781d0 msgr2=0x56514a97a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.302+0000 7f71e13df740 1 --2- 172.21.15.45:0/1112607656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a9781d0 0x56514a97a660 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f71d0096920 tx=0x7f71d00a1c60 comp rx=0 tx=0).stop 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.302+0000 7f71e13df740 1 -- 172.21.15.45:0/1112607656 shutdown_connections 2024-01-31T05:22:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.302+0000 7f71e13df740 1 --2- 172.21.15.45:0/1112607656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a9781d0 0x56514a97a660 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.302+0000 7f71e13df740 1 --2- 172.21.15.45:0/1112607656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a973060 0x56514a977b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.302+0000 7f71e13df740 1 --2- 172.21.15.45:0/1112607656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a972670 0x56514a972a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.302+0000 7f71e13df740 1 -- 172.21.15.45:0/1112607656 >> 172.21.15.45:0/1112607656 conn(0x56514a96dbe0 msgr2=0x56514a96dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.302+0000 7f71e13df740 1 -- 172.21.15.45:0/1112607656 shutdown_connections 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.302+0000 7f71e13df740 1 -- 172.21.15.45:0/1112607656 wait complete. 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.302+0000 7f71e13df740 1 -- start start 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.302+0000 7f71e13df740 1 Processor -- start 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71e13df740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a97c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71e13df740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 0x56514a97cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71e13df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 0x56514a983d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71e13df740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514a8935c0 con 0x56514a972670 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71e13df740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514a893260 con 0x56514a973060 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71e13df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514a8582d0 con 0x56514a9781d0 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71e13df740 10 client.0 _ll_register_callbacks cb 0x56514a88e230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a97c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a97c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39224/0 (socket says 172.21.15.45:39224) 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71d7fff640 1 -- 172.21.15.45:0/3458552670 learned_addr learned my addr 172.21.15.45:0/3458552670 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71dcbbe640 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 0x56514a983d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71d77fe640 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 0x56514a97cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71d7fff640 1 -- 172.21.15.45:0/3458552670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 msgr2=0x56514a983d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71d7fff640 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 0x56514a983d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71d7fff640 1 -- 172.21.15.45:0/3458552670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 msgr2=0x56514a97cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71d7fff640 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 0x56514a97cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71d7fff640 1 -- 172.21.15.45:0/3458552670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d819cff0 con 0x56514a972670 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71d7fff640 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a97c800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009980 tx=0x7f71c8004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843d7dbfb121ab0b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71b77fe640 1 -- 172.21.15.45:0/3458552670 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8004310 con 0x56514a972670 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71b77fe640 1 -- 172.21.15.45:0/3458552670 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f71c8047070 con 0x56514a972670 2024-01-31T05:22:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71e13df740 1 -- 172.21.15.45:0/3458552670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56514a8d2e70 con 0x56514a972670 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71b77fe640 1 -- 172.21.15.45:0/3458552670 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71c8004470 con 0x56514a972670 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71e13df740 10 client.15063 fetch_fsmap learned FSMap version 20 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.303+0000 7f71e13df740 1 -- 172.21.15.45:0/3458552670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56514a8d8660 con 0x56514a972670 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.304+0000 7f71b77fe640 1 -- 172.21.15.45:0/3458552670 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56514a8d2e70 con 0x56514a972670 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.304+0000 7f71b77fe640 1 -- 172.21.15.45:0/3458552670 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f71c803e040 con 0x56514a972670 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.304+0000 7f71e13df740 10 client.15063 fetch_fsmap finished waiting for FSMap version 20 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.304+0000 7f71e13df740 10 client.15063 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.304+0000 7f71e13df740 20 client.15063 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.304+0000 7f71e13df740 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x56514a989a80 0x56514a9884a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.304+0000 7f71e13df740 4 client.15063 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.304+0000 7f71e13df740 1 -- 172.21.15.45:0/3458552670 --> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56514a8d2b90 con 0x56514a989a80 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.304+0000 7f71d77fe640 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x56514a989a80 0x56514a9884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.304+0000 7f71b77fe640 10 client.15063 ms_handle_connect on v2:172.21.15.186:6836/2559285393 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.304+0000 7f71d77fe640 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x56514a989a80 0x56514a9884a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b87e3d676dec80da server_cookie=573ee23465efa0c4 in_seq=0 out_seq=0 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71b77fe640 1 -- 172.21.15.45:0/3458552670 <== mds.2 v2:172.21.15.186:6836/2559285393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56514a8d2b90 con 0x56514a989a80 2024-01-31T05:22:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71b77fe640 10 client.15063 handle_command_reply: tid=0 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71e13df740 1 client.15063 shutdown 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71e13df740 1 -- 172.21.15.45:0/3458552670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 msgr2=0x56514a97c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71e13df740 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a97c800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009980 tx=0x7f71c8004d30 comp rx=0 tx=0).stop 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71e13df740 1 -- 172.21.15.45:0/3458552670 shutdown_connections 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71e13df740 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 0x56514a983d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71e13df740 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 0x56514a97cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71e13df740 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x56514a989a80 0x56514a9884a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71e13df740 1 --2- 172.21.15.45:0/3458552670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a97c800 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71b77fe640 0 client.15063 ms_handle_reset on v2:172.21.15.186:6836/2559285393 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71e13df740 1 -- 172.21.15.45:0/3458552670 >> 172.21.15.45:0/3458552670 conn(0x56514a96dbe0 msgr2=0x56514a96cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.305+0000 7f71e13df740 1 -- 172.21.15.45:0/3458552670 shutdown_connections 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.306+0000 7f71e13df740 1 -- 172.21.15.45:0/3458552670 wait complete. 2024-01-31T05:22:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.306+0000 7f71e13df740 20 client.15063 trim_cache size 0 max 16384 2024-01-31T05:22:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.321+0000 7f71e13df740 1 Processor -- start 2024-01-31T05:22:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.321+0000 7f71e13df740 1 -- start start 2024-01-31T05:22:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.321+0000 7f71e13df740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a988bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.321+0000 7f71e13df740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 0x56514a96c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.321+0000 7f71e13df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 0x56514a96ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.321+0000 7f71e13df740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514a8d2e70 con 0x56514a972670 2024-01-31T05:22:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.321+0000 7f71e13df740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514a972e60 con 0x56514a973060 2024-01-31T05:22:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.321+0000 7f71e13df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514a9694b0 con 0x56514a9781d0 2024-01-31T05:22:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.321+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a988bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.321+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a988bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39230/0 (socket says 172.21.15.45:39230) 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.321+0000 7f71d7fff640 1 -- 172.21.15.45:0/4220595468 learned_addr learned my addr 172.21.15.45:0/4220595468 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71d7fff640 1 -- 172.21.15.45:0/4220595468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 msgr2=0x56514a96ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71d7fff640 1 --2- 172.21.15.45:0/4220595468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 0x56514a96ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71d7fff640 1 -- 172.21.15.45:0/4220595468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 msgr2=0x56514a96c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71d7fff640 1 --2- 172.21.15.45:0/4220595468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 0x56514a96c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71d7fff640 1 -- 172.21.15.45:0/4220595468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56514a875360 con 0x56514a972670 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71d7fff640 1 --2- 172.21.15.45:0/4220595468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a988bb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f71c80098d0 tx=0x7f71c802de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a8d6c3b6b471c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71bdffb640 1 -- 172.21.15.45:0/4220595468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8004510 con 0x56514a972670 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71bdffb640 1 -- 172.21.15.45:0/4220595468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71c8045070 con 0x56514a972670 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71bdffb640 1 -- 172.21.15.45:0/4220595468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8005660 con 0x56514a972670 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71e13df740 1 -- 172.21.15.45:0/4220595468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 msgr2=0x56514a988bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71e13df740 1 --2- 172.21.15.45:0/4220595468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a988bb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f71c80098d0 tx=0x7f71c802de70 comp rx=0 tx=0).stop 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71e13df740 1 -- 172.21.15.45:0/4220595468 shutdown_connections 2024-01-31T05:22:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71e13df740 1 --2- 172.21.15.45:0/4220595468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 0x56514a96ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71e13df740 1 --2- 172.21.15.45:0/4220595468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 0x56514a96c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71e13df740 1 --2- 172.21.15.45:0/4220595468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a988bb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.322+0000 7f71e13df740 1 -- 172.21.15.45:0/4220595468 >> 172.21.15.45:0/4220595468 conn(0x56514a96dbe0 msgr2=0x56514a9884a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71e13df740 1 -- 172.21.15.45:0/4220595468 shutdown_connections 2024-01-31T05:22:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71e13df740 1 -- 172.21.15.45:0/4220595468 wait complete. 2024-01-31T05:22:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71e13df740 1 -- start start 2024-01-31T05:22:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71e13df740 1 Processor -- start 2024-01-31T05:22:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71e13df740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a9dc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71e13df740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 0x56514a9dca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71e13df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 0x56514a9e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71e13df740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514a9e3f40 con 0x56514a972670 2024-01-31T05:22:04.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71e13df740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514a9e4080 con 0x56514a973060 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71e13df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514a9e41f0 con 0x56514a9781d0 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71e13df740 10 client.0 _ll_register_callbacks cb 0x7f71d00961f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a9dc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a9dc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39238/0 (socket says 172.21.15.45:39238) 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71d7fff640 1 -- 172.21.15.45:0/924740313 learned_addr learned my addr 172.21.15.45:0/924740313 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.323+0000 7f71dcbbe640 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 0x56514a9e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71d7fff640 1 -- 172.21.15.45:0/924740313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 msgr2=0x56514a9e3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71d7fff640 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 0x56514a9e3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71d7fff640 1 -- 172.21.15.45:0/924740313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 msgr2=0x56514a9dca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71d7fff640 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 0x56514a9dca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71d7fff640 1 -- 172.21.15.45:0/924740313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56514a8d8660 con 0x56514a972670 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71dcbbe640 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 0x56514a9e3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71d7fff640 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a9dc520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009980 tx=0x7f71c802ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b53a1f5721a80c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71b77fe640 1 -- 172.21.15.45:0/924740313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c80053a0 con 0x56514a972670 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71b77fe640 1 -- 172.21.15.45:0/924740313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f71c8000e40 con 0x56514a972670 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71e13df740 1 -- 172.21.15.45:0/924740313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56514a989790 con 0x56514a972670 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71b77fe640 1 -- 172.21.15.45:0/924740313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71c8005500 con 0x56514a972670 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71e13df740 10 client.15069 fetch_fsmap learned FSMap version 20 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71e13df740 1 -- 172.21.15.45:0/924740313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56514a89aeb0 con 0x56514a972670 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71b77fe640 1 -- 172.21.15.45:0/924740313 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56514a989790 con 0x56514a972670 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71b77fe640 1 -- 172.21.15.45:0/924740313 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f71c803e040 con 0x56514a972670 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71e13df740 10 client.15069 fetch_fsmap finished waiting for FSMap version 20 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71e13df740 10 client.15069 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.324+0000 7f71e13df740 20 client.15069 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.325+0000 7f71e13df740 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x56514a9e9070 0x56514a9e54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.325+0000 7f71e13df740 4 client.15069 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.325+0000 7f71e13df740 1 -- 172.21.15.45:0/924740313 --> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f71d00a0d00 con 0x56514a9e9070 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.325+0000 7f71d77fe640 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x56514a9e9070 0x56514a9e54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.325+0000 7f71d77fe640 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x56514a9e9070 0x56514a9e54e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f78635c7ba54b3d9 server_cookie=db2c398cfc3f3b10 in_seq=0 out_seq=0 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.325+0000 7f71b77fe640 10 client.15069 ms_handle_connect on v2:172.21.15.186:6836/2559285393 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.325+0000 7f71b77fe640 1 -- 172.21.15.45:0/924740313 <== mds.2 v2:172.21.15.186:6836/2559285393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f71d00a0d00 con 0x56514a9e9070 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.325+0000 7f71b77fe640 10 client.15069 handle_command_reply: tid=0 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.325+0000 7f71e13df740 1 client.15069 shutdown 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71e13df740 1 -- 172.21.15.45:0/924740313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 msgr2=0x56514a9dc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71e13df740 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a9dc520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009980 tx=0x7f71c802ebb0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71e13df740 1 -- 172.21.15.45:0/924740313 shutdown_connections 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71e13df740 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56514a9781d0 0x56514a9e3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71e13df740 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56514a973060 0x56514a9dca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71e13df740 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x56514a9e9070 0x56514a9e54e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71e13df740 1 --2- 172.21.15.45:0/924740313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514a972670 0x56514a9dc520 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71b77fe640 0 client.15069 ms_handle_reset on v2:172.21.15.186:6836/2559285393 2024-01-31T05:22:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71e13df740 1 -- 172.21.15.45:0/924740313 >> 172.21.15.45:0/924740313 conn(0x56514a96dbe0 msgr2=0x56514a96c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71e13df740 1 -- 172.21.15.45:0/924740313 shutdown_connections 2024-01-31T05:22:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71e13df740 1 -- 172.21.15.45:0/924740313 wait complete. 2024-01-31T05:22:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71e13df740 20 client.15069 trim_cache size 0 max 16384 2024-01-31T05:22:04.333 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-01-31T05:22:04.333 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "3a1f4122-bff7-11ee-95b3-87774f69a715", 2024-01-31T05:22:04.333 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2024-01-31T05:22:04.333 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24469, 2024-01-31T05:22:04.333 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-01-31T05:22:04.334 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-01-31T05:22:04.334 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-01-31T05:22:04.334 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 80.854493263999998, 2024-01-31T05:22:04.334 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 20, 2024-01-31T05:22:04.334 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 88, 2024-01-31T05:22:04.334 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 86, 2024-01-31T05:22:04.334 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 103.11435990699999 2024-01-31T05:22:04.334 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-01-31T05:22:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71de621640 1 -- 172.21.15.45:0/2721316588 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f71c00a12e0 msgr2=0x7f71c00a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71de621640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f71c00a12e0 0x7f71c00a37a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f71d8195f40 tx=0x7f71cc00be20 comp rx=0 tx=0).stop 2024-01-31T05:22:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71de621640 1 -- 172.21.15.45:0/2721316588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81017a0 msgr2=0x7f71d819bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71de621640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81017a0 0x7f71d819bfe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f71d000eaa0 tx=0x7f71d000ef60 comp rx=0 tx=0).stop 2024-01-31T05:22:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71de621640 1 -- 172.21.15.45:0/2721316588 shutdown_connections 2024-01-31T05:22:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71de621640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f71c00a12e0 0x7f71c00a37a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71de621640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d81017a0 0x7f71d819bfe0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71de621640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71d8100de0 0x7f71d8194f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71de621640 1 --2- 172.21.15.45:0/2721316588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71d80ffbf0 0x7f71d8194a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71de621640 1 -- 172.21.15.45:0/2721316588 >> 172.21.15.45:0/2721316588 conn(0x7f71d80fb180 msgr2=0x7f71d81041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71de621640 1 -- 172.21.15.45:0/2721316588 shutdown_connections 2024-01-31T05:22:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.326+0000 7f71de621640 1 -- 172.21.15.45:0/2721316588 wait complete. 2024-01-31T05:22:04.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-01-31T05:22:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.441+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3502067227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d841028b0 msgr2=0x7f8d84109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.441+0000 7f8d8c1bb640 1 --2- 172.21.15.45:0/3502067227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d841028b0 0x7f8d84109140 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009ca0 tx=0x7f8d7402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.441+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3502067227 shutdown_connections 2024-01-31T05:22:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.441+0000 7f8d8c1bb640 1 --2- 172.21.15.45:0/3502067227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d841028b0 0x7f8d84109140 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.441+0000 7f8d8c1bb640 1 --2- 172.21.15.45:0/3502067227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d84101ef0 0x7f8d84102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.441+0000 7f8d8c1bb640 1 --2- 172.21.15.45:0/3502067227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d84100d00 0x7f8d84101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.441+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3502067227 >> 172.21.15.45:0/3502067227 conn(0x7f8d840fc290 msgr2=0x7f8d840fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.441+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3502067227 shutdown_connections 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.441+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3502067227 wait complete. 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8c1bb640 1 Processor -- start 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8c1bb640 1 -- start start 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8c1bb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d84100d00 0x7f8d84196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8c1bb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d84101ef0 0x7f8d841971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8c1bb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d841028b0 0x7f8d8419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8c1bb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8419e790 con 0x7f8d841028b0 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8c1bb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8419e900 con 0x7f8d84100d00 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8c1bb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8419eac0 con 0x7f8d84101ef0 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8a758640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d841028b0 0x7f8d8419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8a758640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d841028b0 0x7f8d8419e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39266/0 (socket says 172.21.15.45:39266) 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8a758640 1 -- 172.21.15.45:0/3972733826 learned_addr learned my addr 172.21.15.45:0/3972733826 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8a758640 1 -- 172.21.15.45:0/3972733826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d84101ef0 msgr2=0x7f8d841971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d89f57640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d84100d00 0x7f8d84196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8a758640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d84101ef0 0x7f8d841971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8a758640 1 -- 172.21.15.45:0/3972733826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d84100d00 msgr2=0x7f8d84196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8a758640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d84100d00 0x7f8d84196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8a758640 1 -- 172.21.15.45:0/3972733826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74009900 con 0x7f8d841028b0 2024-01-31T05:22:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8a758640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d841028b0 0x7f8d8419e220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8d7402c630 tx=0x7f8d740329c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d82ffd640 1 -- 172.21.15.45:0/3972733826 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74034e80 con 0x7f8d841028b0 2024-01-31T05:22:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3972733826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8419ed40 con 0x7f8d841028b0 2024-01-31T05:22:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d82ffd640 1 -- 172.21.15.45:0/3972733826 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d74032e90 con 0x7f8d841028b0 2024-01-31T05:22:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d82ffd640 1 -- 172.21.15.45:0/3972733826 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d740048a0 con 0x7f8d841028b0 2024-01-31T05:22:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.442+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3972733826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8419f1f0 con 0x7f8d841028b0 2024-01-31T05:22:04.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.444+0000 7f8d82ffd640 1 -- 172.21.15.45:0/3972733826 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f8d7403b040 con 0x7f8d841028b0 2024-01-31T05:22:04.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d8ef78740 1 Processor -- start 2024-01-31T05:22:04.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d8ef78740 1 -- start start 2024-01-31T05:22:04.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d82ffd640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8d6c0a12e0 0x7f8d6c0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d8ef78740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccfea80 0x5633bccfeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d8ef78740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccff470 0x5633bcd03f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d8ef78740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bcd06a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d8ef78740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bcc69000 con 0x5633bcd045e0 2024-01-31T05:22:04.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d8ef78740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bcc666f0 con 0x5633bccff470 2024-01-31T05:22:04.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d8ef78740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bcc61fe0 con 0x5633bccfea80 2024-01-31T05:22:04.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d89756640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8d6c0a12e0 0x7f8d6c0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d82ffd640 1 -- 172.21.15.45:0/3972733826 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8d740ef2c0 con 0x7f8d841028b0 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d8a758640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bcd06a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d8a758640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bcd06a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39278/0 (socket says 172.21.15.45:39278) 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.445+0000 7f8d8a758640 1 -- 172.21.15.45:0/2030395048 learned_addr learned my addr 172.21.15.45:0/2030395048 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d89f57640 1 --2- 172.21.15.45:0/2030395048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccfea80 0x5633bccfeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d89756640 1 --2- 172.21.15.45:0/2030395048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccff470 0x5633bcd03f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d8a758640 1 -- 172.21.15.45:0/2030395048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccfea80 msgr2=0x5633bccfeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d8a758640 1 --2- 172.21.15.45:0/2030395048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccfea80 0x5633bccfeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d8a758640 1 -- 172.21.15.45:0/2030395048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccff470 msgr2=0x5633bcd03f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d8a758640 1 --2- 172.21.15.45:0/2030395048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccff470 0x5633bcd03f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d8a758640 1 -- 172.21.15.45:0/2030395048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633bcc24650 con 0x5633bcd045e0 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d89756640 1 --2- 172.21.15.45:0/2030395048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccff470 0x5633bcd03f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d89756640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8d6c0a12e0 0x7f8d6c0a37a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8d84198180 tx=0x7f8d7800b4b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d8a758640 1 --2- 172.21.15.45:0/2030395048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bcd06a70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8d740c4290 tx=0x7f8d7402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06fae44e679153d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d6bfff640 1 -- 172.21.15.45:0/2030395048 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d740bc430 con 0x5633bcd045e0 2024-01-31T05:22:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d6bfff640 1 -- 172.21.15.45:0/2030395048 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d740c0580 con 0x5633bcd045e0 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.446+0000 7f8d6bfff640 1 -- 172.21.15.45:0/2030395048 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d740bb920 con 0x5633bcd045e0 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.447+0000 7f8d8ef78740 1 -- 172.21.15.45:0/2030395048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 msgr2=0x5633bcd06a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.447+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/2030395048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bcd06a70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8d740c4290 tx=0x7f8d7402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.447+0000 7f8d8ef78740 1 -- 172.21.15.45:0/2030395048 shutdown_connections 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.447+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/2030395048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bcd06a70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.447+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/2030395048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccff470 0x5633bcd03f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.447+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/2030395048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccfea80 0x5633bccfeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.447+0000 7f8d8ef78740 1 -- 172.21.15.45:0/2030395048 >> 172.21.15.45:0/2030395048 conn(0x5633bccf5be0 msgr2=0x5633bccf5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.447+0000 7f8d8ef78740 1 -- 172.21.15.45:0/2030395048 shutdown_connections 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.447+0000 7f8d8ef78740 1 -- 172.21.15.45:0/2030395048 wait complete. 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.447+0000 7f8d8ef78740 1 -- start start 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.447+0000 7f8d8ef78740 1 Processor -- start 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8ef78740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccfea80 0x5633bcd08850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8ef78740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccff470 0x5633bcd08d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8ef78740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bcd0fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8ef78740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bcc1b5c0 con 0x5633bcd045e0 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8ef78740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bcc1b260 con 0x5633bccff470 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8ef78740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bcbe02d0 con 0x5633bccfea80 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8ef78740 10 client.0 _ll_register_callbacks cb 0x5633bcc16230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8a758640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bcd0fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8a758640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bcd0fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39292/0 (socket says 172.21.15.45:39292) 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8a758640 1 -- 172.21.15.45:0/1967157967 learned_addr learned my addr 172.21.15.45:0/1967157967 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d89f57640 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccfea80 0x5633bcd08850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d89756640 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccff470 0x5633bcd08d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8a758640 1 -- 172.21.15.45:0/1967157967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccfea80 msgr2=0x5633bcd08850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8a758640 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccfea80 0x5633bcd08850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8a758640 1 -- 172.21.15.45:0/1967157967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccff470 msgr2=0x5633bcd08d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8a758640 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccff470 0x5633bcd08d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8a758640 1 -- 172.21.15.45:0/1967157967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8419f1f0 con 0x5633bcd045e0 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d89756640 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccff470 0x5633bcd08d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8a758640 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bcd0fdd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8d740b8930 tx=0x7f8d740bbe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08b7e5e09fff72a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d697fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d697fa640 1 -- 172.21.15.45:0/1967157967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d740c0320 con 0x5633bcd045e0 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d697fa640 1 -- 172.21.15.45:0/1967157967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8d740d3030 con 0x5633bcd045e0 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8ef78740 1 -- 172.21.15.45:0/1967157967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633bcc5ae70 con 0x5633bcd045e0 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d697fa640 1 -- 172.21.15.45:0/1967157967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d740bb920 con 0x5633bcd045e0 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8ef78740 10 client.15081 fetch_fsmap learned FSMap version 20 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.448+0000 7f8d8ef78740 1 -- 172.21.15.45:0/1967157967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5633bcc60660 con 0x5633bcd045e0 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.449+0000 7f8d697fa640 1 -- 172.21.15.45:0/1967157967 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633bcc5ae70 con 0x5633bcd045e0 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.449+0000 7f8d697fa640 1 -- 172.21.15.45:0/1967157967 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8d740ee070 con 0x5633bcd045e0 2024-01-31T05:22:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.449+0000 7f8d8ef78740 10 client.15081 fetch_fsmap finished waiting for FSMap version 20 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.449+0000 7f8d8ef78740 10 client.15081 resolve_mds: resolved name 'i' to daemon mds.i 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.449+0000 7f8d8ef78740 20 client.15081 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.449+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x5633bcd15660 0x5633bcd14230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.449+0000 7f8d8ef78740 4 client.15081 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.449+0000 7f8d8ef78740 1 -- 172.21.15.45:0/1967157967 --> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633bcc5ab90 con 0x5633bcd15660 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.449+0000 7f8d89f57640 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x5633bcd15660 0x5633bcd14230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.450+0000 7f8d697fa640 10 client.15081 ms_handle_connect on v2:172.21.15.186:6838/3821255404 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.450+0000 7f8d89f57640 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x5633bcd15660 0x5633bcd14230 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32ea84be39b89218 server_cookie=619fae466e57de9d in_seq=0 out_seq=0 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.450+0000 7f8d697fa640 1 -- 172.21.15.45:0/1967157967 <== mds.1 v2:172.21.15.186:6838/3821255404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5633bcc5ab90 con 0x5633bcd15660 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.450+0000 7f8d697fa640 10 client.15081 handle_command_reply: tid=0 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.450+0000 7f8d8ef78740 1 client.15081 shutdown 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.450+0000 7f8d8ef78740 1 -- 172.21.15.45:0/1967157967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 msgr2=0x5633bcd0fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.450+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bcd0fdd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8d740b8930 tx=0x7f8d740bbe90 comp rx=0 tx=0).stop 2024-01-31T05:22:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.450+0000 7f8d8ef78740 1 -- 172.21.15.45:0/1967157967 shutdown_connections 2024-01-31T05:22:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.451+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x5633bcd15660 0x5633bcd14230 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.451+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bcd0fdd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.451+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccff470 0x5633bcd08d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.451+0000 7f8d697fa640 0 client.15081 ms_handle_reset on v2:172.21.15.186:6838/3821255404 2024-01-31T05:22:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.451+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/1967157967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccfea80 0x5633bcd08850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.451+0000 7f8d8ef78740 1 -- 172.21.15.45:0/1967157967 >> 172.21.15.45:0/1967157967 conn(0x5633bccf5be0 msgr2=0x5633bccf4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.451+0000 7f8d8ef78740 1 -- 172.21.15.45:0/1967157967 shutdown_connections 2024-01-31T05:22:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.451+0000 7f8d8ef78740 1 -- 172.21.15.45:0/1967157967 wait complete. 2024-01-31T05:22:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.451+0000 7f8d8ef78740 20 client.15081 trim_cache size 0 max 16384 2024-01-31T05:22:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.466+0000 7f8d8ef78740 1 Processor -- start 2024-01-31T05:22:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.466+0000 7f8d8ef78740 1 -- start start 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.466+0000 7f8d8ef78740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccfea80 0x5633bcd14940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.466+0000 7f8d8ef78740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccff470 0x5633bcd0fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.466+0000 7f8d8ef78740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bccf44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.466+0000 7f8d8ef78740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bcc69000 con 0x5633bcd045e0 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.466+0000 7f8d8ef78740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bcd15230 con 0x5633bccfea80 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.466+0000 7f8d8ef78740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bcd07130 con 0x5633bccff470 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.466+0000 7f8d8a758640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bccf44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.466+0000 7f8d8a758640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bccf44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39300/0 (socket says 172.21.15.45:39300) 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.466+0000 7f8d8a758640 1 -- 172.21.15.45:0/422507917 learned_addr learned my addr 172.21.15.45:0/422507917 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.467+0000 7f8d89f57640 1 --2- 172.21.15.45:0/422507917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccfea80 0x5633bcd14940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.467+0000 7f8d89756640 1 --2- 172.21.15.45:0/422507917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccff470 0x5633bcd0fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.467+0000 7f8d8a758640 1 -- 172.21.15.45:0/422507917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccff470 msgr2=0x5633bcd0fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.467+0000 7f8d8a758640 1 --2- 172.21.15.45:0/422507917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccff470 0x5633bcd0fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.467+0000 7f8d8a758640 1 -- 172.21.15.45:0/422507917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccfea80 msgr2=0x5633bcd14940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.467+0000 7f8d8a758640 1 --2- 172.21.15.45:0/422507917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccfea80 0x5633bcd14940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.467+0000 7f8d8a758640 1 -- 172.21.15.45:0/422507917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633bcbfd360 con 0x5633bcd045e0 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.467+0000 7f8d89756640 1 --2- 172.21.15.45:0/422507917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccff470 0x5633bcd0fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.467+0000 7f8d8a758640 1 --2- 172.21.15.45:0/422507917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bccf44e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8d740ba780 tx=0x7f8d740c4910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab15de9d253f6d10 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.467+0000 7f8d6b7fe640 1 -- 172.21.15.45:0/422507917 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74032b30 con 0x5633bcd045e0 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.467+0000 7f8d6b7fe640 1 -- 172.21.15.45:0/422507917 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d740040b0 con 0x5633bcd045e0 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.467+0000 7f8d6b7fe640 1 -- 172.21.15.45:0/422507917 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d740c0120 con 0x5633bcd045e0 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 -- 172.21.15.45:0/422507917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 msgr2=0x5633bccf44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/422507917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bccf44e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8d740ba780 tx=0x7f8d740c4910 comp rx=0 tx=0).stop 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 -- 172.21.15.45:0/422507917 shutdown_connections 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/422507917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bcd045e0 0x5633bccf44e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/422507917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bccff470 0x5633bcd0fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/422507917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccfea80 0x5633bcd14940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 -- 172.21.15.45:0/422507917 >> 172.21.15.45:0/422507917 conn(0x5633bccf5be0 msgr2=0x5633bcd14230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 -- 172.21.15.45:0/422507917 shutdown_connections 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 -- 172.21.15.45:0/422507917 wait complete. 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 -- start start 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 Processor -- start 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccfea80 0x5633bcd0fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bccff470 0x5633bcd0c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bcd045e0 0x5633bcd0cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bcd0d400 con 0x5633bccff470 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bcd6a5a0 con 0x5633bccfea80 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bcd6a710 con 0x5633bcd045e0 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8ef78740 10 client.0 _ll_register_callbacks cb 0x7f8d740b8840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d89756640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bccff470 0x5633bcd0c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d89756640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bccff470 0x5633bcd0c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39316/0 (socket says 172.21.15.45:39316) 2024-01-31T05:22:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d89756640 1 -- 172.21.15.45:0/378521585 learned_addr learned my addr 172.21.15.45:0/378521585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d89f57640 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccfea80 0x5633bcd0fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.468+0000 7f8d8a758640 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bcd045e0 0x5633bcd0cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d89756640 1 -- 172.21.15.45:0/378521585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bcd045e0 msgr2=0x5633bcd0cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d89756640 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bcd045e0 0x5633bcd0cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d89756640 1 -- 172.21.15.45:0/378521585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccfea80 msgr2=0x5633bcd0fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d89756640 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccfea80 0x5633bcd0fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d89756640 1 -- 172.21.15.45:0/378521585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633bcc60660 con 0x5633bccff470 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d89756640 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bccff470 0x5633bcd0c810 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5633bccffee0 tx=0x7f8d78005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f639fd32d95dfe6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d697fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d697fa640 1 -- 172.21.15.45:0/378521585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d780117b0 con 0x5633bccff470 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d697fa640 1 -- 172.21.15.45:0/378521585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8d7801a040 con 0x5633bccff470 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d8ef78740 1 -- 172.21.15.45:0/378521585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633bcd15370 con 0x5633bccff470 2024-01-31T05:22:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d697fa640 1 -- 172.21.15.45:0/378521585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d78011910 con 0x5633bccff470 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d8ef78740 10 client.15087 fetch_fsmap learned FSMap version 20 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d8ef78740 1 -- 172.21.15.45:0/378521585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5633bcc22eb0 con 0x5633bccff470 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d697fa640 1 -- 172.21.15.45:0/378521585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633bcd15370 con 0x5633bccff470 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.469+0000 7f8d697fa640 1 -- 172.21.15.45:0/378521585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8d78021cd0 con 0x5633bccff470 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d8ef78740 10 client.15087 fetch_fsmap finished waiting for FSMap version 20 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d8ef78740 10 client.15087 resolve_mds: resolved name 'i' to daemon mds.i 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d8ef78740 20 client.15087 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x5633bcd6fad0 0x5633bcd6bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d8ef78740 4 client.15087 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d8ef78740 1 -- 172.21.15.45:0/378521585 --> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f8d740c0580 con 0x5633bcd6fad0 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d89f57640 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x5633bcd6fad0 0x5633bcd6bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d89f57640 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x5633bcd6fad0 0x5633bcd6bf30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95e5cfa66db29eee server_cookie=990aca45c2a2b2c3 in_seq=0 out_seq=0 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d697fa640 10 client.15087 ms_handle_connect on v2:172.21.15.186:6838/3821255404 2024-01-31T05:22:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d697fa640 1 -- 172.21.15.45:0/378521585 <== mds.1 v2:172.21.15.186:6838/3821255404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f8d740c0580 con 0x5633bcd6fad0 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d697fa640 10 client.15087 handle_command_reply: tid=0 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d8ef78740 1 client.15087 shutdown 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.470+0000 7f8d8ef78740 1 -- 172.21.15.45:0/378521585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bccff470 msgr2=0x5633bcd0c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bccff470 0x5633bcd0c810 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5633bccffee0 tx=0x7f8d78005850 comp rx=0 tx=0).stop 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8ef78740 1 -- 172.21.15.45:0/378521585 shutdown_connections 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x5633bcd6fad0 0x5633bcd6bf30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633bccff470 0x5633bcd0c810 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633bcd045e0 0x5633bcd0cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d697fa640 0 client.15087 ms_handle_reset on v2:172.21.15.186:6838/3821255404 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8ef78740 1 --2- 172.21.15.45:0/378521585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633bccfea80 0x5633bcd0fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8ef78740 1 -- 172.21.15.45:0/378521585 >> 172.21.15.45:0/378521585 conn(0x5633bccf5be0 msgr2=0x5633bccf44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8ef78740 1 -- 172.21.15.45:0/378521585 shutdown_connections 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8ef78740 1 -- 172.21.15.45:0/378521585 wait complete. 2024-01-31T05:22:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8ef78740 20 client.15087 trim_cache size 0 max 16384 2024-01-31T05:22:04.479 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-01-31T05:22:04.479 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "3a1f4122-bff7-11ee-95b3-87774f69a715", 2024-01-31T05:22:04.479 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2024-01-31T05:22:04.479 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24490, 2024-01-31T05:22:04.479 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-01-31T05:22:04.479 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-01-31T05:22:04.479 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-01-31T05:22:04.479 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 82.016163700999996, 2024-01-31T05:22:04.479 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 20, 2024-01-31T05:22:04.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 88, 2024-01-31T05:22:04.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 86, 2024-01-31T05:22:04.479 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 97.656225246000005 2024-01-31T05:22:04.480 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-01-31T05:22:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3972733826 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8d6c0a12e0 msgr2=0x7f8d6c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8c1bb640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8d6c0a12e0 0x7f8d6c0a37a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8d84198180 tx=0x7f8d7800b4b0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3972733826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d841028b0 msgr2=0x7f8d8419e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8c1bb640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d841028b0 0x7f8d8419e220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8d7402c630 tx=0x7f8d740329c0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3972733826 shutdown_connections 2024-01-31T05:22:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8c1bb640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8d6c0a12e0 0x7f8d6c0a37a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8c1bb640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d841028b0 0x7f8d8419e220 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8c1bb640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d84101ef0 0x7f8d841971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8c1bb640 1 --2- 172.21.15.45:0/3972733826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d84100d00 0x7f8d84196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3972733826 >> 172.21.15.45:0/3972733826 conn(0x7f8d840fc290 msgr2=0x7f8d84105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3972733826 shutdown_connections 2024-01-31T05:22:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.471+0000 7f8d8c1bb640 1 -- 172.21.15.45:0/3972733826 wait complete. 2024-01-31T05:22:04.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-31T05:22:04.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.585+0000 7fd66cb89640 1 -- 172.21.15.45:0/562350687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6681017b0 msgr2=0x7fd668108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.585+0000 7fd66cb89640 1 --2- 172.21.15.45:0/562350687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6681017b0 0x7fd668108040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd660009ca0 tx=0x7fd66002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.585+0000 7fd66cb89640 1 -- 172.21.15.45:0/562350687 shutdown_connections 2024-01-31T05:22:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.585+0000 7fd66cb89640 1 --2- 172.21.15.45:0/562350687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6681017b0 0x7fd668108040 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.585+0000 7fd66cb89640 1 --2- 172.21.15.45:0/562350687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd668100df0 0x7fd668101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.585+0000 7fd66cb89640 1 --2- 172.21.15.45:0/562350687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6680ffc00 0x7fd668100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.585+0000 7fd66cb89640 1 -- 172.21.15.45:0/562350687 >> 172.21.15.45:0/562350687 conn(0x7fd6680fb190 msgr2=0x7fd6680fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.585+0000 7fd66cb89640 1 -- 172.21.15.45:0/562350687 shutdown_connections 2024-01-31T05:22:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.585+0000 7fd66cb89640 1 -- 172.21.15.45:0/562350687 wait complete. 2024-01-31T05:22:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd66cb89640 1 Processor -- start 2024-01-31T05:22:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd66cb89640 1 -- start start 2024-01-31T05:22:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd66cb89640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6680ffc00 0x7fd668195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd66cb89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd668100df0 0x7fd6681960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd66cb89640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6681017b0 0x7fd66819d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd66cb89640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66819d690 con 0x7fd668100df0 2024-01-31T05:22:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd66cb89640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66819d800 con 0x7fd6680ffc00 2024-01-31T05:22:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd66cb89640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66819d9c0 con 0x7fd6681017b0 2024-01-31T05:22:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd65ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd668100df0 0x7fd6681960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd65ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd668100df0 0x7fd6681960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39338/0 (socket says 172.21.15.45:39338) 2024-01-31T05:22:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd65ffff640 1 -- 172.21.15.45:0/3705643546 learned_addr learned my addr 172.21.15.45:0/3705643546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd666d9d640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6681017b0 0x7fd66819d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd66659c640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6680ffc00 0x7fd668195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd65ffff640 1 -- 172.21.15.45:0/3705643546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6681017b0 msgr2=0x7fd66819d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd65ffff640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6681017b0 0x7fd66819d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd65ffff640 1 -- 172.21.15.45:0/3705643546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6680ffc00 msgr2=0x7fd668195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd65ffff640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6680ffc00 0x7fd668195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd65ffff640 1 -- 172.21.15.45:0/3705643546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660009900 con 0x7fd668100df0 2024-01-31T05:22:04.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd66659c640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6680ffc00 0x7fd668195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:04.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd65ffff640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd668100df0 0x7fd6681960e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd65000d950 tx=0x7fd65000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.586+0000 7fd65f7fe640 1 -- 172.21.15.45:0/3705643546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd650009c30 con 0x7fd668100df0 2024-01-31T05:22:04.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.587+0000 7fd65f7fe640 1 -- 172.21.15.45:0/3705643546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd650009d90 con 0x7fd668100df0 2024-01-31T05:22:04.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.587+0000 7fd66cb89640 1 -- 172.21.15.45:0/3705643546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66819dca0 con 0x7fd668100df0 2024-01-31T05:22:04.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.587+0000 7fd65f7fe640 1 -- 172.21.15.45:0/3705643546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65000f740 con 0x7fd668100df0 2024-01-31T05:22:04.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.587+0000 7fd66cb89640 1 -- 172.21.15.45:0/3705643546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66819e1b0 con 0x7fd668100df0 2024-01-31T05:22:04.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.589+0000 7fd65f7fe640 1 -- 172.21.15.45:0/3705643546 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fd650010a20 con 0x7fd668100df0 2024-01-31T05:22:04.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.589+0000 7fd66fa28740 1 Processor -- start 2024-01-31T05:22:04.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.589+0000 7fd65f7fe640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd6540a1360 0x7fd6540a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.589+0000 7fd666d9d640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd6540a1360 0x7fd6540a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.589+0000 7fd65f7fe640 1 -- 172.21.15.45:0/3705643546 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd6500cc770 con 0x7fd668100df0 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.589+0000 7fd66fa28740 1 -- start start 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66fa28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 0x555c26e64420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66fa28740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e64cf0 0x555c26e65170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66fa28740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e656b0 0x555c26e6bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66fa28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26dce000 con 0x555c26e64000 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66fa28740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26dcb6f0 con 0x555c26e64cf0 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66fa28740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26dc6fe0 con 0x555c26e656b0 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 0x555c26e64420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 0x555c26e64420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39340/0 (socket says 172.21.15.45:39340) 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66659c640 1 -- 172.21.15.45:0/989202824 learned_addr learned my addr 172.21.15.45:0/989202824 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd666d9d640 1 --2- 172.21.15.45:0/989202824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e656b0 0x555c26e6bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd65ffff640 1 --2- 172.21.15.45:0/989202824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e64cf0 0x555c26e65170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd666d9d640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd6540a1360 0x7fd6540a3820 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd660005e30 tx=0x7fd660032690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66659c640 1 -- 172.21.15.45:0/989202824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e656b0 msgr2=0x555c26e6bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66659c640 1 --2- 172.21.15.45:0/989202824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e656b0 0x555c26e6bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66659c640 1 -- 172.21.15.45:0/989202824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e64cf0 msgr2=0x555c26e65170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66659c640 1 --2- 172.21.15.45:0/989202824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e64cf0 0x555c26e65170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66659c640 1 -- 172.21.15.45:0/989202824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c26d89650 con 0x555c26e64000 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd65ffff640 1 --2- 172.21.15.45:0/989202824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e64cf0 0x555c26e65170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd66659c640 1 --2- 172.21.15.45:0/989202824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 0x555c26e64420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd65800bd30 tx=0x7fd65800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40664af0f5ec03ac server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd65cff9640 1 -- 172.21.15.45:0/989202824 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65801a660 con 0x555c26e64000 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd65cff9640 1 -- 172.21.15.45:0/989202824 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd65800be20 con 0x555c26e64000 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.590+0000 7fd65cff9640 1 -- 172.21.15.45:0/989202824 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658010750 con 0x555c26e64000 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.591+0000 7fd66fa28740 1 -- 172.21.15.45:0/989202824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 msgr2=0x555c26e64420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.591+0000 7fd66fa28740 1 --2- 172.21.15.45:0/989202824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 0x555c26e64420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd65800bd30 tx=0x7fd65800f730 comp rx=0 tx=0).stop 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.591+0000 7fd66fa28740 1 -- 172.21.15.45:0/989202824 shutdown_connections 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.591+0000 7fd66fa28740 1 --2- 172.21.15.45:0/989202824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e656b0 0x555c26e6bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.591+0000 7fd66fa28740 1 --2- 172.21.15.45:0/989202824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e64cf0 0x555c26e65170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.591+0000 7fd66fa28740 1 --2- 172.21.15.45:0/989202824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 0x555c26e64420 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.591+0000 7fd66fa28740 1 -- 172.21.15.45:0/989202824 >> 172.21.15.45:0/989202824 conn(0x555c26e5c220 msgr2=0x555c26e5a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.591+0000 7fd66fa28740 1 -- 172.21.15.45:0/989202824 shutdown_connections 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.591+0000 7fd66fa28740 1 -- 172.21.15.45:0/989202824 wait complete. 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.591+0000 7fd66fa28740 1 -- start start 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd66fa28740 1 Processor -- start 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd66fa28740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e64000 0x555c26e6dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd66fa28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 0x555c26e6e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd66fa28740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e656b0 0x555c26e751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd66fa28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26d805c0 con 0x555c26e64cf0 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd66fa28740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26d80260 con 0x555c26e656b0 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd66fa28740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26d452d0 con 0x555c26e64000 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd66fa28740 10 client.0 _ll_register_callbacks cb 0x555c26d7b230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd65ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 0x555c26e6e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd65ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 0x555c26e6e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39342/0 (socket says 172.21.15.45:39342) 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd65ffff640 1 -- 172.21.15.45:0/2696207583 learned_addr learned my addr 172.21.15.45:0/2696207583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd66659c640 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e64000 0x555c26e6dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd666d9d640 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e656b0 0x555c26e751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd65ffff640 1 -- 172.21.15.45:0/2696207583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e64000 msgr2=0x555c26e6dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd65ffff640 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e64000 0x555c26e6dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd65ffff640 1 -- 172.21.15.45:0/2696207583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e656b0 msgr2=0x555c26e751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd65ffff640 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e656b0 0x555c26e751e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd65ffff640 1 -- 172.21.15.45:0/2696207583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65800b960 con 0x555c26e64cf0 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd65ffff640 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 0x555c26e6e1a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd6500a16e0 tx=0x7fd65009a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcc18a0399d1450 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd64dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd64dffb640 1 -- 172.21.15.45:0/2696207583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6500b90c0 con 0x555c26e64cf0 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.592+0000 7fd64dffb640 1 -- 172.21.15.45:0/2696207583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd6500a1c10 con 0x555c26e64cf0 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd66fa28740 1 -- 172.21.15.45:0/2696207583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c26dbfe70 con 0x555c26e64cf0 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd64dffb640 1 -- 172.21.15.45:0/2696207583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd650098d70 con 0x555c26e64cf0 2024-01-31T05:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd66fa28740 10 client.15099 fetch_fsmap learned FSMap version 20 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd66fa28740 1 -- 172.21.15.45:0/2696207583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555c26dc5660 con 0x555c26e64cf0 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd64dffb640 1 -- 172.21.15.45:0/2696207583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c26dbfe70 con 0x555c26e64cf0 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd64dffb640 1 -- 172.21.15.45:0/2696207583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd65009abb0 con 0x555c26e64cf0 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd66fa28740 10 client.15099 fetch_fsmap finished waiting for FSMap version 20 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd66fa28740 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd66fa28740 20 client.15099 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd66fa28740 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555c26e7ad10 0x555c26e797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd66fa28740 4 client.15099 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd66fa28740 1 -- 172.21.15.45:0/2696207583 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c26dbfb90 con 0x555c26e7ad10 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.593+0000 7fd66659c640 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555c26e7ad10 0x555c26e797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.594+0000 7fd64dffb640 10 client.15099 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.594+0000 7fd66659c640 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555c26e7ad10 0x555c26e797d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=851316f63c46d927 server_cookie=80255383a488bd80 in_seq=0 out_seq=0 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.594+0000 7fd64dffb640 1 -- 172.21.15.45:0/2696207583 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c26dbfb90 con 0x555c26e7ad10 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.594+0000 7fd64dffb640 10 client.15099 handle_command_reply: tid=0 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd66fa28740 1 client.15099 shutdown 2024-01-31T05:22:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd66fa28740 1 -- 172.21.15.45:0/2696207583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 msgr2=0x555c26e6e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd66fa28740 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 0x555c26e6e1a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd6500a16e0 tx=0x7fd65009a040 comp rx=0 tx=0).stop 2024-01-31T05:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd66fa28740 1 -- 172.21.15.45:0/2696207583 shutdown_connections 2024-01-31T05:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd66fa28740 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555c26e7ad10 0x555c26e797d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd66fa28740 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 0x555c26e6e1a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd66fa28740 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e656b0 0x555c26e751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd66fa28740 1 --2- 172.21.15.45:0/2696207583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e64000 0x555c26e6dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd64dffb640 0 client.15099 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd66fa28740 1 -- 172.21.15.45:0/2696207583 >> 172.21.15.45:0/2696207583 conn(0x555c26e5c220 msgr2=0x555c26e59d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd66fa28740 1 -- 172.21.15.45:0/2696207583 shutdown_connections 2024-01-31T05:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd66fa28740 1 -- 172.21.15.45:0/2696207583 wait complete. 2024-01-31T05:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.595+0000 7fd66fa28740 20 client.15099 trim_cache size 0 max 16384 2024-01-31T05:22:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.610+0000 7fd66fa28740 1 Processor -- start 2024-01-31T05:22:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.610+0000 7fd66fa28740 1 -- start start 2024-01-31T05:22:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.610+0000 7fd66fa28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 0x555c26e6dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.610+0000 7fd66fa28740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e64cf0 0x555c26e751a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.610+0000 7fd66fa28740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e656b0 0x555c26e6e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.610+0000 7fd66fa28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26e5aba0 con 0x555c26e64000 2024-01-31T05:22:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.610+0000 7fd66fa28740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26e7c030 con 0x555c26e64cf0 2024-01-31T05:22:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.610+0000 7fd66fa28740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26e783d0 con 0x555c26e656b0 2024-01-31T05:22:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd66659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 0x555c26e6dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd66659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 0x555c26e6dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39350/0 (socket says 172.21.15.45:39350) 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd66659c640 1 -- 172.21.15.45:0/1983791405 learned_addr learned my addr 172.21.15.45:0/1983791405 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd66659c640 1 -- 172.21.15.45:0/1983791405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e656b0 msgr2=0x555c26e6e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd66659c640 1 --2- 172.21.15.45:0/1983791405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e656b0 0x555c26e6e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd66659c640 1 -- 172.21.15.45:0/1983791405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e64cf0 msgr2=0x555c26e751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd65ffff640 1 --2- 172.21.15.45:0/1983791405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e64cf0 0x555c26e751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd66659c640 1 --2- 172.21.15.45:0/1983791405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e64cf0 0x555c26e751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd66659c640 1 -- 172.21.15.45:0/1983791405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c26d62360 con 0x555c26e64000 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd66659c640 1 --2- 172.21.15.45:0/1983791405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 0x555c26e6dc30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd658006010 tx=0x7fd658011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d21354069edc65 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd64ffff640 1 -- 172.21.15.45:0/1983791405 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65802d070 con 0x555c26e64000 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd64ffff640 1 -- 172.21.15.45:0/1983791405 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd658021040 con 0x555c26e64000 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.611+0000 7fd64ffff640 1 -- 172.21.15.45:0/1983791405 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6580116a0 con 0x555c26e64000 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 -- 172.21.15.45:0/1983791405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 msgr2=0x555c26e6dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 --2- 172.21.15.45:0/1983791405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 0x555c26e6dc30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd658006010 tx=0x7fd658011cd0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 -- 172.21.15.45:0/1983791405 shutdown_connections 2024-01-31T05:22:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 --2- 172.21.15.45:0/1983791405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e656b0 0x555c26e6e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 --2- 172.21.15.45:0/1983791405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e64cf0 0x555c26e751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 --2- 172.21.15.45:0/1983791405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64000 0x555c26e6dc30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 -- 172.21.15.45:0/1983791405 >> 172.21.15.45:0/1983791405 conn(0x555c26e5c220 msgr2=0x555c26e68710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 -- 172.21.15.45:0/1983791405 shutdown_connections 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 -- 172.21.15.45:0/1983791405 wait complete. 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 -- start start 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 Processor -- start 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e64000 0x555c26e72e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 0x555c26e73340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e656b0 0x555c26e74300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26e74840 con 0x555c26e64cf0 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26e749b0 con 0x555c26e656b0 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26e74b70 con 0x555c26e64000 2024-01-31T05:22:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd66fa28740 10 client.0 _ll_register_callbacks cb 0x7fd65800b4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd65ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 0x555c26e73340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd65ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 0x555c26e73340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39366/0 (socket says 172.21.15.45:39366) 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd65ffff640 1 -- 172.21.15.45:0/2644737807 learned_addr learned my addr 172.21.15.45:0/2644737807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.612+0000 7fd666d9d640 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e656b0 0x555c26e74300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd65ffff640 1 -- 172.21.15.45:0/2644737807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e64000 msgr2=0x555c26e72e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd65ffff640 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e64000 0x555c26e72e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd65ffff640 1 -- 172.21.15.45:0/2644737807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e656b0 msgr2=0x555c26e74300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd65ffff640 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e656b0 0x555c26e74300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd65ffff640 1 -- 172.21.15.45:0/2644737807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c26d89650 con 0x555c26e64cf0 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd666d9d640 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e656b0 0x555c26e74300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd65ffff640 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 0x555c26e73340 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd650099580 tx=0x7fd6500844a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7776bf034c13c44c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd64dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd64dffb640 1 -- 172.21.15.45:0/2644737807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6500b90b0 con 0x555c26e64cf0 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd64dffb640 1 -- 172.21.15.45:0/2644737807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd650084180 con 0x555c26e64cf0 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd66fa28740 1 -- 172.21.15.45:0/2644737807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c26e7b6e0 con 0x555c26e64cf0 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd64dffb640 1 -- 172.21.15.45:0/2644737807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd650099650 con 0x555c26e64cf0 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd66fa28740 10 client.15105 fetch_fsmap learned FSMap version 20 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd66fa28740 1 -- 172.21.15.45:0/2644737807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555c26d87eb0 con 0x555c26e64cf0 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd64dffb640 1 -- 172.21.15.45:0/2644737807 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c26e7b6e0 con 0x555c26e64cf0 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd64dffb640 1 -- 172.21.15.45:0/2644737807 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd650098870 con 0x555c26e64cf0 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd66fa28740 10 client.15105 fetch_fsmap finished waiting for FSMap version 20 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd66fa28740 10 client.15105 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd66fa28740 20 client.15105 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd66fa28740 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555c26ed6040 0x555c26ed2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd66fa28740 4 client.15105 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.613+0000 7fd66fa28740 1 -- 172.21.15.45:0/2644737807 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555c26e7aa20 con 0x555c26ed6040 2024-01-31T05:22:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66659c640 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555c26ed6040 0x555c26ed2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66659c640 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555c26ed6040 0x555c26ed2480 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4462d9b05a245e9 server_cookie=ea176276a228dc02 in_seq=0 out_seq=0 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd64dffb640 10 client.15105 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd64dffb640 1 -- 172.21.15.45:0/2644737807 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x555c26e7aa20 con 0x555c26ed6040 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd64dffb640 10 client.15105 handle_command_reply: tid=0 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66fa28740 1 client.15105 shutdown 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66fa28740 1 -- 172.21.15.45:0/2644737807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 msgr2=0x555c26e73340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66fa28740 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 0x555c26e73340 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd650099580 tx=0x7fd6500844a0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66fa28740 1 -- 172.21.15.45:0/2644737807 shutdown_connections 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66fa28740 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555c26ed6040 0x555c26ed2480 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66fa28740 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c26e64cf0 0x555c26e73340 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66fa28740 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c26e656b0 0x555c26e74300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd64dffb640 0 client.15105 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66fa28740 1 --2- 172.21.15.45:0/2644737807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c26e64000 0x555c26e72e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66fa28740 1 -- 172.21.15.45:0/2644737807 >> 172.21.15.45:0/2644737807 conn(0x555c26e5c220 msgr2=0x555c26e6dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66fa28740 1 -- 172.21.15.45:0/2644737807 shutdown_connections 2024-01-31T05:22:04.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66fa28740 1 -- 172.21.15.45:0/2644737807 wait complete. 2024-01-31T05:22:04.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.614+0000 7fd66fa28740 20 client.15105 trim_cache size 0 max 16384 2024-01-31T05:22:04.622 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-01-31T05:22:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "3a1f4122-bff7-11ee-95b3-87774f69a715", 2024-01-31T05:22:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-01-31T05:22:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24503, 2024-01-31T05:22:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-01-31T05:22:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-01-31T05:22:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-01-31T05:22:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 83.918730240000002, 2024-01-31T05:22:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 20, 2024-01-31T05:22:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 88, 2024-01-31T05:22:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 86, 2024-01-31T05:22:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 89.618866014000005 2024-01-31T05:22:04.623 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-01-31T05:22:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.615+0000 7fd66cb89640 1 -- 172.21.15.45:0/3705643546 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd6540a1360 msgr2=0x7fd6540a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.615+0000 7fd66cb89640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd6540a1360 0x7fd6540a3820 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd660005e30 tx=0x7fd660032690 comp rx=0 tx=0).stop 2024-01-31T05:22:04.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.615+0000 7fd66cb89640 1 -- 172.21.15.45:0/3705643546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd668100df0 msgr2=0x7fd6681960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:04.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.615+0000 7fd66cb89640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd668100df0 0x7fd6681960e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd65000d950 tx=0x7fd65000de10 comp rx=0 tx=0).stop 2024-01-31T05:22:04.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.615+0000 7fd66cb89640 1 -- 172.21.15.45:0/3705643546 shutdown_connections 2024-01-31T05:22:04.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.615+0000 7fd66cb89640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd6540a1360 0x7fd6540a3820 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.615+0000 7fd66cb89640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd668100df0 0x7fd6681960e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.615+0000 7fd66cb89640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6681017b0 0x7fd66819d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.615+0000 7fd66cb89640 1 --2- 172.21.15.45:0/3705643546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6680ffc00 0x7fd668195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:04.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.615+0000 7fd66cb89640 1 -- 172.21.15.45:0/3705643546 >> 172.21.15.45:0/3705643546 conn(0x7fd6680fb190 msgr2=0x7fd668104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:04.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.615+0000 7fd66cb89640 1 -- 172.21.15.45:0/3705643546 shutdown_connections 2024-01-31T05:22:04.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:04.615+0000 7fd66cb89640 1 -- 172.21.15.45:0/3705643546 wait complete. 2024-01-31T05:22:04.626 INFO:tasks.cephadm:Setup complete, yielding 2024-01-31T05:22:04.626 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-01-31T05:22:04.635 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-01-31T05:22:04.635 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- bash -c 'ceph orch status' 2024-01-31T05:22:04.877 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:22:04.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:04 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3366854746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:04.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:04 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/577441528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:04.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:04 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3366854746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:04.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:04 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/577441528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:05.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:04 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3366854746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:05.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:04 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/577441528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:05.936 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:05 smithi045 ceph-mon[31585]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:22:05.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:05 smithi156 ceph-mon[35720]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:22:06.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:05 smithi186 ceph-mon[35659]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:22:06.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.434+0000 7f6aacc32700 1 -- 172.21.15.45:0/2997665984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa8102330 msgr2=0x7f6aa8104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:06.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.434+0000 7f6aacc32700 1 --2- 172.21.15.45:0/2997665984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa8102330 0x7f6aa8104720 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009990 tx=0x7f6a98009ca0 comp rx=0 tx=0).stop 2024-01-31T05:22:06.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.434+0000 7f6aacc32700 1 -- 172.21.15.45:0/2997665984 shutdown_connections 2024-01-31T05:22:06.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.434+0000 7f6aacc32700 1 --2- 172.21.15.45:0/2997665984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa8107590 0x7f6aa8109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:06.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.434+0000 7f6aacc32700 1 --2- 172.21.15.45:0/2997665984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa8102330 0x7f6aa8104720 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:06.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.434+0000 7f6aacc32700 1 --2- 172.21.15.45:0/2997665984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6aa8104c60 0x7f6aa8107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:06.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.434+0000 7f6aacc32700 1 -- 172.21.15.45:0/2997665984 >> 172.21.15.45:0/2997665984 conn(0x7f6aa80fbca0 msgr2=0x7f6aa80fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.434+0000 7f6aacc32700 1 -- 172.21.15.45:0/2997665984 shutdown_connections 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.434+0000 7f6aacc32700 1 -- 172.21.15.45:0/2997665984 wait complete. 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.434+0000 7f6aacc32700 1 Processor -- start 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.434+0000 7f6aacc32700 1 -- start start 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aacc32700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa8102330 0x7f6aa8197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aacc32700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa8104c60 0x7f6aa8197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aacc32700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6aa8107590 0x7f6aa819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aacc32700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa819f2c0 con 0x7f6aa8104c60 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aacc32700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa819f400 con 0x7f6aa8102330 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aacc32700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa819f590 con 0x7f6aa8107590 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aa5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa8104c60 0x7f6aa8197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aa5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa8104c60 0x7f6aa8197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39384/0 (socket says 172.21.15.45:39384) 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aa5d9b700 1 -- 172.21.15.45:0/819931052 learned_addr learned my addr 172.21.15.45:0/819931052 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aa6d9d700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6aa8107590 0x7f6aa819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aa659c700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa8102330 0x7f6aa8197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aa5d9b700 1 -- 172.21.15.45:0/819931052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6aa8107590 msgr2=0x7f6aa819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aa5d9b700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6aa8107590 0x7f6aa819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aa5d9b700 1 -- 172.21.15.45:0/819931052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa8102330 msgr2=0x7f6aa8197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aa5d9b700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa8102330 0x7f6aa8197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aa5d9b700 1 -- 172.21.15.45:0/819931052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a980095f0 con 0x7f6aa8104c60 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aa659c700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa8102330 0x7f6aa8197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aa5d9b700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa8104c60 0x7f6aa8197c70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00b560 tx=0x7f6a9c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6a977fe700 1 -- 172.21.15.45:0/819931052 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c011830 con 0x7f6aa8104c60 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6a977fe700 1 -- 172.21.15.45:0/819931052 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a9c011e50 con 0x7f6aa8104c60 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6aacc32700 1 -- 172.21.15.45:0/819931052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa819f870 con 0x7f6aa8104c60 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.435+0000 7f6a977fe700 1 -- 172.21.15.45:0/819931052 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00f2e0 con 0x7f6aa8104c60 2024-01-31T05:22:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.436+0000 7f6aacc32700 1 -- 172.21.15.45:0/819931052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa819fd70 con 0x7f6aa8104c60 2024-01-31T05:22:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.437+0000 7f6aacc32700 1 -- 172.21.15.45:0/819931052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa80fd820 con 0x7f6aa8104c60 2024-01-31T05:22:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.438+0000 7f6a977fe700 1 -- 172.21.15.45:0/819931052 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f6a9c010580 con 0x7f6aa8104c60 2024-01-31T05:22:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.438+0000 7f6a977fe700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f6a900a1370 0x7f6a900a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:06.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.438+0000 7f6aa659c700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f6a900a1370 0x7f6a900a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:06.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.438+0000 7f6a977fe700 1 -- 172.21.15.45:0/819931052 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6a9c0cc5a0 con 0x7f6aa8104c60 2024-01-31T05:22:06.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.439+0000 7f6aa659c700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f6a900a1370 0x7f6a900a3830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800b5c0 tx=0x7f6a98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:06.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.443+0000 7f6a977fe700 1 -- 172.21.15.45:0/819931052 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f6a9c095a50 con 0x7f6aa8104c60 2024-01-31T05:22:06.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.658+0000 7f6aacc32700 1 -- 172.21.15.45:0/819931052 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6aa802d4a0 con 0x7f6a900a1370 2024-01-31T05:22:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.660+0000 7f6a977fe700 1 -- 172.21.15.45:0/819931052 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6aa802d4a0 con 0x7f6a900a1370 2024-01-31T05:22:06.660 INFO:teuthology.orchestra.run.smithi045.stdout:Backend: cephadm 2024-01-31T05:22:06.660 INFO:teuthology.orchestra.run.smithi045.stdout:Available: Yes 2024-01-31T05:22:06.660 INFO:teuthology.orchestra.run.smithi045.stdout:Paused: No 2024-01-31T05:22:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.661+0000 7f6aacc32700 1 -- 172.21.15.45:0/819931052 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f6a900a1370 msgr2=0x7f6a900a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.661+0000 7f6aacc32700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f6a900a1370 0x7f6a900a3830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800b5c0 tx=0x7f6a98005fb0 comp rx=0 tx=0).stop 2024-01-31T05:22:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.661+0000 7f6aacc32700 1 -- 172.21.15.45:0/819931052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa8104c60 msgr2=0x7f6aa8197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.662+0000 7f6aacc32700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa8104c60 0x7f6aa8197c70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00b560 tx=0x7f6a9c00b870 comp rx=0 tx=0).stop 2024-01-31T05:22:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.662+0000 7f6aacc32700 1 -- 172.21.15.45:0/819931052 shutdown_connections 2024-01-31T05:22:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.662+0000 7f6aacc32700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6aa8107590 0x7f6aa819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.662+0000 7f6aacc32700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa8102330 0x7f6aa8197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.662+0000 7f6aacc32700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f6a900a1370 0x7f6a900a3830 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:06.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.662+0000 7f6aacc32700 1 --2- 172.21.15.45:0/819931052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa8104c60 0x7f6aa8197c70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:06.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.662+0000 7f6aacc32700 1 -- 172.21.15.45:0/819931052 >> 172.21.15.45:0/819931052 conn(0x7f6aa80fbca0 msgr2=0x7f6aa8102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:06.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.662+0000 7f6aacc32700 1 -- 172.21.15.45:0/819931052 shutdown_connections 2024-01-31T05:22:06.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:06.662+0000 7f6aacc32700 1 -- 172.21.15.45:0/819931052 wait complete. 2024-01-31T05:22:07.272 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- bash -c 'ceph orch ps' 2024-01-31T05:22:07.521 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:22:07.827 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:07 smithi045 ceph-mon[31585]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:22:07.827 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:07 smithi045 ceph-mon[31585]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:07.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:07 smithi156 ceph-mon[35720]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:22:07.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:07 smithi156 ceph-mon[35720]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:08.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:07 smithi186 ceph-mon[35659]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:22:08.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:07 smithi186 ceph-mon[35659]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.232+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/981629105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24100780 msgr2=0x7fdd24100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.232+0000 7fdd2ac8b700 1 --2- 172.21.15.45:0/981629105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24100780 0x7fdd24100ba0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd180099c0 tx=0x7fdd18009cd0 comp rx=0 tx=0).stop 2024-01-31T05:22:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.232+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/981629105 shutdown_connections 2024-01-31T05:22:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.232+0000 7fdd2ac8b700 1 --2- 172.21.15.45:0/981629105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd24102320 0x7fdd24108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.232+0000 7fdd2ac8b700 1 --2- 172.21.15.45:0/981629105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24100780 0x7fdd24100ba0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.232+0000 7fdd2ac8b700 1 --2- 172.21.15.45:0/981629105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd24101980 0x7fdd24101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.232+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/981629105 >> 172.21.15.45:0/981629105 conn(0x7fdd240fbd40 msgr2=0x7fdd240fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.232+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/981629105 shutdown_connections 2024-01-31T05:22:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.232+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/981629105 wait complete. 2024-01-31T05:22:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.233+0000 7fdd2ac8b700 1 Processor -- start 2024-01-31T05:22:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.233+0000 7fdd2ac8b700 1 -- start start 2024-01-31T05:22:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.233+0000 7fdd2ac8b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd24100780 0x7fdd2410b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.233+0000 7fdd2ac8b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd24101980 0x7fdd241096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.233+0000 7fdd2ac8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24102320 0x7fdd24109c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.233+0000 7fdd2ac8b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2410a3a0 con 0x7fdd24102320 2024-01-31T05:22:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.233+0000 7fdd2ac8b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2410a510 con 0x7fdd24100780 2024-01-31T05:22:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.233+0000 7fdd2ac8b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd241a1780 con 0x7fdd24101980 2024-01-31T05:22:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.233+0000 7fdd29228700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24102320 0x7fdd24109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.233+0000 7fdd29228700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24102320 0x7fdd24109c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48906/0 (socket says 172.21.15.45:48906) 2024-01-31T05:22:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.233+0000 7fdd29228700 1 -- 172.21.15.45:0/2918643327 learned_addr learned my addr 172.21.15.45:0/2918643327 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.233+0000 7fdd28a27700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd24100780 0x7fdd2410b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd23fff700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd24101980 0x7fdd241096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd29228700 1 -- 172.21.15.45:0/2918643327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd24101980 msgr2=0x7fdd241096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd29228700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd24101980 0x7fdd241096e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd29228700 1 -- 172.21.15.45:0/2918643327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd24100780 msgr2=0x7fdd2410b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd29228700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd24100780 0x7fdd2410b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd29228700 1 -- 172.21.15.45:0/2918643327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd180095f0 con 0x7fdd24102320 2024-01-31T05:22:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd23fff700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd24101980 0x7fdd241096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd29228700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24102320 0x7fdd24109c20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c007ee0 tx=0x7fdd1c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd21ffb700 1 -- 172.21.15.45:0/2918643327 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c017070 con 0x7fdd24102320 2024-01-31T05:22:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd21ffb700 1 -- 172.21.15.45:0/2918643327 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd1c00f040 con 0x7fdd24102320 2024-01-31T05:22:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/2918643327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd241a1a60 con 0x7fdd24102320 2024-01-31T05:22:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd21ffb700 1 -- 172.21.15.45:0/2918643327 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c005740 con 0x7fdd24102320 2024-01-31T05:22:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.234+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/2918643327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd241a1f10 con 0x7fdd24102320 2024-01-31T05:22:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.235+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/2918643327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd24107bb0 con 0x7fdd24102320 2024-01-31T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.236+0000 7fdd21ffb700 1 -- 172.21.15.45:0/2918643327 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fdd1c00dc60 con 0x7fdd24102320 2024-01-31T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.237+0000 7fdd21ffb700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdd140a1370 0x7fdd140a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.237+0000 7fdd28a27700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdd140a1370 0x7fdd140a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.237+0000 7fdd21ffb700 1 -- 172.21.15.45:0/2918643327 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdd1c0cc310 con 0x7fdd24102320 2024-01-31T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.237+0000 7fdd28a27700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdd140a1370 0x7fdd140a3830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdd18003820 tx=0x7fdd18005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.241+0000 7fdd21ffb700 1 -- 172.21.15.45:0/2918643327 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fdd1c0957c0 con 0x7fdd24102320 2024-01-31T05:22:09.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.452+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/2918643327 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd2402d4a0 con 0x7fdd140a1370 2024-01-31T05:22:09.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.466+0000 7fdd21ffb700 1 -- 172.21.15.45:0/2918643327 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4154 (secure 0 0 0) 0x7fdd2402d4a0 con 0x7fdd140a1370 2024-01-31T05:22:09.465 INFO:teuthology.orchestra.run.smithi045.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-01-31T05:22:09.465 INFO:teuthology.orchestra.run.smithi045.stdout:mds.a smithi045 running (97s) 17s ago 97s 12.9M - 19.0.0-885-g661946e7 0b5dc47a38d7 63269242125a 2024-01-31T05:22:09.465 INFO:teuthology.orchestra.run.smithi045.stdout:mds.b smithi156 running (94s) 28s ago 94s 27.9M - 19.0.0-885-g661946e7 0b5dc47a38d7 d817dbe780f4 2024-01-31T05:22:09.466 INFO:teuthology.orchestra.run.smithi045.stdout:mds.c smithi186 running (108s) 28s ago 108s 18.4M - 19.0.0-885-g661946e7 0b5dc47a38d7 3505dd956348 2024-01-31T05:22:09.466 INFO:teuthology.orchestra.run.smithi045.stdout:mds.d smithi045 running (116s) 17s ago 116s 15.2M - 19.0.0-885-g661946e7 0b5dc47a38d7 0500c71b12ff 2024-01-31T05:22:09.466 INFO:teuthology.orchestra.run.smithi045.stdout:mds.e smithi156 running (113s) 28s ago 113s 21.1M - 19.0.0-885-g661946e7 0b5dc47a38d7 d7a87260df21 2024-01-31T05:22:09.466 INFO:teuthology.orchestra.run.smithi045.stdout:mds.f smithi186 running (111s) 28s ago 110s 16.4M - 19.0.0-885-g661946e7 0b5dc47a38d7 c7d29b680cd7 2024-01-31T05:22:09.466 INFO:teuthology.orchestra.run.smithi045.stdout:mds.g smithi045 running (100s) 17s ago 99s 12.7M - 19.0.0-885-g661946e7 0b5dc47a38d7 4dc919400729 2024-01-31T05:22:09.466 INFO:teuthology.orchestra.run.smithi045.stdout:mds.h smithi156 running (105s) 28s ago 105s 20.2M - 19.0.0-885-g661946e7 0b5dc47a38d7 cdc5ea8ec148 2024-01-31T05:22:09.466 INFO:teuthology.orchestra.run.smithi045.stdout:mds.i smithi186 running (103s) 28s ago 102s 16.2M - 19.0.0-885-g661946e7 0b5dc47a38d7 d1b57c01fe3a 2024-01-31T05:22:09.466 INFO:teuthology.orchestra.run.smithi045.stdout:mds.j smithi045 running (119s) 17s ago 119s 13.9M - 19.0.0-885-g661946e7 0b5dc47a38d7 aa11c2272224 2024-01-31T05:22:09.466 INFO:teuthology.orchestra.run.smithi045.stdout:mds.k smithi156 running (2m) 28s ago 2m 15.2M - 19.0.0-885-g661946e7 0b5dc47a38d7 6ee9d87f4ebf 2024-01-31T05:22:09.466 INFO:teuthology.orchestra.run.smithi045.stdout:mds.l smithi186 running (2m) 28s ago 2m 16.3M - 19.0.0-885-g661946e7 0b5dc47a38d7 ab8db6740b0a 2024-01-31T05:22:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.x smithi045 *:9283,8765 running (9m) 17s ago 9m 504M - 19.0.0-885-g661946e7 0b5dc47a38d7 c66a60e0cf9e 2024-01-31T05:22:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.y smithi156 *:8443,8765 running (7m) 28s ago 7m 442M - 19.0.0-885-g661946e7 0b5dc47a38d7 d98f0bf733a7 2024-01-31T05:22:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.z smithi186 *:8443,8765 running (7m) 28s ago 7m 442M - 19.0.0-885-g661946e7 0b5dc47a38d7 371604f44df0 2024-01-31T05:22:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a smithi045 running (9m) 17s ago 9m 53.9M 2048M 19.0.0-885-g661946e7 0b5dc47a38d7 6db52a688cd2 2024-01-31T05:22:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b smithi156 running (7m) 28s ago 7m 39.6M 2048M 19.0.0-885-g661946e7 0b5dc47a38d7 5ab2d0d9b769 2024-01-31T05:22:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c smithi186 running (7m) 28s ago 7m 40.1M 2048M 19.0.0-885-g661946e7 0b5dc47a38d7 fad13bf4d04c 2024-01-31T05:22:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 smithi045 running (6m) 17s ago 6m 55.4M 4273M 19.0.0-885-g661946e7 0b5dc47a38d7 7e46a11e9297 2024-01-31T05:22:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 smithi156 running (6m) 28s ago 6m 49.3M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 8c803c56beee 2024-01-31T05:22:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 smithi186 running (6m) 28s ago 6m 49.9M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 cdec22b13c6d 2024-01-31T05:22:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 smithi045 running (5m) 17s ago 5m 48.4M 4273M 19.0.0-885-g661946e7 0b5dc47a38d7 b9dc97c7ce0e 2024-01-31T05:22:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 smithi156 running (5m) 28s ago 5m 51.7M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 0b8d1860f711 2024-01-31T05:22:09.468 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 smithi186 running (4m) 28s ago 4m 48.7M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 f8fb18af320c 2024-01-31T05:22:09.468 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 smithi045 running (4m) 17s ago 4m 50.9M 4273M 19.0.0-885-g661946e7 0b5dc47a38d7 c5b8a08fde1f 2024-01-31T05:22:09.468 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 smithi156 running (4m) 28s ago 4m 46.5M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 1d56671cd591 2024-01-31T05:22:09.468 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 smithi186 running (3m) 28s ago 3m 53.3M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 ec75bc9a6720 2024-01-31T05:22:09.468 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 smithi045 running (3m) 17s ago 3m 49.5M 4273M 19.0.0-885-g661946e7 0b5dc47a38d7 78e372d5937c 2024-01-31T05:22:09.468 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 smithi156 running (2m) 28s ago 2m 53.5M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 9ec11346a9b6 2024-01-31T05:22:09.468 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 smithi186 running (2m) 28s ago 2m 45.5M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 2e5486c92ec8 2024-01-31T05:22:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.468+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/2918643327 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdd140a1370 msgr2=0x7fdd140a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.468+0000 7fdd2ac8b700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdd140a1370 0x7fdd140a3830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdd18003820 tx=0x7fdd18005ad0 comp rx=0 tx=0).stop 2024-01-31T05:22:09.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.468+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/2918643327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24102320 msgr2=0x7fdd24109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:09.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.468+0000 7fdd2ac8b700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24102320 0x7fdd24109c20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c007ee0 tx=0x7fdd1c00d3b0 comp rx=0 tx=0).stop 2024-01-31T05:22:09.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.468+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/2918643327 shutdown_connections 2024-01-31T05:22:09.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.468+0000 7fdd2ac8b700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fdd140a1370 0x7fdd140a3830 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:09.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.468+0000 7fdd2ac8b700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24102320 0x7fdd24109c20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:09.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.468+0000 7fdd2ac8b700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd24100780 0x7fdd2410b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:09.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.468+0000 7fdd2ac8b700 1 --2- 172.21.15.45:0/2918643327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd24101980 0x7fdd241096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:09.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.468+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/2918643327 >> 172.21.15.45:0/2918643327 conn(0x7fdd240fbd40 msgr2=0x7fdd24104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:09.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.468+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/2918643327 shutdown_connections 2024-01-31T05:22:09.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:09.468+0000 7fdd2ac8b700 1 -- 172.21.15.45:0/2918643327 wait complete. 2024-01-31T05:22:09.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:09 smithi045 ceph-mon[31585]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:22:09.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:09 smithi156 ceph-mon[35720]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:22:10.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:09 smithi186 ceph-mon[35659]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:22:10.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- bash -c 'ceph orch ls' 2024-01-31T05:22:10.364 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:22:10.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:10 smithi045 ceph-mon[31585]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:10 smithi156 ceph-mon[35720]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:11.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:10 smithi186 ceph-mon[35659]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:11 smithi156 ceph-mon[35720]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:22:12.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:11 smithi045 ceph-mon[31585]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:22:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.007+0000 7ff3bb516700 1 -- 172.21.15.45:0/3342993449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b40ffaf0 msgr2=0x7ff3b40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.007+0000 7ff3bb516700 1 --2- 172.21.15.45:0/3342993449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b40ffaf0 0x7ff3b40fff10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009990 tx=0x7ff3ac009ca0 comp rx=0 tx=0).stop 2024-01-31T05:22:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.007+0000 7ff3bb516700 1 -- 172.21.15.45:0/3342993449 shutdown_connections 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.007+0000 7ff3bb516700 1 --2- 172.21.15.45:0/3342993449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b4103510 0x7ff3b4109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.007+0000 7ff3bb516700 1 --2- 172.21.15.45:0/3342993449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b40ffaf0 0x7ff3b40fff10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.007+0000 7ff3bb516700 1 --2- 172.21.15.45:0/3342993449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b4102b70 0x7ff3b4102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.007+0000 7ff3bb516700 1 -- 172.21.15.45:0/3342993449 >> 172.21.15.45:0/3342993449 conn(0x7ff3b40fb450 msgr2=0x7ff3b40fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.007+0000 7ff3bb516700 1 -- 172.21.15.45:0/3342993449 shutdown_connections 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.007+0000 7ff3bb516700 1 -- 172.21.15.45:0/3342993449 wait complete. 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3bb516700 1 Processor -- start 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3bb516700 1 -- start start 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3bb516700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b40ffaf0 0x7ff3b4197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3bb516700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b4102b70 0x7ff3b4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3bb516700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b4103510 0x7ff3b419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3bb516700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b419f1d0 con 0x7ff3b4103510 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3bb516700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b419f310 con 0x7ff3b40ffaf0 2024-01-31T05:22:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3bb516700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b419f4a0 con 0x7ff3b4102b70 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3b9ab3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b4103510 0x7ff3b419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3b9ab3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b4103510 0x7ff3b419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48926/0 (socket says 172.21.15.45:48926) 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3b9ab3700 1 -- 172.21.15.45:0/3995853784 learned_addr learned my addr 172.21.15.45:0/3995853784 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3b8ab1700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b4102b70 0x7ff3b4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3b92b2700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b40ffaf0 0x7ff3b4197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3b9ab3700 1 -- 172.21.15.45:0/3995853784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b4102b70 msgr2=0x7ff3b4197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3b9ab3700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b4102b70 0x7ff3b4197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3b9ab3700 1 -- 172.21.15.45:0/3995853784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b40ffaf0 msgr2=0x7ff3b4197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3b9ab3700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b40ffaf0 0x7ff3b4197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3b9ab3700 1 -- 172.21.15.45:0/3995853784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac0095f0 con 0x7ff3b4103510 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3b92b2700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b40ffaf0 0x7ff3b4197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.008+0000 7ff3b8ab1700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b4102b70 0x7ff3b4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.009+0000 7ff3b9ab3700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b4103510 0x7ff3b419ec00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4007da0 tx=0x7ff3a400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.009+0000 7ff3b27fc700 1 -- 172.21.15.45:0/3995853784 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400ad00 con 0x7ff3b4103510 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.009+0000 7ff3b27fc700 1 -- 172.21.15.45:0/3995853784 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3a400ae60 con 0x7ff3b4103510 2024-01-31T05:22:12.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.009+0000 7ff3bb516700 1 -- 172.21.15.45:0/3995853784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b419f780 con 0x7ff3b4103510 2024-01-31T05:22:12.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.009+0000 7ff3b27fc700 1 -- 172.21.15.45:0/3995853784 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4011780 con 0x7ff3b4103510 2024-01-31T05:22:12.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.009+0000 7ff3bb516700 1 -- 172.21.15.45:0/3995853784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b419fc30 con 0x7ff3b4103510 2024-01-31T05:22:12.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.010+0000 7ff3bb516700 1 -- 172.21.15.45:0/3995853784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b4108d30 con 0x7ff3b4103510 2024-01-31T05:22:12.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.011+0000 7ff3b27fc700 1 -- 172.21.15.45:0/3995853784 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7ff3a4012a70 con 0x7ff3b4103510 2024-01-31T05:22:12.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.012+0000 7ff3b27fc700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ff39c0a1370 0x7ff39c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:12.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.012+0000 7ff3b92b2700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ff39c0a1370 0x7ff39c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:12.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.012+0000 7ff3b27fc700 1 -- 172.21.15.45:0/3995853784 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff3a40cd2b0 con 0x7ff3b4103510 2024-01-31T05:22:12.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.012+0000 7ff3b92b2700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ff39c0a1370 0x7ff39c0a3830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac000c00 tx=0x7ff3ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:12.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.015+0000 7ff3b27fc700 1 -- 172.21.15.45:0/3995853784 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7ff3a4096700 con 0x7ff3b4103510 2024-01-31T05:22:12.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:11 smithi186 ceph-mon[35659]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:22:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.224+0000 7ff3bb516700 1 -- 172.21.15.45:0/3995853784 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3b402d540 con 0x7ff39c0a1370 2024-01-31T05:22:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.230+0000 7ff3b27fc700 1 -- 172.21.15.45:0/3995853784 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7ff3b402d540 con 0x7ff39c0a1370 2024-01-31T05:22:12.229 INFO:teuthology.orchestra.run.smithi045.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-01-31T05:22:12.229 INFO:teuthology.orchestra.run.smithi045.stdout:mds.all 12/12 31s ago 2m smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-01-31T05:22:12.229 INFO:teuthology.orchestra.run.smithi045.stdout:mgr 3/3 31s ago 7m smithi045=x;smithi156=y;smithi186=z;count:3 2024-01-31T05:22:12.229 INFO:teuthology.orchestra.run.smithi045.stdout:mon 3/3 31s ago 7m smithi045:172.21.15.45=a;smithi156:172.21.15.156=b;smithi186:172.21.15.186=c;count:3 2024-01-31T05:22:12.229 INFO:teuthology.orchestra.run.smithi045.stdout:osd 12 31s ago - 2024-01-31T05:22:12.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.232+0000 7ff3bb516700 1 -- 172.21.15.45:0/3995853784 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ff39c0a1370 msgr2=0x7ff39c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:12.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.232+0000 7ff3bb516700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ff39c0a1370 0x7ff39c0a3830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac000c00 tx=0x7ff3ac005fb0 comp rx=0 tx=0).stop 2024-01-31T05:22:12.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.232+0000 7ff3bb516700 1 -- 172.21.15.45:0/3995853784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b4103510 msgr2=0x7ff3b419ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:12.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.232+0000 7ff3bb516700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b4103510 0x7ff3b419ec00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4007da0 tx=0x7ff3a400a340 comp rx=0 tx=0).stop 2024-01-31T05:22:12.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.232+0000 7ff3bb516700 1 -- 172.21.15.45:0/3995853784 shutdown_connections 2024-01-31T05:22:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.232+0000 7ff3bb516700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ff39c0a1370 0x7ff39c0a3830 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.232+0000 7ff3bb516700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b4103510 0x7ff3b419ec00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.232+0000 7ff3bb516700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b40ffaf0 0x7ff3b4197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.232+0000 7ff3bb516700 1 --2- 172.21.15.45:0/3995853784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b4102b70 0x7ff3b4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.232+0000 7ff3bb516700 1 -- 172.21.15.45:0/3995853784 >> 172.21.15.45:0/3995853784 conn(0x7ff3b40fb450 msgr2=0x7ff3b4105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.232+0000 7ff3bb516700 1 -- 172.21.15.45:0/3995853784 shutdown_connections 2024-01-31T05:22:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:12.232+0000 7ff3bb516700 1 -- 172.21.15.45:0/3995853784 wait complete. 2024-01-31T05:22:12.893 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- bash -c 'ceph orch host ls' 2024-01-31T05:22:13.136 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:22:13.594 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:13 smithi045 ceph-mon[31585]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:13.594 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:13 smithi045 ceph-mon[31585]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:22:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:13 smithi186 ceph-mon[35659]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:13 smithi186 ceph-mon[35659]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:22:13.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:13 smithi156 ceph-mon[35720]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:13.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:13 smithi156 ceph-mon[35720]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:22:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.695+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3255290267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d64101380 msgr2=0x7f3d64108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.695+0000 7f3d6c7be700 1 --2- 172.21.15.45:0/3255290267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d64101380 0x7f3d64108c20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00b0a0 tx=0x7f3d5c00b3b0 comp rx=0 tx=0).stop 2024-01-31T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3255290267 shutdown_connections 2024-01-31T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 --2- 172.21.15.45:0/3255290267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d64101380 0x7f3d64108c20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 --2- 172.21.15.45:0/3255290267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d64101d60 0x7f3d64102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 --2- 172.21.15.45:0/3255290267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d641009c0 0x7f3d64100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3255290267 >> 172.21.15.45:0/3255290267 conn(0x7f3d640fb470 msgr2=0x7f3d640fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3255290267 shutdown_connections 2024-01-31T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3255290267 wait complete. 2024-01-31T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 Processor -- start 2024-01-31T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 -- start start 2024-01-31T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d641009c0 0x7f3d64100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d64101380 0x7f3d6419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d64101d60 0x7f3d6419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6419f220 con 0x7f3d64101d60 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6419f390 con 0x7f3d641009c0 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.696+0000 7f3d6c7be700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6419f550 con 0x7f3d64101380 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d6ad5b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d64101d60 0x7f3d6419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d6a55a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d641009c0 0x7f3d64100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d6ad5b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d64101d60 0x7f3d6419ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48950/0 (socket says 172.21.15.45:48950) 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d6ad5b700 1 -- 172.21.15.45:0/3748762319 learned_addr learned my addr 172.21.15.45:0/3748762319 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d69d59700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d64101380 0x7f3d6419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d6ad5b700 1 -- 172.21.15.45:0/3748762319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d64101380 msgr2=0x7f3d6419c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d6ad5b700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d64101380 0x7f3d6419c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d6ad5b700 1 -- 172.21.15.45:0/3748762319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d641009c0 msgr2=0x7f3d64100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d6ad5b700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d641009c0 0x7f3d64100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d6ad5b700 1 -- 172.21.15.45:0/3748762319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c009d00 con 0x7f3d64101d60 2024-01-31T05:22:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d6ad5b700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d64101d60 0x7f3d6419ea80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c004d00 tx=0x7f3d5c004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d5b7fe700 1 -- 172.21.15.45:0/3748762319 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c00e040 con 0x7f3d64101d60 2024-01-31T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3748762319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6419f7d0 con 0x7f3d64101d60 2024-01-31T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d5b7fe700 1 -- 172.21.15.45:0/3748762319 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d5c009440 con 0x7f3d64101d60 2024-01-31T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d5b7fe700 1 -- 172.21.15.45:0/3748762319 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c01b910 con 0x7f3d64101d60 2024-01-31T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.697+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3748762319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6419fc40 con 0x7f3d64101d60 2024-01-31T05:22:14.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.698+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3748762319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d64107ac0 con 0x7f3d64101d60 2024-01-31T05:22:14.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.699+0000 7f3d5b7fe700 1 -- 172.21.15.45:0/3748762319 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f3d5c01dbe0 con 0x7f3d64101d60 2024-01-31T05:22:14.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.700+0000 7f3d5b7fe700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3d500a1370 0x7f3d500a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:14.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.700+0000 7f3d6a55a700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3d500a1370 0x7f3d500a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:14.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.700+0000 7f3d5b7fe700 1 -- 172.21.15.45:0/3748762319 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3d5c0cec90 con 0x7f3d64101d60 2024-01-31T05:22:14.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.700+0000 7f3d6a55a700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3d500a1370 0x7f3d500a3830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3d64069d00 tx=0x7f3d6000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:14.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.703+0000 7f3d5b7fe700 1 -- 172.21.15.45:0/3748762319 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f3d5c0981d0 con 0x7f3d64101d60 2024-01-31T05:22:14.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.909+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3748762319 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d6402d5a0 con 0x7f3d500a1370 2024-01-31T05:22:14.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.911+0000 7f3d5b7fe700 1 -- 172.21.15.45:0/3748762319 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f3d6402d5a0 con 0x7f3d500a1370 2024-01-31T05:22:14.910 INFO:teuthology.orchestra.run.smithi045.stdout:HOST ADDR LABELS STATUS 2024-01-31T05:22:14.910 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 172.21.15.45 2024-01-31T05:22:14.910 INFO:teuthology.orchestra.run.smithi045.stdout:smithi156 172.21.15.156 2024-01-31T05:22:14.910 INFO:teuthology.orchestra.run.smithi045.stdout:smithi186 172.21.15.186 2024-01-31T05:22:14.910 INFO:teuthology.orchestra.run.smithi045.stdout:3 hosts in cluster 2024-01-31T05:22:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.913+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3748762319 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3d500a1370 msgr2=0x7f3d500a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.913+0000 7f3d6c7be700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3d500a1370 0x7f3d500a3830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3d64069d00 tx=0x7f3d6000a380 comp rx=0 tx=0).stop 2024-01-31T05:22:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.913+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3748762319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d64101d60 msgr2=0x7f3d6419ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.913+0000 7f3d6c7be700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d64101d60 0x7f3d6419ea80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c004d00 tx=0x7f3d5c004d30 comp rx=0 tx=0).stop 2024-01-31T05:22:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.913+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3748762319 shutdown_connections 2024-01-31T05:22:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.913+0000 7f3d6c7be700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f3d500a1370 0x7f3d500a3830 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.913+0000 7f3d6c7be700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d64101d60 0x7f3d6419ea80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.913+0000 7f3d6c7be700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d641009c0 0x7f3d64100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:14.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.913+0000 7f3d6c7be700 1 --2- 172.21.15.45:0/3748762319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d64101380 0x7f3d6419c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:14.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.913+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3748762319 >> 172.21.15.45:0/3748762319 conn(0x7f3d640fb470 msgr2=0x7f3d64104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:14.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.913+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3748762319 shutdown_connections 2024-01-31T05:22:14.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:14.913+0000 7f3d6c7be700 1 -- 172.21.15.45:0/3748762319 wait complete. 2024-01-31T05:22:15.524 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- bash -c 'ceph orch device ls' 2024-01-31T05:22:15.770 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:22:15.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:15 smithi045 ceph-mon[31585]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:22:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:15 smithi156 ceph-mon[35720]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:22:16.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:15 smithi186 ceph-mon[35659]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:22:16.924 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:16 smithi045 ceph-mon[31585]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:16.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:16 smithi156 ceph-mon[35720]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:17.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:16 smithi186 ceph-mon[35659]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.397+0000 7f2c867a1700 1 -- 172.21.15.45:0/3256643392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c801021e0 msgr2=0x7f2c80108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.397+0000 7f2c867a1700 1 --2- 172.21.15.45:0/3256643392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c801021e0 0x7f2c80108a90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800b0d0 tx=0x7f2c7800b3e0 comp rx=0 tx=0).stop 2024-01-31T05:22:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.397+0000 7f2c867a1700 1 -- 172.21.15.45:0/3256643392 shutdown_connections 2024-01-31T05:22:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.397+0000 7f2c867a1700 1 --2- 172.21.15.45:0/3256643392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c801021e0 0x7f2c80108a90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.397+0000 7f2c867a1700 1 --2- 172.21.15.45:0/3256643392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c80100640 0x7f2c80100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.397+0000 7f2c867a1700 1 --2- 172.21.15.45:0/3256643392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c80101840 0x7f2c80101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.397+0000 7f2c867a1700 1 -- 172.21.15.45:0/3256643392 >> 172.21.15.45:0/3256643392 conn(0x7f2c800fbbc0 msgr2=0x7f2c800fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.397+0000 7f2c867a1700 1 -- 172.21.15.45:0/3256643392 shutdown_connections 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.397+0000 7f2c867a1700 1 -- 172.21.15.45:0/3256643392 wait complete. 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c867a1700 1 Processor -- start 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c867a1700 1 -- start start 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c867a1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c80100640 0x7f2c8006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c867a1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80101840 0x7f2c8019c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c867a1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801021e0 0x7f2c8019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c867a1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8019f360 con 0x7f2c80101840 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c867a1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8019f4d0 con 0x7f2c801021e0 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c867a1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8019f690 con 0x7f2c80100640 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c84f9e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80101840 0x7f2c8019c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c84f9e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80101840 0x7f2c8019c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48960/0 (socket says 172.21.15.45:48960) 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c84f9e700 1 -- 172.21.15.45:0/327289535 learned_addr learned my addr 172.21.15.45:0/327289535 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c8579f700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c80100640 0x7f2c8006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c85fa0700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801021e0 0x7f2c8019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c84f9e700 1 -- 172.21.15.45:0/327289535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c80100640 msgr2=0x7f2c8006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c84f9e700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c80100640 0x7f2c8006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c84f9e700 1 -- 172.21.15.45:0/327289535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801021e0 msgr2=0x7f2c8019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c84f9e700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801021e0 0x7f2c8019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c84f9e700 1 -- 172.21.15.45:0/327289535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c78009d00 con 0x7f2c80101840 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c85fa0700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801021e0 0x7f2c8019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.398+0000 7f2c84f9e700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80101840 0x7f2c8019c570 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b530 tx=0x7f2c7c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.399+0000 7f2c767fc700 1 -- 172.21.15.45:0/327289535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c011830 con 0x7f2c80101840 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.399+0000 7f2c867a1700 1 -- 172.21.15.45:0/327289535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8019f970 con 0x7f2c80101840 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.399+0000 7f2c767fc700 1 -- 172.21.15.45:0/327289535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c7c011e50 con 0x7f2c80101840 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.399+0000 7f2c767fc700 1 -- 172.21.15.45:0/327289535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00f2e0 con 0x7f2c80101840 2024-01-31T05:22:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.399+0000 7f2c867a1700 1 -- 172.21.15.45:0/327289535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8019ff40 con 0x7f2c80101840 2024-01-31T05:22:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.400+0000 7f2c867a1700 1 -- 172.21.15.45:0/327289535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c80107a80 con 0x7f2c80101840 2024-01-31T05:22:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.401+0000 7f2c767fc700 1 -- 172.21.15.45:0/327289535 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f2c7c010380 con 0x7f2c80101840 2024-01-31T05:22:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.401+0000 7f2c767fc700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2c6c0a1370 0x7f2c6c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.401+0000 7f2c8579f700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2c6c0a1370 0x7f2c6c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.401+0000 7f2c767fc700 1 -- 172.21.15.45:0/327289535 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2c7c0cc660 con 0x7f2c80101840 2024-01-31T05:22:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.402+0000 7f2c8579f700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2c6c0a1370 0x7f2c6c0a3830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2c801016a0 tx=0x7f2c70009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.406+0000 7f2c767fc700 1 -- 172.21.15.45:0/327289535 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f2c7c095b10 con 0x7f2c80101840 2024-01-31T05:22:17.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.616+0000 7f2c867a1700 1 -- 172.21.15.45:0/327289535 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c8002d4a0 con 0x7f2c6c0a1370 2024-01-31T05:22:17.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.618+0000 7f2c767fc700 1 -- 172.21.15.45:0/327289535 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f2c8002d4a0 con 0x7f2c6c0a1370 2024-01-31T05:22:17.617 INFO:teuthology.orchestra.run.smithi045.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-01-31T05:22:17.617 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-01-31T05:22:17.617 INFO:teuthology.orchestra.run.smithi045.stdout:smithi156 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760040480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-01-31T05:22:17.617 INFO:teuthology.orchestra.run.smithi045.stdout:smithi186 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-01-31T05:22:17.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.620+0000 7f2c867a1700 1 -- 172.21.15.45:0/327289535 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2c6c0a1370 msgr2=0x7f2c6c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:17.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.620+0000 7f2c867a1700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2c6c0a1370 0x7f2c6c0a3830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2c801016a0 tx=0x7f2c70009190 comp rx=0 tx=0).stop 2024-01-31T05:22:17.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.620+0000 7f2c867a1700 1 -- 172.21.15.45:0/327289535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80101840 msgr2=0x7f2c8019c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:17.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.620+0000 7f2c867a1700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80101840 0x7f2c8019c570 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b530 tx=0x7f2c7c00b840 comp rx=0 tx=0).stop 2024-01-31T05:22:17.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.620+0000 7f2c867a1700 1 -- 172.21.15.45:0/327289535 shutdown_connections 2024-01-31T05:22:17.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.620+0000 7f2c867a1700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801021e0 0x7f2c8019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:17.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.620+0000 7f2c867a1700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c80100640 0x7f2c8006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:17.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.620+0000 7f2c867a1700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2c6c0a1370 0x7f2c6c0a3830 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:17.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.620+0000 7f2c867a1700 1 --2- 172.21.15.45:0/327289535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80101840 0x7f2c8019c570 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:17.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.620+0000 7f2c867a1700 1 -- 172.21.15.45:0/327289535 >> 172.21.15.45:0/327289535 conn(0x7f2c800fbbc0 msgr2=0x7f2c80104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:17.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.620+0000 7f2c867a1700 1 -- 172.21.15.45:0/327289535 shutdown_connections 2024-01-31T05:22:17.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:17.620+0000 7f2c867a1700 1 -- 172.21.15.45:0/327289535 wait complete. 2024-01-31T05:22:17.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:17 smithi045 ceph-mon[31585]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:22:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:17 smithi156 ceph-mon[35720]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:22:18.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:17 smithi186 ceph-mon[35659]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:22:18.335 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-01-31T05:22:18.344 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-01-31T05:22:18.345 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- bash -c 'ceph fs dump' 2024-01-31T05:22:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:22:18.869 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:18 smithi045 ceph-mon[31585]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:18 smithi156 ceph-mon[35720]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:19.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:18 smithi186 ceph-mon[35659]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:19.833 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:19 smithi045 ceph-mon[31585]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:22:19.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:19 smithi156 ceph-mon[35720]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:22:20.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:19 smithi186 ceph-mon[35659]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:22:20.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.299+0000 7f2001f7d700 1 -- 172.21.15.45:0/540571796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ffc100770 msgr2=0x7f1ffc100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:20.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.299+0000 7f2001f7d700 1 --2- 172.21.15.45:0/540571796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ffc100770 0x7f1ffc100b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0099c0 tx=0x7f1fec009cd0 comp rx=0 tx=0).stop 2024-01-31T05:22:20.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.299+0000 7f2001f7d700 1 -- 172.21.15.45:0/540571796 shutdown_connections 2024-01-31T05:22:20.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.299+0000 7f2001f7d700 1 --2- 172.21.15.45:0/540571796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ffc102310 0x7f1ffc108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:20.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.299+0000 7f2001f7d700 1 --2- 172.21.15.45:0/540571796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ffc100770 0x7f1ffc100b90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.299+0000 7f2001f7d700 1 --2- 172.21.15.45:0/540571796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ffc101970 0x7f1ffc101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.299+0000 7f2001f7d700 1 -- 172.21.15.45:0/540571796 >> 172.21.15.45:0/540571796 conn(0x7f1ffc0fbd30 msgr2=0x7f1ffc0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.299+0000 7f2001f7d700 1 -- 172.21.15.45:0/540571796 shutdown_connections 2024-01-31T05:22:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.299+0000 7f2001f7d700 1 -- 172.21.15.45:0/540571796 wait complete. 2024-01-31T05:22:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.299+0000 7f2001f7d700 1 Processor -- start 2024-01-31T05:22:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f2001f7d700 1 -- start start 2024-01-31T05:22:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f2001f7d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ffc100770 0x7f1ffc197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f2001f7d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ffc101970 0x7f1ffc197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f2001f7d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ffc102310 0x7f1ffc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f2001f7d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc19f300 con 0x7f1ffc100770 2024-01-31T05:22:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f2001f7d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc19f440 con 0x7f1ffc101970 2024-01-31T05:22:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f2001f7d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc19f5d0 con 0x7f1ffc102310 2024-01-31T05:22:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f1ffb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ffc100770 0x7f1ffc197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f1ffb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ffc100770 0x7f1ffc197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48656/0 (socket says 172.21.15.45:48656) 2024-01-31T05:22:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f1ffb7fe700 1 -- 172.21.15.45:0/829672034 learned_addr learned my addr 172.21.15.45:0/829672034 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f1ffbfff700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ffc102310 0x7f1ffc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f1ffaffd700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ffc101970 0x7f1ffc197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f1ffb7fe700 1 -- 172.21.15.45:0/829672034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ffc102310 msgr2=0x7f1ffc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f1ffb7fe700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ffc102310 0x7f1ffc19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f1ffb7fe700 1 -- 172.21.15.45:0/829672034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ffc101970 msgr2=0x7f1ffc197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f1ffb7fe700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ffc101970 0x7f1ffc197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f1ffb7fe700 1 -- 172.21.15.45:0/829672034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec0095f0 con 0x7f1ffc100770 2024-01-31T05:22:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.300+0000 7f1ffb7fe700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ffc100770 0x7f1ffc197770 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1fec000c00 tx=0x7f1fec0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.301+0000 7f1ff8ff9700 1 -- 172.21.15.45:0/829672034 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec01d070 con 0x7f1ffc100770 2024-01-31T05:22:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.301+0000 7f2001f7d700 1 -- 172.21.15.45:0/829672034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ffc19f850 con 0x7f1ffc100770 2024-01-31T05:22:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.301+0000 7f1ff8ff9700 1 -- 172.21.15.45:0/829672034 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fec003f90 con 0x7f1ffc100770 2024-01-31T05:22:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.301+0000 7f1ff8ff9700 1 -- 172.21.15.45:0/829672034 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec022c20 con 0x7f1ffc100770 2024-01-31T05:22:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.301+0000 7f2001f7d700 1 -- 172.21.15.45:0/829672034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc19fcf0 con 0x7f1ffc100770 2024-01-31T05:22:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.302+0000 7f2001f7d700 1 -- 172.21.15.45:0/829672034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ffc0684a0 con 0x7f1ffc100770 2024-01-31T05:22:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.303+0000 7f1ff8ff9700 1 -- 172.21.15.45:0/829672034 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f1fec022470 con 0x7f1ffc100770 2024-01-31T05:22:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.303+0000 7f1ff8ff9700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1fe40a1370 0x7f1fe40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.303+0000 7f1ffaffd700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1fe40a1370 0x7f1fe40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.303+0000 7f1ff8ff9700 1 -- 172.21.15.45:0/829672034 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1fec0ce570 con 0x7f1ffc100770 2024-01-31T05:22:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.304+0000 7f1ffaffd700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1fe40a1370 0x7f1fe40a3830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0005950 tx=0x7f1ff000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.307+0000 7f1ff8ff9700 1 -- 172.21.15.45:0/829672034 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1fec097ab0 con 0x7f1ffc100770 2024-01-31T05:22:20.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.552+0000 7f2001f7d700 1 -- 172.21.15.45:0/829672034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1ffc10c110 con 0x7f1ffc100770 2024-01-31T05:22:20.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.552+0000 7f1ff8ff9700 1 -- 172.21.15.45:0/829672034 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+2943 (secure 0 0 0) 0x7f1fec09b0d0 con 0x7f1ffc100770 2024-01-31T05:22:20.551 INFO:teuthology.orchestra.run.smithi045.stdout:e20 2024-01-31T05:22:20.552 INFO:teuthology.orchestra.run.smithi045.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-01-31T05:22:20.552 INFO:teuthology.orchestra.run.smithi045.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-01-31T05:22:20.552 INFO:teuthology.orchestra.run.smithi045.stdout:legacy client fscid: 1 2024-01-31T05:22:20.552 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:20.552 INFO:teuthology.orchestra.run.smithi045.stdout:Filesystem 'cephfs' (1) 2024-01-31T05:22:20.552 INFO:teuthology.orchestra.run.smithi045.stdout:fs_name cephfs 2024-01-31T05:22:20.552 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 20 2024-01-31T05:22:20.552 INFO:teuthology.orchestra.run.smithi045.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-01-31T05:22:20.552 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-01-31T05:20:40.057668+0000 2024-01-31T05:22:20.552 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-01-31T05:20:44.476398+0000 2024-01-31T05:22:20.552 INFO:teuthology.orchestra.run.smithi045.stdout:tableserver 0 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:root 0 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:session_timeout 300 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:session_autoclose 300 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:max_file_size 1099511627776 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:max_xattr_size 65536 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:required_client_features {} 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure 0 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure_osd_epoch 0 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:max_mds 3 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:in 0,1,2 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:up {0=24503,1=24490,2=24469} 2024-01-31T05:22:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:failed 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:damaged 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:stopped 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:data_pools [3] 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:metadata_pool 2 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:inline_data disabled 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:balancer 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:bal_rank_mask -1 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:standby_count_wanted 0 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b{0:24503} state up:active seq 3 addr [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] compat {c=[1],r=[1],i=[fff]}] 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.h{0:24467} state up:standby-replay seq 1 addr [v2:172.21.15.156:6836/3782584640,v1:172.21.15.156:6837/3782584640] compat {c=[1],r=[1],i=[fff]}] 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.i{1:24490} state up:active seq 5 addr [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] compat {c=[1],r=[1],i=[fff]}] 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.f{1:24460} state up:standby-replay seq 1 addr [v2:172.21.15.186:6834/696958495,v1:172.21.15.186:6835/696958495] compat {c=[1],r=[1],i=[fff]}] 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c{2:24469} state up:active seq 7 addr [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] compat {c=[1],r=[1],i=[fff]}] 2024-01-31T05:22:20.554 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.e{2:24440} state up:standby-replay seq 1 addr [v2:172.21.15.156:6834/4045803513,v1:172.21.15.156:6835/4045803513] compat {c=[1],r=[1],i=[fff]}] 2024-01-31T05:22:20.555 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:20.555 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:20.555 INFO:teuthology.orchestra.run.smithi045.stdout:Standby daemons: 2024-01-31T05:22:20.555 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:20.555 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.j{-1:14550} state up:standby seq 1 addr [v2:172.21.15.45:6834/363421179,v1:172.21.15.45:6835/363421179] compat {c=[1],r=[1],i=[fff]}] 2024-01-31T05:22:20.555 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d{-1:14568} state up:standby seq 1 addr [v2:172.21.15.45:6836/176161833,v1:172.21.15.45:6837/176161833] compat {c=[1],r=[1],i=[fff]}] 2024-01-31T05:22:20.555 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.g{-1:14631} state up:standby seq 1 addr [v2:172.21.15.45:6838/1801709710,v1:172.21.15.45:6839/1801709710] compat {c=[1],r=[1],i=[fff]}] 2024-01-31T05:22:20.555 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a{-1:14643} state up:standby seq 1 addr [v2:172.21.15.45:6840/1148586810,v1:172.21.15.45:6841/1148586810] compat {c=[1],r=[1],i=[fff]}] 2024-01-31T05:22:20.555 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.k{-1:24398} state up:standby seq 1 addr [v2:172.21.15.156:6832/2627326005,v1:172.21.15.156:6833/2627326005] compat {c=[1],r=[1],i=[fff]}] 2024-01-31T05:22:20.555 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.l{-1:24415} state up:standby seq 1 addr [v2:172.21.15.186:6832/3707036230,v1:172.21.15.186:6833/3707036230] compat {c=[1],r=[1],i=[fff]}] 2024-01-31T05:22:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.554+0000 7f2001f7d700 1 -- 172.21.15.45:0/829672034 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1fe40a1370 msgr2=0x7f1fe40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.554+0000 7f2001f7d700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1fe40a1370 0x7f1fe40a3830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0005950 tx=0x7f1ff000b410 comp rx=0 tx=0).stop 2024-01-31T05:22:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.554+0000 7f2001f7d700 1 -- 172.21.15.45:0/829672034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ffc100770 msgr2=0x7f1ffc197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.554+0000 7f2001f7d700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ffc100770 0x7f1ffc197770 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1fec000c00 tx=0x7f1fec0057c0 comp rx=0 tx=0).stop 2024-01-31T05:22:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.554+0000 7f2001f7d700 1 -- 172.21.15.45:0/829672034 shutdown_connections 2024-01-31T05:22:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.554+0000 7f2001f7d700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ffc102310 0x7f1ffc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.554+0000 7f2001f7d700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1fe40a1370 0x7f1fe40a3830 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.554+0000 7f2001f7d700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ffc100770 0x7f1ffc197770 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.554+0000 7f2001f7d700 1 --2- 172.21.15.45:0/829672034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ffc101970 0x7f1ffc197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.554+0000 7f2001f7d700 1 -- 172.21.15.45:0/829672034 >> 172.21.15.45:0/829672034 conn(0x7f1ffc0fbd30 msgr2=0x7f1ffc0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.554+0000 7f2001f7d700 1 -- 172.21.15.45:0/829672034 shutdown_connections 2024-01-31T05:22:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:20.554+0000 7f2001f7d700 1 -- 172.21.15.45:0/829672034 wait complete. 2024-01-31T05:22:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 20 2024-01-31T05:22:20.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:20 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/829672034' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-01-31T05:22:20.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:20 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/829672034' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-01-31T05:22:21.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:20 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/829672034' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-01-31T05:22:21.355 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- bash -c 'ceph osd dump' 2024-01-31T05:22:21.603 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:22:21.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:21 smithi156 ceph-mon[35720]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:22:22.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:21 smithi045 ceph-mon[31585]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:22:22.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:21 smithi186 ceph-mon[35659]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.284+0000 7fa82a8f0700 1 -- 172.21.15.45:0/1009826497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa824100710 msgr2=0x7fa824100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.284+0000 7fa82a8f0700 1 --2- 172.21.15.45:0/1009826497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa824100710 0x7fa824100b30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa8140099c0 tx=0x7fa814009cd0 comp rx=0 tx=0).stop 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.284+0000 7fa82a8f0700 1 -- 172.21.15.45:0/1009826497 shutdown_connections 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.284+0000 7fa82a8f0700 1 --2- 172.21.15.45:0/1009826497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8241022b0 0x7fa824108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.284+0000 7fa82a8f0700 1 --2- 172.21.15.45:0/1009826497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa824100710 0x7fa824100b30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.284+0000 7fa82a8f0700 1 --2- 172.21.15.45:0/1009826497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa824101910 0x7fa824101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.284+0000 7fa82a8f0700 1 -- 172.21.15.45:0/1009826497 >> 172.21.15.45:0/1009826497 conn(0x7fa8240fbcd0 msgr2=0x7fa8240fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.284+0000 7fa82a8f0700 1 -- 172.21.15.45:0/1009826497 shutdown_connections 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.284+0000 7fa82a8f0700 1 -- 172.21.15.45:0/1009826497 wait complete. 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.285+0000 7fa82a8f0700 1 Processor -- start 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.285+0000 7fa82a8f0700 1 -- start start 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.285+0000 7fa82a8f0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa824100710 0x7fa8241977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:23.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.285+0000 7fa82a8f0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa824101910 0x7fa824197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.285+0000 7fa82a8f0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8241022b0 0x7fa82419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.285+0000 7fa82a8f0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82419f320 con 0x7fa8241022b0 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.285+0000 7fa82a8f0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82419f460 con 0x7fa824100710 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.285+0000 7fa82a8f0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82419f5f0 con 0x7fa824101910 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa828e8d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8241022b0 0x7fa82419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa828e8d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8241022b0 0x7fa82419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48664/0 (socket says 172.21.15.45:48664) 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa828e8d700 1 -- 172.21.15.45:0/377101544 learned_addr learned my addr 172.21.15.45:0/377101544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa823fff700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa824100710 0x7fa8241977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa8237fe700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa824101910 0x7fa824197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa828e8d700 1 -- 172.21.15.45:0/377101544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa824101910 msgr2=0x7fa824197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa828e8d700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa824101910 0x7fa824197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa828e8d700 1 -- 172.21.15.45:0/377101544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa824100710 msgr2=0x7fa8241977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa828e8d700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa824100710 0x7fa8241977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa828e8d700 1 -- 172.21.15.45:0/377101544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140095f0 con 0x7fa8241022b0 2024-01-31T05:22:23.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa828e8d700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8241022b0 0x7fa82419ed50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa81c007f10 tx=0x7fa81c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:23.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa8217fa700 1 -- 172.21.15.45:0/377101544 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c017070 con 0x7fa8241022b0 2024-01-31T05:22:23.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa82a8f0700 1 -- 172.21.15.45:0/377101544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82419f8d0 con 0x7fa8241022b0 2024-01-31T05:22:23.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa8217fa700 1 -- 172.21.15.45:0/377101544 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa81c00f040 con 0x7fa8241022b0 2024-01-31T05:22:23.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa8217fa700 1 -- 172.21.15.45:0/377101544 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c005610 con 0x7fa8241022b0 2024-01-31T05:22:23.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.286+0000 7fa82a8f0700 1 -- 172.21.15.45:0/377101544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82419fdd0 con 0x7fa8241022b0 2024-01-31T05:22:23.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.287+0000 7fa82a8f0700 1 -- 172.21.15.45:0/377101544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8240684a0 con 0x7fa8241022b0 2024-01-31T05:22:23.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.288+0000 7fa8217fa700 1 -- 172.21.15.45:0/377101544 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fa81c004ad0 con 0x7fa8241022b0 2024-01-31T05:22:23.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.289+0000 7fa8217fa700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa80c0a1370 0x7fa80c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:23.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.289+0000 7fa823fff700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa80c0a1370 0x7fa80c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:23.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.289+0000 7fa8217fa700 1 -- 172.21.15.45:0/377101544 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa81c0cc270 con 0x7fa8241022b0 2024-01-31T05:22:23.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.289+0000 7fa823fff700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa80c0a1370 0x7fa80c0a3830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa814003820 tx=0x7fa814005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:23.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.293+0000 7fa8217fa700 1 -- 172.21.15.45:0/377101544 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fa81c095720 con 0x7fa8241022b0 2024-01-31T05:22:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.496+0000 7fa82a8f0700 1 -- 172.21.15.45:0/377101544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fa8241a01a0 con 0x7fa8241022b0 2024-01-31T05:22:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.497+0000 7fa8217fa700 1 -- 172.21.15.45:0/377101544 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5263 (secure 0 0 0) 0x7fa81c098d40 con 0x7fa8241022b0 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 88 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-01-31T05:12:51.341846+0000 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-01-31T05:21:47.682334+0000 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:crush_version 26 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:full_ratio 0.95 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:backfillfull_ratio 0.9 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:nearfull_ratio 0.85 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:require_min_compat_client luminous 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:min_compat_client luminous 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:require_osd_release squid 2024-01-31T05:22:23.497 INFO:teuthology.orchestra.run.smithi045.stdout:stretch_mode_enabled false 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:max_osd 12 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6802/4026202784,v1:172.21.15.45:6803/4026202784] [v2:172.21.15.45:6804/4026202784,v1:172.21.15.45:6805/4026202784] exists,up 5b06f7af-6fee-4838-afe2-f178362f0a88 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6800/2623092514,v1:172.21.15.156:6801/2623092514] [v2:172.21.15.156:6802/2623092514,v1:172.21.15.156:6803/2623092514] exists,up 2ae3ab3f-342a-4e5a-93a2-a674831b30bd 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6800/811586396,v1:172.21.15.186:6801/811586396] [v2:172.21.15.186:6802/811586396,v1:172.21.15.186:6803/811586396] exists,up 6c8acbd3-748d-4107-85e4-766d8820c5ee 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6810/3653915645,v1:172.21.15.45:6811/3653915645] [v2:172.21.15.45:6812/3653915645,v1:172.21.15.45:6813/3653915645] exists,up d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6808/2291851752,v1:172.21.15.156:6809/2291851752] [v2:172.21.15.156:6810/2291851752,v1:172.21.15.156:6811/2291851752] exists,up 4ae77293-4355-4aac-87a5-220b0754e21c 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6808/2142758307,v1:172.21.15.186:6809/2142758307] [v2:172.21.15.186:6810/2142758307,v1:172.21.15.186:6811/2142758307] exists,up 69641e46-7955-4b81-990e-bf656f6d7ff2 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6818/2753791681,v1:172.21.15.45:6819/2753791681] [v2:172.21.15.45:6820/2753791681,v1:172.21.15.45:6821/2753791681] exists,up 9449230f-ba33-45ee-993a-12a376e2c9ff 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6816/2686361879,v1:172.21.15.156:6817/2686361879] [v2:172.21.15.156:6818/2686361879,v1:172.21.15.156:6819/2686361879] exists,up de8fa2cd-6e29-40b8-836f-90a64e4b09a1 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6816/2054139759,v1:172.21.15.186:6817/2054139759] [v2:172.21.15.186:6818/2054139759,v1:172.21.15.186:6819/2054139759] exists,up 023fed4e-b2c5-4931-9bc8-04461d8bd939 2024-01-31T05:22:23.498 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6826/717478931,v1:172.21.15.45:6827/717478931] [v2:172.21.15.45:6828/717478931,v1:172.21.15.45:6829/717478931] exists,up bfaec4e4-25e7-40b8-a571-0b8a0bc40e75 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6824/2036478526,v1:172.21.15.156:6825/2036478526] [v2:172.21.15.156:6826/2036478526,v1:172.21.15.156:6827/2036478526] exists,up 2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6824/3877143910,v1:172.21.15.186:6825/3877143910] [v2:172.21.15.186:6826/3877143910,v1:172.21.15.186:6827/3877143910] exists,up a26f7c8a-679e-43d2-b923-01f3358ffcc7 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.8 [2,11] 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.1d [8,11] 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.e [4,8] 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.1b [4,6] 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.21 [5,9] 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.29 [3,9] 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.3c [4,9] 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3329080521 expires 2024-02-01T05:13:46.510704+0000 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/745566822 expires 2024-02-01T05:13:46.510704+0000 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/4050317147 expires 2024-02-01T05:13:46.510704+0000 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/4050317147 expires 2024-02-01T05:13:46.510704+0000 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2019451955 expires 2024-02-01T05:13:20.248709+0000 2024-01-31T05:22:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3020673835 expires 2024-02-01T05:13:20.248709+0000 2024-01-31T05:22:23.500 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/1162729130 expires 2024-02-01T05:13:20.248709+0000 2024-01-31T05:22:23.500 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/2712881513 expires 2024-02-01T05:13:20.248709+0000 2024-01-31T05:22:23.500 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3852874833 expires 2024-02-01T05:13:46.510704+0000 2024-01-31T05:22:23.500 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/2712881513 expires 2024-02-01T05:13:20.248709+0000 2024-01-31T05:22:23.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.499+0000 7fa82a8f0700 1 -- 172.21.15.45:0/377101544 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa80c0a1370 msgr2=0x7fa80c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:23.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.499+0000 7fa82a8f0700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa80c0a1370 0x7fa80c0a3830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa814003820 tx=0x7fa814005ad0 comp rx=0 tx=0).stop 2024-01-31T05:22:23.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.499+0000 7fa82a8f0700 1 -- 172.21.15.45:0/377101544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8241022b0 msgr2=0x7fa82419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:23.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.499+0000 7fa82a8f0700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8241022b0 0x7fa82419ed50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa81c007f10 tx=0x7fa81c00d3b0 comp rx=0 tx=0).stop 2024-01-31T05:22:23.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.499+0000 7fa82a8f0700 1 -- 172.21.15.45:0/377101544 shutdown_connections 2024-01-31T05:22:23.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.499+0000 7fa82a8f0700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fa80c0a1370 0x7fa80c0a3830 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:23.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.499+0000 7fa82a8f0700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8241022b0 0x7fa82419ed50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:23.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.499+0000 7fa82a8f0700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa824100710 0x7fa8241977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:23.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.499+0000 7fa82a8f0700 1 --2- 172.21.15.45:0/377101544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa824101910 0x7fa824197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:23.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.499+0000 7fa82a8f0700 1 -- 172.21.15.45:0/377101544 >> 172.21.15.45:0/377101544 conn(0x7fa8240fbcd0 msgr2=0x7fa824069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:23.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.499+0000 7fa82a8f0700 1 -- 172.21.15.45:0/377101544 shutdown_connections 2024-01-31T05:22:23.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:23.500+0000 7fa82a8f0700 1 -- 172.21.15.45:0/377101544 wait complete. 2024-01-31T05:22:23.813 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:23 smithi045 ceph-mon[31585]: pgmap v336: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:22:23.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:23 smithi186 ceph-mon[35659]: pgmap v336: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:22:23.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:23 smithi156 ceph-mon[35720]: pgmap v336: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:22:24.194 INFO:teuthology.run_tasks:Running task fs.ready... 2024-01-31T05:22:24.205 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:22:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.318+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/2244400586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b54106ad0 msgr2=0x7f0b54108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.318+0000 7f0b5b2f8640 1 --2- 172.21.15.45:0/2244400586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b54106ad0 0x7f0b54108ec0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009ca0 tx=0x7f0b4c02c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.318+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/2244400586 shutdown_connections 2024-01-31T05:22:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.318+0000 7f0b5b2f8640 1 --2- 172.21.15.45:0/2244400586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b54106ad0 0x7f0b54108ec0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.318+0000 7f0b5b2f8640 1 --2- 172.21.15.45:0/2244400586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b541041a0 0x7f0b54106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.318+0000 7f0b5b2f8640 1 --2- 172.21.15.45:0/2244400586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b54101870 0x7f0b54103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.318+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/2244400586 >> 172.21.15.45:0/2244400586 conn(0x7f0b540fb220 msgr2=0x7f0b540fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.318+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/2244400586 shutdown_connections 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.318+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/2244400586 wait complete. 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b5b2f8640 1 Processor -- start 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b5b2f8640 1 -- start start 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b5b2f8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b54101870 0x7f0b54196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b5b2f8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b541041a0 0x7f0b54197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b5b2f8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b54106ad0 0x7f0b5419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b5b2f8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5419e7f0 con 0x7f0b54106ad0 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b5b2f8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5419e960 con 0x7f0b541041a0 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b5b2f8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5419eb20 con 0x7f0b54101870 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b59895640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b54106ad0 0x7f0b5419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b59895640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b54106ad0 0x7f0b5419e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48674/0 (socket says 172.21.15.45:48674) 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b59895640 1 -- 172.21.15.45:0/3452279340 learned_addr learned my addr 172.21.15.45:0/3452279340 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b59895640 1 -- 172.21.15.45:0/3452279340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b54101870 msgr2=0x7f0b54196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b59895640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b54101870 0x7f0b54196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b59895640 1 -- 172.21.15.45:0/3452279340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b541041a0 msgr2=0x7f0b54197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b58893640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b541041a0 0x7f0b54197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b59895640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b541041a0 0x7f0b54197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b59895640 1 -- 172.21.15.45:0/3452279340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c009900 con 0x7f0b54106ad0 2024-01-31T05:22:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b58893640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b541041a0 0x7f0b54197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b59895640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b54106ad0 0x7f0b5419e280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c037040 tx=0x7f0b4c032a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.319+0000 7f0b4a7fc640 1 -- 172.21.15.45:0/3452279340 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c003970 con 0x7f0b54106ad0 2024-01-31T05:22:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.320+0000 7f0b4a7fc640 1 -- 172.21.15.45:0/3452279340 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4c004040 con 0x7f0b54106ad0 2024-01-31T05:22:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.320+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/3452279340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5419eda0 con 0x7f0b54106ad0 2024-01-31T05:22:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.320+0000 7f0b4a7fc640 1 -- 172.21.15.45:0/3452279340 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c03d650 con 0x7f0b54106ad0 2024-01-31T05:22:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.320+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/3452279340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5419f270 con 0x7f0b54106ad0 2024-01-31T05:22:24.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.320+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/3452279340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b540fcd10 con 0x7f0b54106ad0 2024-01-31T05:22:24.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.322+0000 7f0b4a7fc640 1 -- 172.21.15.45:0/3452279340 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f0b4c03b040 con 0x7f0b54106ad0 2024-01-31T05:22:24.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.322+0000 7f0b4a7fc640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0b400a12e0 0x7f0b400a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:24.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.322+0000 7f0b59094640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0b400a12e0 0x7f0b400a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:24.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.322+0000 7f0b4a7fc640 1 -- 172.21.15.45:0/3452279340 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0b4c0ef970 con 0x7f0b54106ad0 2024-01-31T05:22:24.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.323+0000 7f0b59094640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0b400a12e0 0x7f0b400a37a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0b541981e0 tx=0x7f0b50006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:24.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.326+0000 7f0b4a7fc640 1 -- 172.21.15.45:0/3452279340 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f0b4c0b8d90 con 0x7f0b54106ad0 2024-01-31T05:22:24.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.507+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/3452279340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b5410c890 con 0x7f0b54106ad0 2024-01-31T05:22:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.508+0000 7f0b4a7fc640 1 -- 172.21.15.45:0/3452279340 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22520 (secure 0 0 0) 0x7f0b4c0bacc0 con 0x7f0b54106ad0 2024-01-31T05:22:24.508 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:24.508 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":88,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:21:47.682334+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.510+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/3452279340 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0b400a12e0 msgr2=0x7f0b400a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.510+0000 7f0b5b2f8640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0b400a12e0 0x7f0b400a37a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0b541981e0 tx=0x7f0b50006c90 comp rx=0 tx=0).stop 2024-01-31T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.510+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/3452279340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b54106ad0 msgr2=0x7f0b5419e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.510+0000 7f0b5b2f8640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b54106ad0 0x7f0b5419e280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c037040 tx=0x7f0b4c032a40 comp rx=0 tx=0).stop 2024-01-31T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.510+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/3452279340 shutdown_connections 2024-01-31T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.510+0000 7f0b5b2f8640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f0b400a12e0 0x7f0b400a37a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.510+0000 7f0b5b2f8640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b54106ad0 0x7f0b5419e280 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.510+0000 7f0b5b2f8640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b541041a0 0x7f0b54197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.510+0000 7f0b5b2f8640 1 --2- 172.21.15.45:0/3452279340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b54101870 0x7f0b54196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.510+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/3452279340 >> 172.21.15.45:0/3452279340 conn(0x7f0b540fb220 msgr2=0x7f0b541021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.511+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/3452279340 shutdown_connections 2024-01-31T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.511+0000 7f0b5b2f8640 1 -- 172.21.15.45:0/3452279340 wait complete. 2024-01-31T05:22:24.521 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:16:10.856581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:20:38.347599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:20:39.270522+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:22:24.521 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:22:24.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.627+0000 7fd0eff42640 1 -- 172.21.15.45:0/253510347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0e81008c0 msgr2=0x7fd0e8100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.627+0000 7fd0eff42640 1 --2- 172.21.15.45:0/253510347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0e81008c0 0x7fd0e8100d40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009ca0 tx=0x7fd0e402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.627+0000 7fd0eff42640 1 -- 172.21.15.45:0/253510347 shutdown_connections 2024-01-31T05:22:24.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.627+0000 7fd0eff42640 1 --2- 172.21.15.45:0/253510347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0e81008c0 0x7fd0e8100d40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.627+0000 7fd0eff42640 1 --2- 172.21.15.45:0/253510347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0e80fff20 0x7fd0e8100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.627+0000 7fd0eff42640 1 --2- 172.21.15.45:0/253510347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0e80ff5c0 0x7fd0e80ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.627+0000 7fd0eff42640 1 -- 172.21.15.45:0/253510347 >> 172.21.15.45:0/253510347 conn(0x7fd0e80fb180 msgr2=0x7fd0e80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.627+0000 7fd0eff42640 1 -- 172.21.15.45:0/253510347 shutdown_connections 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.627+0000 7fd0eff42640 1 -- 172.21.15.45:0/253510347 wait complete. 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0eff42640 1 Processor -- start 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0eff42640 1 -- start start 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0eff42640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0e80ff5c0 0x7fd0e8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0eff42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0e80fff20 0x7fd0e81993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0eff42640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0e81008c0 0x7fd0e81a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0eff42640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e81a0990 con 0x7fd0e80fff20 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0eff42640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e81a0b00 con 0x7fd0e80ff5c0 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0eff42640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e81a0cc0 con 0x7fd0e81008c0 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0ed4dd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0e80fff20 0x7fd0e81993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0ed4dd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0e80fff20 0x7fd0e81993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48692/0 (socket says 172.21.15.45:48692) 2024-01-31T05:22:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0ed4dd640 1 -- 172.21.15.45:0/2211880192 learned_addr learned my addr 172.21.15.45:0/2211880192 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0ed4dd640 1 -- 172.21.15.45:0/2211880192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0e81008c0 msgr2=0x7fd0e81a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0ed4dd640 1 --2- 172.21.15.45:0/2211880192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0e81008c0 0x7fd0e81a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0ed4dd640 1 -- 172.21.15.45:0/2211880192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0e80ff5c0 msgr2=0x7fd0e8198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0ed4dd640 1 --2- 172.21.15.45:0/2211880192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0e80ff5c0 0x7fd0e8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.628+0000 7fd0ed4dd640 1 -- 172.21.15.45:0/2211880192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e4009900 con 0x7fd0e80fff20 2024-01-31T05:22:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.629+0000 7fd0ed4dd640 1 --2- 172.21.15.45:0/2211880192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0e80fff20 0x7fd0e81993e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800d950 tx=0x7fd0d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.629+0000 7fd0deffd640 1 -- 172.21.15.45:0/2211880192 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8009c30 con 0x7fd0e80fff20 2024-01-31T05:22:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.629+0000 7fd0deffd640 1 -- 172.21.15.45:0/2211880192 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0d8009d90 con 0x7fd0e80fff20 2024-01-31T05:22:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.629+0000 7fd0eff42640 1 -- 172.21.15.45:0/2211880192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e81a0fa0 con 0x7fd0e80fff20 2024-01-31T05:22:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.629+0000 7fd0deffd640 1 -- 172.21.15.45:0/2211880192 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d800f740 con 0x7fd0e80fff20 2024-01-31T05:22:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.629+0000 7fd0eff42640 1 -- 172.21.15.45:0/2211880192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e81a1420 con 0x7fd0e80fff20 2024-01-31T05:22:24.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.630+0000 7fd0eff42640 1 -- 172.21.15.45:0/2211880192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e810a4e0 con 0x7fd0e80fff20 2024-01-31T05:22:24.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.631+0000 7fd0deffd640 1 -- 172.21.15.45:0/2211880192 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fd0d8010a20 con 0x7fd0e80fff20 2024-01-31T05:22:24.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.631+0000 7fd0deffd640 1 --2- 172.21.15.45:0/2211880192 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd0d40a12e0 0x7fd0d40a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:24.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.632+0000 7fd0edcde640 1 --2- 172.21.15.45:0/2211880192 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd0d40a12e0 0x7fd0d40a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:24.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.632+0000 7fd0deffd640 1 -- 172.21.15.45:0/2211880192 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd0d80cc810 con 0x7fd0e80fff20 2024-01-31T05:22:24.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.632+0000 7fd0edcde640 1 --2- 172.21.15.45:0/2211880192 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd0d40a12e0 0x7fd0d40a37a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0005fd0 tx=0x7fd0e0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:24.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.636+0000 7fd0deffd640 1 -- 172.21.15.45:0/2211880192 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd0d8095cc0 con 0x7fd0e80fff20 2024-01-31T05:22:24.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.818+0000 7fd0eff42640 1 -- 172.21.15.45:0/2211880192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0e802cbb0 con 0x7fd0e80fff20 2024-01-31T05:22:24.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.818+0000 7fd0deffd640 1 -- 172.21.15.45:0/2211880192 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd0d80992e0 con 0x7fd0e80fff20 2024-01-31T05:22:24.817 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-01-31T05:22:24.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.820+0000 7fd0eff42640 1 -- 172.21.15.45:0/2211880192 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd0d40a12e0 msgr2=0x7fd0d40a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.820+0000 7fd0eff42640 1 --2- 172.21.15.45:0/2211880192 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd0d40a12e0 0x7fd0d40a37a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0005fd0 tx=0x7fd0e0005850 comp rx=0 tx=0).stop 2024-01-31T05:22:24.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.820+0000 7fd0eff42640 1 -- 172.21.15.45:0/2211880192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0e80fff20 msgr2=0x7fd0e81993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.820+0000 7fd0eff42640 1 --2- 172.21.15.45:0/2211880192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0e80fff20 0x7fd0e81993e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800d950 tx=0x7fd0d800de10 comp rx=0 tx=0).stop 2024-01-31T05:22:24.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.820+0000 7fd0eff42640 1 -- 172.21.15.45:0/2211880192 shutdown_connections 2024-01-31T05:22:24.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.820+0000 7fd0eff42640 1 --2- 172.21.15.45:0/2211880192 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fd0d40a12e0 0x7fd0d40a37a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.820+0000 7fd0eff42640 1 --2- 172.21.15.45:0/2211880192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0e80fff20 0x7fd0e81993e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.820+0000 7fd0eff42640 1 --2- 172.21.15.45:0/2211880192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0e81008c0 0x7fd0e81a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.820+0000 7fd0eff42640 1 --2- 172.21.15.45:0/2211880192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0e80ff5c0 0x7fd0e8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.820+0000 7fd0eff42640 1 -- 172.21.15.45:0/2211880192 >> 172.21.15.45:0/2211880192 conn(0x7fd0e80fb180 msgr2=0x7fd0e80fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:24.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.820+0000 7fd0eff42640 1 -- 172.21.15.45:0/2211880192 shutdown_connections 2024-01-31T05:22:24.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.820+0000 7fd0eff42640 1 -- 172.21.15.45:0/2211880192 wait complete. 2024-01-31T05:22:24.829 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:24 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/377101544' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-01-31T05:22:24.830 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:22:24.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:24 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/377101544' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-01-31T05:22:24.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 -- 172.21.15.45:0/1161744199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0100de0 msgr2=0x7fabf0101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 --2- 172.21.15.45:0/1161744199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0100de0 0x7fabf0101260 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009900 tx=0x7fabe002bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 -- 172.21.15.45:0/1161744199 shutdown_connections 2024-01-31T05:22:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 --2- 172.21.15.45:0/1161744199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0100de0 0x7fabf0101260 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 --2- 172.21.15.45:0/1161744199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabf01017a0 0x7fabf0108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 --2- 172.21.15.45:0/1161744199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf00ffbf0 0x7fabf0100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 -- 172.21.15.45:0/1161744199 >> 172.21.15.45:0/1161744199 conn(0x7fabf00fb160 msgr2=0x7fabf00fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 -- 172.21.15.45:0/1161744199 shutdown_connections 2024-01-31T05:22:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 -- 172.21.15.45:0/1161744199 wait complete. 2024-01-31T05:22:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 Processor -- start 2024-01-31T05:22:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 -- start start 2024-01-31T05:22:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabf00ffbf0 0x7fabf0195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.936+0000 7fabf79c6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf0100de0 0x7fabf01960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf79c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf01017a0 0x7fabf019d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf79c6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf019d690 con 0x7fabf01017a0 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf79c6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf019d800 con 0x7fabf0100de0 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf79c6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf019d9c0 con 0x7fabf00ffbf0 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf5f63640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf01017a0 0x7fabf019d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf5f63640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf01017a0 0x7fabf019d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48704/0 (socket says 172.21.15.45:48704) 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf5f63640 1 -- 172.21.15.45:0/606438989 learned_addr learned my addr 172.21.15.45:0/606438989 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf5762640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabf00ffbf0 0x7fabf0195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf4f61640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf0100de0 0x7fabf01960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf5f63640 1 -- 172.21.15.45:0/606438989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabf00ffbf0 msgr2=0x7fabf0195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf5f63640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabf00ffbf0 0x7fabf0195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf5f63640 1 -- 172.21.15.45:0/606438989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf0100de0 msgr2=0x7fabf01960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf5f63640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf0100de0 0x7fabf01960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf5f63640 1 -- 172.21.15.45:0/606438989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe0009560 con 0x7fabf01017a0 2024-01-31T05:22:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf4f61640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf0100de0 0x7fabf01960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf5f63640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf01017a0 0x7fabf019d150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fabe800eaa0 tx=0x7fabe800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabe67fc640 1 -- 172.21.15.45:0/606438989 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe800cdb0 con 0x7fabf01017a0 2024-01-31T05:22:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf79c6640 1 -- 172.21.15.45:0/606438989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf019dca0 con 0x7fabf01017a0 2024-01-31T05:22:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabe67fc640 1 -- 172.21.15.45:0/606438989 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabe8005020 con 0x7fabf01017a0 2024-01-31T05:22:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabe67fc640 1 -- 172.21.15.45:0/606438989 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80116d0 con 0x7fabf01017a0 2024-01-31T05:22:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.937+0000 7fabf79c6640 1 -- 172.21.15.45:0/606438989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf019e1d0 con 0x7fabf01017a0 2024-01-31T05:22:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.938+0000 7fabf79c6640 1 -- 172.21.15.45:0/606438989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf0106fd0 con 0x7fabf01017a0 2024-01-31T05:22:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.939+0000 7fabe67fc640 1 -- 172.21.15.45:0/606438989 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fabe8005420 con 0x7fabf01017a0 2024-01-31T05:22:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.940+0000 7fabe67fc640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fabdc0a12e0 0x7fabdc0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.940+0000 7fabf5762640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fabdc0a12e0 0x7fabdc0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.940+0000 7fabe67fc640 1 -- 172.21.15.45:0/606438989 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fabe80ccca0 con 0x7fabf01017a0 2024-01-31T05:22:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.940+0000 7fabf5762640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fabdc0a12e0 0x7fabdc0a37a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fabf0100c40 tx=0x7fabec00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:24.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:24.945+0000 7fabe67fc640 1 -- 172.21.15.45:0/606438989 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fabe8096150 con 0x7fabf01017a0 2024-01-31T05:22:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:24 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/377101544' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-01-31T05:22:25.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.123+0000 7fabf79c6640 1 -- 172.21.15.45:0/606438989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fabf002cb70 con 0x7fabf01017a0 2024-01-31T05:22:25.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.123+0000 7fabe67fc640 1 -- 172.21.15.45:0/606438989 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fabe801f0a0 con 0x7fabf01017a0 2024-01-31T05:22:25.122 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-01-31T05:22:25.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.125+0000 7fabf79c6640 1 -- 172.21.15.45:0/606438989 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fabdc0a12e0 msgr2=0x7fabdc0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.125+0000 7fabf79c6640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fabdc0a12e0 0x7fabdc0a37a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fabf0100c40 tx=0x7fabec00a2d0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.125+0000 7fabf79c6640 1 -- 172.21.15.45:0/606438989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf01017a0 msgr2=0x7fabf019d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.125+0000 7fabf79c6640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf01017a0 0x7fabf019d150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fabe800eaa0 tx=0x7fabe800ef60 comp rx=0 tx=0).stop 2024-01-31T05:22:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.125+0000 7fabf79c6640 1 -- 172.21.15.45:0/606438989 shutdown_connections 2024-01-31T05:22:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.125+0000 7fabf79c6640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fabdc0a12e0 0x7fabdc0a37a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.125+0000 7fabf79c6640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf01017a0 0x7fabf019d150 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.125+0000 7fabf79c6640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf0100de0 0x7fabf01960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.125+0000 7fabf79c6640 1 --2- 172.21.15.45:0/606438989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabf00ffbf0 0x7fabf0195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.125+0000 7fabf79c6640 1 -- 172.21.15.45:0/606438989 >> 172.21.15.45:0/606438989 conn(0x7fabf00fb160 msgr2=0x7fabf01041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.125+0000 7fabf79c6640 1 -- 172.21.15.45:0/606438989 shutdown_connections 2024-01-31T05:22:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.125+0000 7fabf79c6640 1 -- 172.21.15.45:0/606438989 wait complete. 2024-01-31T05:22:25.135 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:22:25.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.242+0000 7f5f811d9640 1 -- 172.21.15.45:0/1979691812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f7c106a40 msgr2=0x7f5f7c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.242+0000 7f5f811d9640 1 --2- 172.21.15.45:0/1979691812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f7c106a40 0x7f5f7c108e30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009ca0 tx=0x7f5f6c02c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.242+0000 7f5f811d9640 1 -- 172.21.15.45:0/1979691812 shutdown_connections 2024-01-31T05:22:25.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.242+0000 7f5f811d9640 1 --2- 172.21.15.45:0/1979691812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f7c106a40 0x7f5f7c108e30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.242+0000 7f5f811d9640 1 --2- 172.21.15.45:0/1979691812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f7c104110 0x7f5f7c106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.242+0000 7f5f811d9640 1 --2- 172.21.15.45:0/1979691812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7c1017e0 0x7f5f7c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.242+0000 7f5f811d9640 1 -- 172.21.15.45:0/1979691812 >> 172.21.15.45:0/1979691812 conn(0x7f5f7c0fb190 msgr2=0x7f5f7c0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.242+0000 7f5f811d9640 1 -- 172.21.15.45:0/1979691812 shutdown_connections 2024-01-31T05:22:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.242+0000 7f5f811d9640 1 -- 172.21.15.45:0/1979691812 wait complete. 2024-01-31T05:22:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.242+0000 7f5f811d9640 1 Processor -- start 2024-01-31T05:22:25.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f811d9640 1 -- start start 2024-01-31T05:22:25.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f811d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f7c1017e0 0x7f5f7c196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:25.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f811d9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f7c104110 0x7f5f7c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:25.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f811d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7c106a40 0x7f5f7c19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:25.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f811d9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c19e810 con 0x7f5f7c1017e0 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f811d9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c19e980 con 0x7f5f7c104110 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f811d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c19eb40 con 0x7f5f7c106a40 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f7ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f7c1017e0 0x7f5f7c196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f7ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f7c1017e0 0x7f5f7c196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48710/0 (socket says 172.21.15.45:48710) 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f7ad9d640 1 -- 172.21.15.45:0/2022754587 learned_addr learned my addr 172.21.15.45:0/2022754587 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f7a59c640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f7c104110 0x7f5f7c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f7b59e640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7c106a40 0x7f5f7c19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f7ad9d640 1 -- 172.21.15.45:0/2022754587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7c106a40 msgr2=0x7f5f7c19e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f7ad9d640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7c106a40 0x7f5f7c19e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f7ad9d640 1 -- 172.21.15.45:0/2022754587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f7c104110 msgr2=0x7f5f7c197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f7ad9d640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f7c104110 0x7f5f7c197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f7ad9d640 1 -- 172.21.15.45:0/2022754587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c009900 con 0x7f5f7c1017e0 2024-01-31T05:22:25.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f7a59c640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f7c104110 0x7f5f7c197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:25.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f7ad9d640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f7c1017e0 0x7f5f7c196d30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400d770 tx=0x7f5f7400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:25.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f73fff640 1 -- 172.21.15.45:0/2022754587 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74009c30 con 0x7f5f7c1017e0 2024-01-31T05:22:25.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f73fff640 1 -- 172.21.15.45:0/2022754587 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f7400de80 con 0x7f5f7c1017e0 2024-01-31T05:22:25.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.243+0000 7f5f811d9640 1 -- 172.21.15.45:0/2022754587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7c19ee20 con 0x7f5f7c1017e0 2024-01-31T05:22:25.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.244+0000 7f5f73fff640 1 -- 172.21.15.45:0/2022754587 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74004de0 con 0x7f5f7c1017e0 2024-01-31T05:22:25.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.244+0000 7f5f811d9640 1 -- 172.21.15.45:0/2022754587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7c19f1f0 con 0x7f5f7c1017e0 2024-01-31T05:22:25.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.244+0000 7f5f811d9640 1 -- 172.21.15.45:0/2022754587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7c0fcc80 con 0x7f5f7c1017e0 2024-01-31T05:22:25.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.246+0000 7f5f73fff640 1 -- 172.21.15.45:0/2022754587 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f5f74004f40 con 0x7f5f7c1017e0 2024-01-31T05:22:25.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.246+0000 7f5f73fff640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5f680a12e0 0x7f5f680a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:25.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.247+0000 7f5f7a59c640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5f680a12e0 0x7f5f680a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:25.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.247+0000 7f5f73fff640 1 -- 172.21.15.45:0/2022754587 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5f740cca80 con 0x7f5f7c1017e0 2024-01-31T05:22:25.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.247+0000 7f5f7a59c640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5f680a12e0 0x7f5f680a37a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c069dc0 tx=0x7f5f64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:25.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.250+0000 7f5f73fff640 1 -- 172.21.15.45:0/2022754587 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5f74095ea0 con 0x7f5f7c1017e0 2024-01-31T05:22:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.429+0000 7f5f811d9640 1 -- 172.21.15.45:0/2022754587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5f7c10c840 con 0x7f5f7c1017e0 2024-01-31T05:22:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.429+0000 7f5f73fff640 1 -- 172.21.15.45:0/2022754587 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f740170a0 con 0x7f5f7c1017e0 2024-01-31T05:22:25.429 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-01-31T05:22:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.431+0000 7f5f811d9640 1 -- 172.21.15.45:0/2022754587 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5f680a12e0 msgr2=0x7f5f680a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.431+0000 7f5f811d9640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5f680a12e0 0x7f5f680a37a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c069dc0 tx=0x7f5f64005f90 comp rx=0 tx=0).stop 2024-01-31T05:22:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.432+0000 7f5f811d9640 1 -- 172.21.15.45:0/2022754587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f7c1017e0 msgr2=0x7f5f7c196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.432+0000 7f5f811d9640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f7c1017e0 0x7f5f7c196d30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400d770 tx=0x7f5f7400dc30 comp rx=0 tx=0).stop 2024-01-31T05:22:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.432+0000 7f5f811d9640 1 -- 172.21.15.45:0/2022754587 shutdown_connections 2024-01-31T05:22:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.432+0000 7f5f811d9640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f5f680a12e0 0x7f5f680a37a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.432+0000 7f5f811d9640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7c106a40 0x7f5f7c19e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.432+0000 7f5f811d9640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f7c104110 0x7f5f7c197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.432+0000 7f5f811d9640 1 --2- 172.21.15.45:0/2022754587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f7c1017e0 0x7f5f7c196d30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.432+0000 7f5f811d9640 1 -- 172.21.15.45:0/2022754587 >> 172.21.15.45:0/2022754587 conn(0x7f5f7c0fb190 msgr2=0x7f5f7c102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.432+0000 7f5f811d9640 1 -- 172.21.15.45:0/2022754587 shutdown_connections 2024-01-31T05:22:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.432+0000 7f5f811d9640 1 -- 172.21.15.45:0/2022754587 wait complete. 2024-01-31T05:22:25.442 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:22:25.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.546+0000 7f8e639fb640 1 -- 172.21.15.45:0/3396382480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5c1040b0 msgr2=0x7f8e5c1064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.546+0000 7f8e639fb640 1 --2- 172.21.15.45:0/3396382480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5c1040b0 0x7f8e5c1064a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009900 tx=0x7f8e4c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:25.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.546+0000 7f8e639fb640 1 -- 172.21.15.45:0/3396382480 shutdown_connections 2024-01-31T05:22:25.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.546+0000 7f8e639fb640 1 --2- 172.21.15.45:0/3396382480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5c1040b0 0x7f8e5c1064a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.546+0000 7f8e639fb640 1 --2- 172.21.15.45:0/3396382480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e5c1069e0 0x7f8e5c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.546+0000 7f8e639fb640 1 --2- 172.21.15.45:0/3396382480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e5c101780 0x7f8e5c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.546+0000 7f8e639fb640 1 -- 172.21.15.45:0/3396382480 >> 172.21.15.45:0/3396382480 conn(0x7f8e5c0fb170 msgr2=0x7f8e5c0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:25.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.546+0000 7f8e639fb640 1 -- 172.21.15.45:0/3396382480 shutdown_connections 2024-01-31T05:22:25.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.546+0000 7f8e639fb640 1 -- 172.21.15.45:0/3396382480 wait complete. 2024-01-31T05:22:25.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e639fb640 1 Processor -- start 2024-01-31T05:22:25.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e639fb640 1 -- start start 2024-01-31T05:22:25.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e639fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5c101780 0x7f8e5c196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:25.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e639fb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e5c1040b0 0x7f8e5c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:25.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e639fb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e5c1069e0 0x7f8e5c19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:25.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e639fb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c19e7f0 con 0x7f8e5c101780 2024-01-31T05:22:25.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e639fb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c19e960 con 0x7f8e5c1040b0 2024-01-31T05:22:25.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e639fb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c19eb20 con 0x7f8e5c1069e0 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e61797640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5c101780 0x7f8e5c196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e61797640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5c101780 0x7f8e5c196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48728/0 (socket says 172.21.15.45:48728) 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e61797640 1 -- 172.21.15.45:0/2213415978 learned_addr learned my addr 172.21.15.45:0/2213415978 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e60f96640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e5c1040b0 0x7f8e5c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e61797640 1 -- 172.21.15.45:0/2213415978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e5c1069e0 msgr2=0x7f8e5c19e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e61797640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e5c1069e0 0x7f8e5c19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.547+0000 7f8e61797640 1 -- 172.21.15.45:0/2213415978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e5c1040b0 msgr2=0x7f8e5c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.548+0000 7f8e61797640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e5c1040b0 0x7f8e5c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.548+0000 7f8e61797640 1 -- 172.21.15.45:0/2213415978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c009560 con 0x7f8e5c101780 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.548+0000 7f8e60f96640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e5c1040b0 0x7f8e5c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.548+0000 7f8e61797640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5c101780 0x7f8e5c196d10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400d770 tx=0x7f8e5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.548+0000 7f8e527fc640 1 -- 172.21.15.45:0/2213415978 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54009c30 con 0x7f8e5c101780 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.548+0000 7f8e527fc640 1 -- 172.21.15.45:0/2213415978 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e5400de80 con 0x7f8e5c101780 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.548+0000 7f8e527fc640 1 -- 172.21.15.45:0/2213415978 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54004de0 con 0x7f8e5c101780 2024-01-31T05:22:25.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.548+0000 7f8e639fb640 1 -- 172.21.15.45:0/2213415978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c19ee00 con 0x7f8e5c101780 2024-01-31T05:22:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.548+0000 7f8e639fb640 1 -- 172.21.15.45:0/2213415978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c19f230 con 0x7f8e5c101780 2024-01-31T05:22:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.549+0000 7f8e639fb640 1 -- 172.21.15.45:0/2213415978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c0fcc60 con 0x7f8e5c101780 2024-01-31T05:22:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.550+0000 7f8e527fc640 1 -- 172.21.15.45:0/2213415978 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f8e54004f40 con 0x7f8e5c101780 2024-01-31T05:22:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.550+0000 7f8e527fc640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8e480a12e0 0x7f8e480a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.550+0000 7f8e60f96640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8e480a12e0 0x7f8e480a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.550+0000 7f8e527fc640 1 -- 172.21.15.45:0/2213415978 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8e540cc720 con 0x7f8e5c101780 2024-01-31T05:22:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.551+0000 7f8e60f96640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8e480a12e0 0x7f8e480a37a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c02c6b0 tx=0x7f8e4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.555+0000 7f8e527fc640 1 -- 172.21.15.45:0/2213415978 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8e54095ae0 con 0x7f8e5c101780 2024-01-31T05:22:25.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.773+0000 7f8e639fb640 1 -- 172.21.15.45:0/2213415978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e5c10c5d0 con 0x7f8e5c101780 2024-01-31T05:22:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.774+0000 7f8e527fc640 1 -- 172.21.15.45:0/2213415978 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f8e54097a10 con 0x7f8e5c101780 2024-01-31T05:22:25.773 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:25.773 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1801709710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1801709710},{"type":"v1","addr":"172.21.15.45:6839","nonce":1801709710}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/1148586810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1148586810},{"type":"v1","addr":"172.21.15.45:6841","nonce":1148586810}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-31T05:20:40.057668+0000","modified":"2024-01-31T05:20:44.476398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3782584640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3782584640},{"type":"v1","addr":"172.21.15.156:6837","nonce":3782584640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/3821255404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3821255404},{"type":"v1","addr":"172.21.15.186:6839","nonce":3821255404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/3137398384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3137398384},{"type":"v1","addr":"172.21.15.156:6839","nonce":3137398384}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:22:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 20 2024-01-31T05:22:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.775+0000 7f8e639fb640 1 -- 172.21.15.45:0/2213415978 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8e480a12e0 msgr2=0x7f8e480a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.775+0000 7f8e639fb640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8e480a12e0 0x7f8e480a37a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c02c6b0 tx=0x7f8e4c005fd0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.775+0000 7f8e639fb640 1 -- 172.21.15.45:0/2213415978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5c101780 msgr2=0x7f8e5c196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.775+0000 7f8e639fb640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5c101780 0x7f8e5c196d10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400d770 tx=0x7f8e5400dc30 comp rx=0 tx=0).stop 2024-01-31T05:22:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.775+0000 7f8e639fb640 1 -- 172.21.15.45:0/2213415978 shutdown_connections 2024-01-31T05:22:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.775+0000 7f8e639fb640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f8e480a12e0 0x7f8e480a37a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.775+0000 7f8e639fb640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e5c1069e0 0x7f8e5c19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.775+0000 7f8e639fb640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e5c1040b0 0x7f8e5c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.775+0000 7f8e639fb640 1 --2- 172.21.15.45:0/2213415978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5c101780 0x7f8e5c196d10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.776+0000 7f8e639fb640 1 -- 172.21.15.45:0/2213415978 >> 172.21.15.45:0/2213415978 conn(0x7f8e5c0fb170 msgr2=0x7f8e5c102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.776+0000 7f8e639fb640 1 -- 172.21.15.45:0/2213415978 shutdown_connections 2024-01-31T05:22:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.776+0000 7f8e639fb640 1 -- 172.21.15.45:0/2213415978 wait complete. 2024-01-31T05:22:25.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:25 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3452279340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:25.785 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:25 smithi045 ceph-mon[31585]: pgmap v337: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:22:25.785 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:25 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2211880192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:22:25.785 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:25 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/606438989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:22:25.785 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:25 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2022754587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:22:25.786 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:22:25.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:25 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3452279340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:25.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:25 smithi186 ceph-mon[35659]: pgmap v337: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:22:25.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:25 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2211880192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:22:25.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:25 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/606438989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:22:25.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:25 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2022754587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:22:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.889+0000 7f857a2b1640 1 -- 172.21.15.45:0/1252927574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85741040c0 msgr2=0x7f85741064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.889+0000 7f857a2b1640 1 --2- 172.21.15.45:0/1252927574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85741040c0 0x7f85741064b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f855c009900 tx=0x7f855c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.889+0000 7f857a2b1640 1 -- 172.21.15.45:0/1252927574 shutdown_connections 2024-01-31T05:22:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.889+0000 7f857a2b1640 1 --2- 172.21.15.45:0/1252927574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85741040c0 0x7f85741064b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.889+0000 7f857a2b1640 1 --2- 172.21.15.45:0/1252927574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85741069f0 0x7f8574108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.889+0000 7f857a2b1640 1 --2- 172.21.15.45:0/1252927574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8574101790 0x7f8574103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.889+0000 7f857a2b1640 1 -- 172.21.15.45:0/1252927574 >> 172.21.15.45:0/1252927574 conn(0x7f85740fb180 msgr2=0x7f85740fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.889+0000 7f857a2b1640 1 -- 172.21.15.45:0/1252927574 shutdown_connections 2024-01-31T05:22:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.889+0000 7f857a2b1640 1 -- 172.21.15.45:0/1252927574 wait complete. 2024-01-31T05:22:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.889+0000 7f857a2b1640 1 Processor -- start 2024-01-31T05:22:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.889+0000 7f857a2b1640 1 -- start start 2024-01-31T05:22:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f857a2b1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8574101790 0x7f8574198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f857a2b1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85741040c0 0x7f8574199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f857a2b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85741069f0 0x7f85741a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f857a2b1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85741a09e0 con 0x7f85741040c0 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f857a2b1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85741a0b50 con 0x7f85741069f0 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f857a2b1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85741a0d10 con 0x7f8574101790 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f85737fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85741040c0 0x7f8574199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f85737fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85741040c0 0x7f8574199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48746/0 (socket says 172.21.15.45:48746) 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f85737fe640 1 -- 172.21.15.45:0/942452059 learned_addr learned my addr 172.21.15.45:0/942452059 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f8573fff640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8574101790 0x7f8574198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f85737fe640 1 -- 172.21.15.45:0/942452059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8574101790 msgr2=0x7f8574198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f85737fe640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8574101790 0x7f8574198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f85737fe640 1 -- 172.21.15.45:0/942452059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85741069f0 msgr2=0x7f85741a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f85737fe640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85741069f0 0x7f85741a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f85737fe640 1 -- 172.21.15.45:0/942452059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c009560 con 0x7f85741040c0 2024-01-31T05:22:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f85737fe640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85741040c0 0x7f8574199430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f855c009230 tx=0x7f855c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f85717fa640 1 -- 172.21.15.45:0/942452059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c0044c0 con 0x7f85741040c0 2024-01-31T05:22:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f85717fa640 1 -- 172.21.15.45:0/942452059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f855c0054c0 con 0x7f85741040c0 2024-01-31T05:22:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f857a2b1640 1 -- 172.21.15.45:0/942452059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85741a0f90 con 0x7f85741040c0 2024-01-31T05:22:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f85717fa640 1 -- 172.21.15.45:0/942452059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c03e450 con 0x7f85741040c0 2024-01-31T05:22:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.890+0000 7f857a2b1640 1 -- 172.21.15.45:0/942452059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85741a1490 con 0x7f85741040c0 2024-01-31T05:22:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.891+0000 7f857a2b1640 1 -- 172.21.15.45:0/942452059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8574108540 con 0x7f85741040c0 2024-01-31T05:22:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.893+0000 7f85717fa640 1 -- 172.21.15.45:0/942452059 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f855c0036f0 con 0x7f85741040c0 2024-01-31T05:22:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.893+0000 7f85717fa640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f85600a12e0 0x7f85600a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.893+0000 7f8573fff640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f85600a12e0 0x7f85600a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.893+0000 7f85717fa640 1 -- 172.21.15.45:0/942452059 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f855c0ef9d0 con 0x7f85741040c0 2024-01-31T05:22:25.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.894+0000 7f8573fff640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f85600a12e0 0x7f85600a37a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f856400ba40 tx=0x7f8564005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:25.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:25.897+0000 7f85717fa640 1 -- 172.21.15.45:0/942452059 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f855c0b8e80 con 0x7f85741040c0 2024-01-31T05:22:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:25 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3452279340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:25 smithi156 ceph-mon[35720]: pgmap v337: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-31T05:22:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:25 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2211880192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:22:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:25 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/606438989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:22:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:25 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2022754587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:22:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.077+0000 7f857a2b1640 1 -- 172.21.15.45:0/942452059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f857402cb50 con 0x7f85741040c0 2024-01-31T05:22:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.078+0000 7f85717fa640 1 -- 172.21.15.45:0/942452059 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22520 (secure 0 0 0) 0x7f855c0a7020 con 0x7f85741040c0 2024-01-31T05:22:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:26.078 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":88,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:21:47.682334+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.080+0000 7f857a2b1640 1 -- 172.21.15.45:0/942452059 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f85600a12e0 msgr2=0x7f85600a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.080+0000 7f857a2b1640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f85600a12e0 0x7f85600a37a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f856400ba40 tx=0x7f8564005cc0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.080+0000 7f857a2b1640 1 -- 172.21.15.45:0/942452059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85741040c0 msgr2=0x7f8574199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.080+0000 7f857a2b1640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85741040c0 0x7f8574199430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f855c009230 tx=0x7f855c0040c0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.081+0000 7f857a2b1640 1 -- 172.21.15.45:0/942452059 shutdown_connections 2024-01-31T05:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.081+0000 7f857a2b1640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f85600a12e0 0x7f85600a37a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.081+0000 7f857a2b1640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85741040c0 0x7f8574199430 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.081+0000 7f857a2b1640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85741069f0 0x7f85741a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.081+0000 7f857a2b1640 1 --2- 172.21.15.45:0/942452059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8574101790 0x7f8574198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.081+0000 7f857a2b1640 1 -- 172.21.15.45:0/942452059 >> 172.21.15.45:0/942452059 conn(0x7f85740fb180 msgr2=0x7f8574100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.081+0000 7f857a2b1640 1 -- 172.21.15.45:0/942452059 shutdown_connections 2024-01-31T05:22:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.081+0000 7f857a2b1640 1 -- 172.21.15.45:0/942452059 wait complete. 2024-01-31T05:22:26.091 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:16:10.856581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:20:38.347599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:20:39.270522+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:22:26.091 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:22:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.197+0000 7f2e71c4f640 1 -- 172.21.15.45:0/1453414168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e6c100de0 msgr2=0x7f2e6c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.197+0000 7f2e71c4f640 1 --2- 172.21.15.45:0/1453414168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e6c100de0 0x7f2e6c101260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009900 tx=0x7f2e6002bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.197+0000 7f2e71c4f640 1 -- 172.21.15.45:0/1453414168 shutdown_connections 2024-01-31T05:22:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.197+0000 7f2e71c4f640 1 --2- 172.21.15.45:0/1453414168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e6c100de0 0x7f2e6c101260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.197+0000 7f2e71c4f640 1 --2- 172.21.15.45:0/1453414168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e6c1017a0 0x7f2e6c108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.197+0000 7f2e71c4f640 1 --2- 172.21.15.45:0/1453414168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e6c0ffbf0 0x7f2e6c100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.197+0000 7f2e71c4f640 1 -- 172.21.15.45:0/1453414168 >> 172.21.15.45:0/1453414168 conn(0x7f2e6c0fb180 msgr2=0x7f2e6c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.197+0000 7f2e71c4f640 1 -- 172.21.15.45:0/1453414168 shutdown_connections 2024-01-31T05:22:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.197+0000 7f2e71c4f640 1 -- 172.21.15.45:0/1453414168 wait complete. 2024-01-31T05:22:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.197+0000 7f2e71c4f640 1 Processor -- start 2024-01-31T05:22:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e71c4f640 1 -- start start 2024-01-31T05:22:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e71c4f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e6c0ffbf0 0x7f2e6c0689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e71c4f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e6c100de0 0x7f2e6c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e71c4f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e6c1017a0 0x7f2e6c19e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e71c4f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c19e7f0 con 0x7f2e6c1017a0 2024-01-31T05:22:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e71c4f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c19e960 con 0x7f2e6c0ffbf0 2024-01-31T05:22:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e71c4f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c19eb20 con 0x7f2e6c100de0 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e6bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e6c1017a0 0x7f2e6c19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e6bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e6c1017a0 0x7f2e6c19e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48770/0 (socket says 172.21.15.45:48770) 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e6bfff640 1 -- 172.21.15.45:0/3065855429 learned_addr learned my addr 172.21.15.45:0/3065855429 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e6b7fe640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e6c0ffbf0 0x7f2e6c0689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e6affd640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e6c100de0 0x7f2e6c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e6bfff640 1 -- 172.21.15.45:0/3065855429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e6c100de0 msgr2=0x7f2e6c19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e6bfff640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e6c100de0 0x7f2e6c19ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e6bfff640 1 -- 172.21.15.45:0/3065855429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e6c0ffbf0 msgr2=0x7f2e6c0689e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e6bfff640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e6c0ffbf0 0x7f2e6c0689e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e6bfff640 1 -- 172.21.15.45:0/3065855429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e60009560 con 0x7f2e6c1017a0 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e6affd640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e6c100de0 0x7f2e6c19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e6bfff640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e6c1017a0 0x7f2e6c19e060 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400eb10 tx=0x7f2e6400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.198+0000 7f2e68ff9640 1 -- 172.21.15.45:0/3065855429 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e64004b20 con 0x7f2e6c1017a0 2024-01-31T05:22:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.199+0000 7f2e71c4f640 1 -- 172.21.15.45:0/3065855429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6c19ee00 con 0x7f2e6c1017a0 2024-01-31T05:22:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.199+0000 7f2e68ff9640 1 -- 172.21.15.45:0/3065855429 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e64004c80 con 0x7f2e6c1017a0 2024-01-31T05:22:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.199+0000 7f2e68ff9640 1 -- 172.21.15.45:0/3065855429 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e64010690 con 0x7f2e6c1017a0 2024-01-31T05:22:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.199+0000 7f2e71c4f640 1 -- 172.21.15.45:0/3065855429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6c19f290 con 0x7f2e6c1017a0 2024-01-31T05:22:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.199+0000 7f2e71c4f640 1 -- 172.21.15.45:0/3065855429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6c106fb0 con 0x7f2e6c1017a0 2024-01-31T05:22:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.201+0000 7f2e68ff9640 1 -- 172.21.15.45:0/3065855429 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f2e64004fd0 con 0x7f2e6c1017a0 2024-01-31T05:22:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.201+0000 7f2e68ff9640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2e540a12e0 0x7f2e540a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.201+0000 7f2e6b7fe640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2e540a12e0 0x7f2e540a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.201+0000 7f2e68ff9640 1 -- 172.21.15.45:0/3065855429 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2e64014070 con 0x7f2e6c1017a0 2024-01-31T05:22:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.202+0000 7f2e6b7fe640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2e540a12e0 0x7f2e540a37a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c005fd0 tx=0x7f2e5c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.205+0000 7f2e68ff9640 1 -- 172.21.15.45:0/3065855429 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f2e640967b0 con 0x7f2e6c1017a0 2024-01-31T05:22:26.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.383+0000 7f2e71c4f640 1 -- 172.21.15.45:0/3065855429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e6c10b700 con 0x7f2e6c1017a0 2024-01-31T05:22:26.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.384+0000 7f2e68ff9640 1 -- 172.21.15.45:0/3065855429 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e64099dd0 con 0x7f2e6c1017a0 2024-01-31T05:22:26.383 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-01-31T05:22:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.386+0000 7f2e71c4f640 1 -- 172.21.15.45:0/3065855429 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2e540a12e0 msgr2=0x7f2e540a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.386+0000 7f2e71c4f640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2e540a12e0 0x7f2e540a37a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c005fd0 tx=0x7f2e5c00b390 comp rx=0 tx=0).stop 2024-01-31T05:22:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.386+0000 7f2e71c4f640 1 -- 172.21.15.45:0/3065855429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e6c1017a0 msgr2=0x7f2e6c19e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.386+0000 7f2e71c4f640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e6c1017a0 0x7f2e6c19e060 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400eb10 tx=0x7f2e6400c2d0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.386+0000 7f2e71c4f640 1 -- 172.21.15.45:0/3065855429 shutdown_connections 2024-01-31T05:22:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.386+0000 7f2e71c4f640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f2e540a12e0 0x7f2e540a37a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.386+0000 7f2e71c4f640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e6c1017a0 0x7f2e6c19e060 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.386+0000 7f2e71c4f640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e6c100de0 0x7f2e6c19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.386+0000 7f2e71c4f640 1 --2- 172.21.15.45:0/3065855429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e6c0ffbf0 0x7f2e6c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.386+0000 7f2e71c4f640 1 -- 172.21.15.45:0/3065855429 >> 172.21.15.45:0/3065855429 conn(0x7f2e6c0fb180 msgr2=0x7f2e6c104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.386+0000 7f2e71c4f640 1 -- 172.21.15.45:0/3065855429 shutdown_connections 2024-01-31T05:22:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.386+0000 7f2e71c4f640 1 -- 172.21.15.45:0/3065855429 wait complete. 2024-01-31T05:22:26.396 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:22:26.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.502+0000 7faab84b1640 1 -- 172.21.15.45:0/4216910593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab01017a0 msgr2=0x7faab0108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.502+0000 7faab84b1640 1 --2- 172.21.15.45:0/4216910593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab01017a0 0x7faab0108030 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009ca0 tx=0x7faaa002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.502+0000 7faab84b1640 1 -- 172.21.15.45:0/4216910593 shutdown_connections 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.502+0000 7faab84b1640 1 --2- 172.21.15.45:0/4216910593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab01017a0 0x7faab0108030 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.502+0000 7faab84b1640 1 --2- 172.21.15.45:0/4216910593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faab0100de0 0x7faab0101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.502+0000 7faab84b1640 1 --2- 172.21.15.45:0/4216910593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faab00ffbf0 0x7faab0100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.502+0000 7faab84b1640 1 -- 172.21.15.45:0/4216910593 >> 172.21.15.45:0/4216910593 conn(0x7faab00fb180 msgr2=0x7faab00fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.502+0000 7faab84b1640 1 -- 172.21.15.45:0/4216910593 shutdown_connections 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.503+0000 7faab84b1640 1 -- 172.21.15.45:0/4216910593 wait complete. 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.503+0000 7faab84b1640 1 Processor -- start 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.503+0000 7faab84b1640 1 -- start start 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.503+0000 7faab84b1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab00ffbf0 0x7faab0196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.503+0000 7faab84b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faab0100de0 0x7faab01971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.503+0000 7faab84b1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faab01017a0 0x7faab019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.503+0000 7faab84b1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab019e7a0 con 0x7faab00ffbf0 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.503+0000 7faab84b1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab019e910 con 0x7faab0100de0 2024-01-31T05:22:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.503+0000 7faab84b1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab019ead0 con 0x7faab01017a0 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.503+0000 7faab624d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab00ffbf0 0x7faab0196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.503+0000 7faab624d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab00ffbf0 0x7faab0196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48802/0 (socket says 172.21.15.45:48802) 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.503+0000 7faab624d640 1 -- 172.21.15.45:0/52823088 learned_addr learned my addr 172.21.15.45:0/52823088 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faab6a4e640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faab01017a0 0x7faab019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faab5a4c640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faab0100de0 0x7faab01971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faab624d640 1 -- 172.21.15.45:0/52823088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faab01017a0 msgr2=0x7faab019e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faab624d640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faab01017a0 0x7faab019e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faab624d640 1 -- 172.21.15.45:0/52823088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faab0100de0 msgr2=0x7faab01971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faab624d640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faab0100de0 0x7faab01971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faab624d640 1 -- 172.21.15.45:0/52823088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa0009900 con 0x7faab00ffbf0 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faab624d640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab00ffbf0 0x7faab0196c90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faaa800d950 tx=0x7faaa800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faaaf7fe640 1 -- 172.21.15.45:0/52823088 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8009c30 con 0x7faab00ffbf0 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faaaf7fe640 1 -- 172.21.15.45:0/52823088 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faaa8009d90 con 0x7faab00ffbf0 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faab84b1640 1 -- 172.21.15.45:0/52823088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab019edb0 con 0x7faab00ffbf0 2024-01-31T05:22:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faaaf7fe640 1 -- 172.21.15.45:0/52823088 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa800f6b0 con 0x7faab00ffbf0 2024-01-31T05:22:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.504+0000 7faab84b1640 1 -- 172.21.15.45:0/52823088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab019f200 con 0x7faab00ffbf0 2024-01-31T05:22:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.505+0000 7faab84b1640 1 -- 172.21.15.45:0/52823088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab010b4b0 con 0x7faab00ffbf0 2024-01-31T05:22:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.506+0000 7faaaf7fe640 1 -- 172.21.15.45:0/52823088 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7faaa80109b0 con 0x7faab00ffbf0 2024-01-31T05:22:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.506+0000 7faaaf7fe640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa980a1360 0x7faa980a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.507+0000 7faab5a4c640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa980a1360 0x7faa980a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:26.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.507+0000 7faaaf7fe640 1 -- 172.21.15.45:0/52823088 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7faaa80cc940 con 0x7faab00ffbf0 2024-01-31T05:22:26.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.507+0000 7faab5a4c640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa980a1360 0x7faa980a3820 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faab0198190 tx=0x7faaa4005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:26.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.511+0000 7faaaf7fe640 1 -- 172.21.15.45:0/52823088 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7faaa8095d90 con 0x7faab00ffbf0 2024-01-31T05:22:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.689+0000 7faab84b1640 1 -- 172.21.15.45:0/52823088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faab019f420 con 0x7faab00ffbf0 2024-01-31T05:22:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.690+0000 7faaaf7fe640 1 -- 172.21.15.45:0/52823088 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7faaa80993b0 con 0x7faab00ffbf0 2024-01-31T05:22:26.689 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-01-31T05:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.692+0000 7faab84b1640 1 -- 172.21.15.45:0/52823088 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa980a1360 msgr2=0x7faa980a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.692+0000 7faab84b1640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa980a1360 0x7faa980a3820 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faab0198190 tx=0x7faaa4005c60 comp rx=0 tx=0).stop 2024-01-31T05:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.692+0000 7faab84b1640 1 -- 172.21.15.45:0/52823088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab00ffbf0 msgr2=0x7faab0196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.692+0000 7faab84b1640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab00ffbf0 0x7faab0196c90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faaa800d950 tx=0x7faaa800de10 comp rx=0 tx=0).stop 2024-01-31T05:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.692+0000 7faab84b1640 1 -- 172.21.15.45:0/52823088 shutdown_connections 2024-01-31T05:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.692+0000 7faab84b1640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa980a1360 0x7faa980a3820 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.692+0000 7faab84b1640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faab01017a0 0x7faab019e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.692+0000 7faab84b1640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faab0100de0 0x7faab01971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.692+0000 7faab84b1640 1 --2- 172.21.15.45:0/52823088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab00ffbf0 0x7faab0196c90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.692+0000 7faab84b1640 1 -- 172.21.15.45:0/52823088 >> 172.21.15.45:0/52823088 conn(0x7faab00fb180 msgr2=0x7faab0067e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:26.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.692+0000 7faab84b1640 1 -- 172.21.15.45:0/52823088 shutdown_connections 2024-01-31T05:22:26.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.692+0000 7faab84b1640 1 -- 172.21.15.45:0/52823088 wait complete. 2024-01-31T05:22:26.701 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:26 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2213415978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:26.701 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:26 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/942452059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:26.701 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:26 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3065855429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:22:26.702 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:22:26.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:26 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2213415978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:26.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:26 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/942452059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:26.743 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:26 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3065855429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:22:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.808+0000 7faa2afdd640 1 -- 172.21.15.45:0/4123412666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa24100f40 msgr2=0x7faa241077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.808+0000 7faa2afdd640 1 --2- 172.21.15.45:0/4123412666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa24100f40 0x7faa241077d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faa1400aa10 tx=0x7faa1402cf70 comp rx=0 tx=0).stop 2024-01-31T05:22:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.808+0000 7faa2afdd640 1 -- 172.21.15.45:0/4123412666 shutdown_connections 2024-01-31T05:22:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.808+0000 7faa2afdd640 1 --2- 172.21.15.45:0/4123412666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa24100f40 0x7faa241077d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.808+0000 7faa2afdd640 1 --2- 172.21.15.45:0/4123412666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa24100580 0x7faa24100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.808+0000 7faa2afdd640 1 --2- 172.21.15.45:0/4123412666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa240ff390 0x7faa240ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.808+0000 7faa2afdd640 1 -- 172.21.15.45:0/4123412666 >> 172.21.15.45:0/4123412666 conn(0x7faa240fa900 msgr2=0x7faa240fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:26.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.808+0000 7faa2afdd640 1 -- 172.21.15.45:0/4123412666 shutdown_connections 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.808+0000 7faa2afdd640 1 -- 172.21.15.45:0/4123412666 wait complete. 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa2afdd640 1 Processor -- start 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa2afdd640 1 -- start start 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa2afdd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa240ff390 0x7faa24196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa2afdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa24100580 0x7faa24197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa2afdd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa24100f40 0x7faa2419e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa2afdd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2419e910 con 0x7faa24100580 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa2afdd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2419ea80 con 0x7faa24100f40 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa2afdd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2419ec40 con 0x7faa240ff390 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa23fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa24100580 0x7faa24197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa23fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa24100580 0x7faa24197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48826/0 (socket says 172.21.15.45:48826) 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa23fff640 1 -- 172.21.15.45:0/982551987 learned_addr learned my addr 172.21.15.45:0/982551987 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa28d79640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa240ff390 0x7faa24196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa2957a640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa24100f40 0x7faa2419e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:26.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa23fff640 1 -- 172.21.15.45:0/982551987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa240ff390 msgr2=0x7faa24196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa23fff640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa240ff390 0x7faa24196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa23fff640 1 -- 172.21.15.45:0/982551987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa24100f40 msgr2=0x7faa2419e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa23fff640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa24100f40 0x7faa2419e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa23fff640 1 -- 172.21.15.45:0/982551987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1400a670 con 0x7faa24100580 2024-01-31T05:22:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa2957a640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa24100f40 0x7faa2419e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa23fff640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa24100580 0x7faa24197360 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00c970 tx=0x7faa0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa21ffb640 1 -- 172.21.15.45:0/982551987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c007bb0 con 0x7faa24100580 2024-01-31T05:22:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa21ffb640 1 -- 172.21.15.45:0/982551987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa0c007d10 con 0x7faa24100580 2024-01-31T05:22:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.809+0000 7faa2afdd640 1 -- 172.21.15.45:0/982551987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2419ef20 con 0x7faa24100580 2024-01-31T05:22:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.810+0000 7faa21ffb640 1 -- 172.21.15.45:0/982551987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c004e40 con 0x7faa24100580 2024-01-31T05:22:26.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.810+0000 7faa2afdd640 1 -- 172.21.15.45:0/982551987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2419f3a0 con 0x7faa24100580 2024-01-31T05:22:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.811+0000 7faa2afdd640 1 -- 172.21.15.45:0/982551987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2410a400 con 0x7faa24100580 2024-01-31T05:22:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.812+0000 7faa21ffb640 1 -- 172.21.15.45:0/982551987 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7faa0c005020 con 0x7faa24100580 2024-01-31T05:22:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.812+0000 7faa21ffb640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa100a1360 0x7faa100a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.812+0000 7faa28d79640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa100a1360 0x7faa100a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.812+0000 7faa21ffb640 1 -- 172.21.15.45:0/982551987 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7faa0c0cc870 con 0x7faa24100580 2024-01-31T05:22:26.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.813+0000 7faa28d79640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa100a1360 0x7faa100a3820 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00a300 tx=0x7faa1c00a290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:26.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.816+0000 7faa21ffb640 1 -- 172.21.15.45:0/982551987 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7faa0c095d20 con 0x7faa24100580 2024-01-31T05:22:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:26 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2213415978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:26 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/942452059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:26 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3065855429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:22:26.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.995+0000 7faa2afdd640 1 -- 172.21.15.45:0/982551987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa2410a5c0 con 0x7faa24100580 2024-01-31T05:22:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.995+0000 7faa21ffb640 1 -- 172.21.15.45:0/982551987 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa2410a5c0 con 0x7faa24100580 2024-01-31T05:22:26.994 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-01-31T05:22:26.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.997+0000 7faa2afdd640 1 -- 172.21.15.45:0/982551987 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa100a1360 msgr2=0x7faa100a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.997+0000 7faa2afdd640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa100a1360 0x7faa100a3820 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00a300 tx=0x7faa1c00a290 comp rx=0 tx=0).stop 2024-01-31T05:22:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.997+0000 7faa2afdd640 1 -- 172.21.15.45:0/982551987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa24100580 msgr2=0x7faa24197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.997+0000 7faa2afdd640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa24100580 0x7faa24197360 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00c970 tx=0x7faa0c00ce30 comp rx=0 tx=0).stop 2024-01-31T05:22:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.997+0000 7faa2afdd640 1 -- 172.21.15.45:0/982551987 shutdown_connections 2024-01-31T05:22:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.997+0000 7faa2afdd640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7faa100a1360 0x7faa100a3820 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.997+0000 7faa2afdd640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa24100580 0x7faa24197360 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.997+0000 7faa2afdd640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa24100f40 0x7faa2419e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.997+0000 7faa2afdd640 1 --2- 172.21.15.45:0/982551987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa240ff390 0x7faa24196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.997+0000 7faa2afdd640 1 -- 172.21.15.45:0/982551987 >> 172.21.15.45:0/982551987 conn(0x7faa240fa900 msgr2=0x7faa24105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.997+0000 7faa2afdd640 1 -- 172.21.15.45:0/982551987 shutdown_connections 2024-01-31T05:22:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:26.997+0000 7faa2afdd640 1 -- 172.21.15.45:0/982551987 wait complete. 2024-01-31T05:22:27.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:22:27.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.110+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/1324468479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2b40ffc90 msgr2=0x7fe2b41000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.110+0000 7fe2b8bd0640 1 --2- 172.21.15.45:0/1324468479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2b40ffc90 0x7fe2b41000b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009900 tx=0x7fe2ac02bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:27.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.110+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/1324468479 shutdown_connections 2024-01-31T05:22:27.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.110+0000 7fe2b8bd0640 1 --2- 172.21.15.45:0/1324468479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b4101840 0x7fe2b41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.110+0000 7fe2b8bd0640 1 --2- 172.21.15.45:0/1324468479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2b4100e80 0x7fe2b4101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.110+0000 7fe2b8bd0640 1 --2- 172.21.15.45:0/1324468479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2b40ffc90 0x7fe2b41000b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.110+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/1324468479 >> 172.21.15.45:0/1324468479 conn(0x7fe2b40fb220 msgr2=0x7fe2b40fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.110+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/1324468479 shutdown_connections 2024-01-31T05:22:27.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.110+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/1324468479 wait complete. 2024-01-31T05:22:27.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.111+0000 7fe2b8bd0640 1 Processor -- start 2024-01-31T05:22:27.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.111+0000 7fe2b8bd0640 1 -- start start 2024-01-31T05:22:27.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.111+0000 7fe2b8bd0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2b40ffc90 0x7fe2b4196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.111+0000 7fe2b8bd0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2b4100e80 0x7fe2b41972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.111+0000 7fe2b8bd0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b4101840 0x7fe2b419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.111+0000 7fe2b8bd0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b419e880 con 0x7fe2b4100e80 2024-01-31T05:22:27.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.111+0000 7fe2b8bd0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b419e9f0 con 0x7fe2b4101840 2024-01-31T05:22:27.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.111+0000 7fe2b8bd0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b419ebb0 con 0x7fe2b40ffc90 2024-01-31T05:22:27.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.111+0000 7fe2b1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2b4100e80 0x7fe2b41972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.111+0000 7fe2b1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2b4100e80 0x7fe2b41972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48844/0 (socket says 172.21.15.45:48844) 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.111+0000 7fe2b1d9b640 1 -- 172.21.15.45:0/549556922 learned_addr learned my addr 172.21.15.45:0/549556922 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.111+0000 7fe2b2d9d640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b4101840 0x7fe2b419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2b1d9b640 1 -- 172.21.15.45:0/549556922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2b40ffc90 msgr2=0x7fe2b4196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2b259c640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2b40ffc90 0x7fe2b4196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2b1d9b640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2b40ffc90 0x7fe2b4196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2b1d9b640 1 -- 172.21.15.45:0/549556922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b4101840 msgr2=0x7fe2b419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2b1d9b640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b4101840 0x7fe2b419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2b1d9b640 1 -- 172.21.15.45:0/549556922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac009560 con 0x7fe2b4100e80 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2b2d9d640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b4101840 0x7fe2b419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2b259c640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2b40ffc90 0x7fe2b4196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2b1d9b640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2b4100e80 0x7fe2b41972e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00b4a0 tx=0x7fe29c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2ab7fe640 1 -- 172.21.15.45:0/549556922 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c010610 con 0x7fe2b4100e80 2024-01-31T05:22:27.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2ab7fe640 1 -- 172.21.15.45:0/549556922 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe29c010c30 con 0x7fe2b4100e80 2024-01-31T05:22:27.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/549556922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b419ee90 con 0x7fe2b4100e80 2024-01-31T05:22:27.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2ab7fe640 1 -- 172.21.15.45:0/549556922 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c017dd0 con 0x7fe2b4100e80 2024-01-31T05:22:27.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.112+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/549556922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b419f2e0 con 0x7fe2b4100e80 2024-01-31T05:22:27.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.113+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/549556922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b41070d0 con 0x7fe2b4100e80 2024-01-31T05:22:27.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.114+0000 7fe2ab7fe640 1 -- 172.21.15.45:0/549556922 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe29c0173f0 con 0x7fe2b4100e80 2024-01-31T05:22:27.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.115+0000 7fe2ab7fe640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe2a00a12e0 0x7fe2a00a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.115+0000 7fe2b259c640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe2a00a12e0 0x7fe2a00a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.115+0000 7fe2ab7fe640 1 -- 172.21.15.45:0/549556922 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe29c0cc2e0 con 0x7fe2b4100e80 2024-01-31T05:22:27.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.115+0000 7fe2b259c640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe2a00a12e0 0x7fe2a00a37a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006010 tx=0x7fe2ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.119+0000 7fe2ab7fe640 1 -- 172.21.15.45:0/549556922 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe29c095700 con 0x7fe2b4100e80 2024-01-31T05:22:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.337+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/549556922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2b410c630 con 0x7fe2b4100e80 2024-01-31T05:22:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.338+0000 7fe2ab7fe640 1 -- 172.21.15.45:0/549556922 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11346 (secure 0 0 0) 0x7fe29c0092b0 con 0x7fe2b4100e80 2024-01-31T05:22:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:27.338 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1801709710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1801709710},{"type":"v1","addr":"172.21.15.45:6839","nonce":1801709710}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/1148586810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1148586810},{"type":"v1","addr":"172.21.15.45:6841","nonce":1148586810}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-31T05:20:40.057668+0000","modified":"2024-01-31T05:20:44.476398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3782584640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3782584640},{"type":"v1","addr":"172.21.15.156:6837","nonce":3782584640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/3821255404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3821255404},{"type":"v1","addr":"172.21.15.186:6839","nonce":3821255404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/3137398384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3137398384},{"type":"v1","addr":"172.21.15.156:6839","nonce":3137398384}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:22:27.338 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 20 2024-01-31T05:22:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.340+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/549556922 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe2a00a12e0 msgr2=0x7fe2a00a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.340+0000 7fe2b8bd0640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe2a00a12e0 0x7fe2a00a37a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006010 tx=0x7fe2ac000bc0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.340+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/549556922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2b4100e80 msgr2=0x7fe2b41972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.340+0000 7fe2b8bd0640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2b4100e80 0x7fe2b41972e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00b4a0 tx=0x7fe29c00b960 comp rx=0 tx=0).stop 2024-01-31T05:22:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.340+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/549556922 shutdown_connections 2024-01-31T05:22:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.340+0000 7fe2b8bd0640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe2a00a12e0 0x7fe2a00a37a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.340+0000 7fe2b8bd0640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2b4100e80 0x7fe2b41972e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.340+0000 7fe2b8bd0640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b4101840 0x7fe2b419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.340+0000 7fe2b8bd0640 1 --2- 172.21.15.45:0/549556922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2b40ffc90 0x7fe2b4196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.340+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/549556922 >> 172.21.15.45:0/549556922 conn(0x7fe2b40fb220 msgr2=0x7fe2b41042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.340+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/549556922 shutdown_connections 2024-01-31T05:22:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.340+0000 7fe2b8bd0640 1 -- 172.21.15.45:0/549556922 wait complete. 2024-01-31T05:22:27.351 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:22:27.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 -- 172.21.15.45:0/4056546581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde101040d0 msgr2=0x7fde101064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 --2- 172.21.15.45:0/4056546581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde101040d0 0x7fde101064c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fde04009900 tx=0x7fde0402bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:27.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 -- 172.21.15.45:0/4056546581 shutdown_connections 2024-01-31T05:22:27.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 --2- 172.21.15.45:0/4056546581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde101040d0 0x7fde101064c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 --2- 172.21.15.45:0/4056546581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde10106a00 0x7fde10108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 --2- 172.21.15.45:0/4056546581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde101017a0 0x7fde10103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 -- 172.21.15.45:0/4056546581 >> 172.21.15.45:0/4056546581 conn(0x7fde100fb1d0 msgr2=0x7fde100fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 -- 172.21.15.45:0/4056546581 shutdown_connections 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 -- 172.21.15.45:0/4056546581 wait complete. 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 Processor -- start 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 -- start start 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde101017a0 0x7fde10196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde101040d0 0x7fde10197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde10106a00 0x7fde1019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1019e7f0 con 0x7fde101040d0 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1019e960 con 0x7fde101017a0 2024-01-31T05:22:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde16d2b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1019eb20 con 0x7fde10106a00 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde0ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde101040d0 0x7fde10197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde0ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde101040d0 0x7fde10197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48862/0 (socket says 172.21.15.45:48862) 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.454+0000 7fde0ffff640 1 -- 172.21.15.45:0/1143987580 learned_addr learned my addr 172.21.15.45:0/1143987580 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.455+0000 7fde14ac7640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde101017a0 0x7fde10196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.455+0000 7fde0ffff640 1 -- 172.21.15.45:0/1143987580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde10106a00 msgr2=0x7fde1019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.455+0000 7fde0ffff640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde10106a00 0x7fde1019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.455+0000 7fde0ffff640 1 -- 172.21.15.45:0/1143987580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde101017a0 msgr2=0x7fde10196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.455+0000 7fde0ffff640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde101017a0 0x7fde10196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.455+0000 7fde0ffff640 1 -- 172.21.15.45:0/1143987580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde04009560 con 0x7fde101040d0 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.455+0000 7fde0ffff640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde101040d0 0x7fde10197250 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fde04009230 tx=0x7fde040040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.455+0000 7fde0dffb640 1 -- 172.21.15.45:0/1143987580 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde040044c0 con 0x7fde101040d0 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.455+0000 7fde0dffb640 1 -- 172.21.15.45:0/1143987580 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde040054c0 con 0x7fde101040d0 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.455+0000 7fde16d2b640 1 -- 172.21.15.45:0/1143987580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1019eda0 con 0x7fde101040d0 2024-01-31T05:22:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.455+0000 7fde0dffb640 1 -- 172.21.15.45:0/1143987580 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0403e450 con 0x7fde101040d0 2024-01-31T05:22:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.455+0000 7fde16d2b640 1 -- 172.21.15.45:0/1143987580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1019f270 con 0x7fde101040d0 2024-01-31T05:22:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.456+0000 7fde16d2b640 1 -- 172.21.15.45:0/1143987580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde100fccc0 con 0x7fde101040d0 2024-01-31T05:22:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.457+0000 7fde0dffb640 1 -- 172.21.15.45:0/1143987580 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fde040036f0 con 0x7fde101040d0 2024-01-31T05:22:27.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.458+0000 7fde0dffb640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fddf80a12e0 0x7fddf80a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.458+0000 7fde14ac7640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fddf80a12e0 0x7fddf80a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.458+0000 7fde0dffb640 1 -- 172.21.15.45:0/1143987580 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fde040ef6f0 con 0x7fde101040d0 2024-01-31T05:22:27.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.458+0000 7fde14ac7640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fddf80a12e0 0x7fddf80a37a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde101981e0 tx=0x7fde080094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.462+0000 7fde0dffb640 1 -- 172.21.15.45:0/1143987580 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fde040b8b10 con 0x7fde101040d0 2024-01-31T05:22:27.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:27 smithi045 ceph-mon[31585]: pgmap v338: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:22:27.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:27 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/52823088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:22:27.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:27 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/982551987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:22:27.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:27 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/549556922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:27.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.645+0000 7fde16d2b640 1 -- 172.21.15.45:0/1143987580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde1002cce0 con 0x7fde101040d0 2024-01-31T05:22:27.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.647+0000 7fde0dffb640 1 -- 172.21.15.45:0/1143987580 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22520 (secure 0 0 0) 0x7fde040baa40 con 0x7fde101040d0 2024-01-31T05:22:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:22:27.646 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":88,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:21:47.682334+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:22:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.649+0000 7fde16d2b640 1 -- 172.21.15.45:0/1143987580 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fddf80a12e0 msgr2=0x7fddf80a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.649+0000 7fde16d2b640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fddf80a12e0 0x7fddf80a37a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde101981e0 tx=0x7fde080094f0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.649+0000 7fde16d2b640 1 -- 172.21.15.45:0/1143987580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde101040d0 msgr2=0x7fde10197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.649+0000 7fde16d2b640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde101040d0 0x7fde10197250 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fde04009230 tx=0x7fde040040c0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.649+0000 7fde16d2b640 1 -- 172.21.15.45:0/1143987580 shutdown_connections 2024-01-31T05:22:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.649+0000 7fde16d2b640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fddf80a12e0 0x7fddf80a37a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.649+0000 7fde16d2b640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde101040d0 0x7fde10197250 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.649+0000 7fde16d2b640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde10106a00 0x7fde1019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.649+0000 7fde16d2b640 1 --2- 172.21.15.45:0/1143987580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde101017a0 0x7fde10196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.649+0000 7fde16d2b640 1 -- 172.21.15.45:0/1143987580 >> 172.21.15.45:0/1143987580 conn(0x7fde100fb1d0 msgr2=0x7fde10102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.649+0000 7fde16d2b640 1 -- 172.21.15.45:0/1143987580 shutdown_connections 2024-01-31T05:22:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.649+0000 7fde16d2b640 1 -- 172.21.15.45:0/1143987580 wait complete. 2024-01-31T05:22:27.659 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-01-31T05:20:40.057668+0000', 'modified': '2024-01-31T05:20:44.476398+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24503, 'mds_1': 24490, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/4045803513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 4045803513}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 4045803513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/696958495', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 696958495}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 696958495}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6837/3782584640', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 3782584640}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 3782584640}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.186:6837/2559285393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 2559285393}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 2559285393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 16, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.186:6839/3821255404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 3821255404}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 3821255404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24503': {'gid': 24503, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6839/3137398384', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 3137398384}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 3137398384}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-31T05:22:27.660 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-01-31T05:22:27.660 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-31T05:22:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.764+0000 7f9397054640 1 -- 172.21.15.45:0/3520488016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93901040a0 msgr2=0x7f9390106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.764+0000 7f9397054640 1 --2- 172.21.15.45:0/3520488016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93901040a0 0x7f9390106490 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9384009900 tx=0x7f938402bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.764+0000 7f9397054640 1 -- 172.21.15.45:0/3520488016 shutdown_connections 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.764+0000 7f9397054640 1 --2- 172.21.15.45:0/3520488016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93901040a0 0x7f9390106490 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.764+0000 7f9397054640 1 --2- 172.21.15.45:0/3520488016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93901069d0 0x7f9390108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.764+0000 7f9397054640 1 --2- 172.21.15.45:0/3520488016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9390101770 0x7f9390103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.764+0000 7f9397054640 1 -- 172.21.15.45:0/3520488016 >> 172.21.15.45:0/3520488016 conn(0x7f93900fb180 msgr2=0x7f93900fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f9397054640 1 -- 172.21.15.45:0/3520488016 shutdown_connections 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f9397054640 1 -- 172.21.15.45:0/3520488016 wait complete. 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f9397054640 1 Processor -- start 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f9397054640 1 -- start start 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f9397054640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9390101770 0x7f9390196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f9397054640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93901040a0 0x7f93901971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f9397054640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93901069d0 0x7f939019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f9397054640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939019e7c0 con 0x7f93901069d0 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f9397054640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939019e930 con 0x7f93901040a0 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f9397054640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939019eaf0 con 0x7f9390101770 2024-01-31T05:22:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f93955f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93901069d0 0x7f939019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f93955f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93901069d0 0x7f939019e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58594/0 (socket says 172.21.15.45:58594) 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f93955f1640 1 -- 172.21.15.45:0/219521467 learned_addr learned my addr 172.21.15.45:0/219521467 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.765+0000 7f938ffff640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93901040a0 0x7f93901971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.766+0000 7f93955f1640 1 -- 172.21.15.45:0/219521467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9390101770 msgr2=0x7f9390196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.766+0000 7f93955f1640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9390101770 0x7f9390196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.766+0000 7f93955f1640 1 -- 172.21.15.45:0/219521467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93901040a0 msgr2=0x7f93901971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.766+0000 7f93955f1640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93901040a0 0x7f93901971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.766+0000 7f93955f1640 1 -- 172.21.15.45:0/219521467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9384009560 con 0x7f93901069d0 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.766+0000 7f93955f1640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93901069d0 0x7f939019e250 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f938000c970 tx=0x7f938000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.766+0000 7f938dffb640 1 -- 172.21.15.45:0/219521467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380007bb0 con 0x7f93901069d0 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.766+0000 7f938dffb640 1 -- 172.21.15.45:0/219521467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9380007d10 con 0x7f93901069d0 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.766+0000 7f9397054640 1 -- 172.21.15.45:0/219521467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939019edd0 con 0x7f93901069d0 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.766+0000 7f938dffb640 1 -- 172.21.15.45:0/219521467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93800186c0 con 0x7f93901069d0 2024-01-31T05:22:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.766+0000 7f9397054640 1 -- 172.21.15.45:0/219521467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939019f200 con 0x7f93901069d0 2024-01-31T05:22:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.769+0000 7f938dffb640 1 -- 172.21.15.45:0/219521467 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f938000f450 con 0x7f93901069d0 2024-01-31T05:22:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.769+0000 7f9399e0f740 1 Processor -- start 2024-01-31T05:22:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.769+0000 7f9399e0f740 1 -- start start 2024-01-31T05:22:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.769+0000 7f9399e0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 0x5646f37b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.769+0000 7f9399e0f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b8470 0x5646f37bcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.769+0000 7f9399e0f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37bd5e0 0x5646f37bfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.769+0000 7f9399e0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f3722000 con 0x5646f37b7a80 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.769+0000 7f9399e0f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f371f6f0 con 0x5646f37b8470 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.769+0000 7f9399e0f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f371afe0 con 0x5646f37bd5e0 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.769+0000 7f938dffb640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f93780a12e0 0x7f93780a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.769+0000 7f938ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 0x5646f37b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f938ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 0x5646f37b7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58604/0 (socket says 172.21.15.45:58604) 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f938ffff640 1 -- 172.21.15.45:0/1756421724 learned_addr learned my addr 172.21.15.45:0/1756421724 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f9394df0640 1 --2- 172.21.15.45:0/1756421724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b8470 0x5646f37bcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f93955f1640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f93780a12e0 0x7f93780a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f938ffff640 1 --2- 172.21.15.45:0/1756421724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37bd5e0 0x5646f37bfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f938dffb640 1 -- 172.21.15.45:0/219521467 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f93800cccd0 con 0x7f93901069d0 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f938ffff640 1 -- 172.21.15.45:0/1756421724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37bd5e0 msgr2=0x5646f37bfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f938ffff640 1 --2- 172.21.15.45:0/1756421724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37bd5e0 0x5646f37bfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f938ffff640 1 -- 172.21.15.45:0/1756421724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b8470 msgr2=0x5646f37bcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f938ffff640 1 --2- 172.21.15.45:0/1756421724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b8470 0x5646f37bcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f938ffff640 1 -- 172.21.15.45:0/1756421724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646f36dd650 con 0x5646f37b7a80 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f9394df0640 1 --2- 172.21.15.45:0/1756421724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b8470 0x5646f37bcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f938ffff640 1 --2- 172.21.15.45:0/1756421724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 0x5646f37b7ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9384005ab0 tx=0x7f9384005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6fc4efb4182a69 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f93955f1640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f93780a12e0 0x7f93780a37a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9380096850 tx=0x7f938008f5a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f9376ffd640 1 -- 172.21.15.45:0/1756421724 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384047070 con 0x5646f37b7a80 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f9376ffd640 1 -- 172.21.15.45:0/1756421724 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9384004030 con 0x5646f37b7a80 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.770+0000 7f9376ffd640 1 -- 172.21.15.45:0/1756421724 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384005570 con 0x5646f37b7a80 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.771+0000 7f9399e0f740 1 -- 172.21.15.45:0/1756421724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 msgr2=0x5646f37b7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.771+0000 7f9399e0f740 1 --2- 172.21.15.45:0/1756421724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 0x5646f37b7ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9384005ab0 tx=0x7f9384005a20 comp rx=0 tx=0).stop 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.771+0000 7f9399e0f740 1 -- 172.21.15.45:0/1756421724 shutdown_connections 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.771+0000 7f9399e0f740 1 --2- 172.21.15.45:0/1756421724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37bd5e0 0x5646f37bfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.771+0000 7f9399e0f740 1 --2- 172.21.15.45:0/1756421724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b8470 0x5646f37bcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.771+0000 7f9399e0f740 1 --2- 172.21.15.45:0/1756421724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 0x5646f37b7ea0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.771+0000 7f9399e0f740 1 -- 172.21.15.45:0/1756421724 >> 172.21.15.45:0/1756421724 conn(0x5646f37b2ff0 msgr2=0x5646f37b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.771+0000 7f9399e0f740 1 -- 172.21.15.45:0/1756421724 shutdown_connections 2024-01-31T05:22:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.771+0000 7f9399e0f740 1 -- 172.21.15.45:0/1756421724 wait complete. 2024-01-31T05:22:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.771+0000 7f9399e0f740 1 -- start start 2024-01-31T05:22:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.771+0000 7f9399e0f740 1 Processor -- start 2024-01-31T05:22:27.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f9399e0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 0x5646f37c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f9399e0f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b8470 0x5646f37c2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f9399e0f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37bd5e0 0x5646f37c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f9399e0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f36d45c0 con 0x5646f37b7a80 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f9399e0f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f36d4260 con 0x5646f37bd5e0 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f9399e0f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f36992d0 con 0x5646f37b8470 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f9399e0f740 10 client.0 _ll_register_callbacks cb 0x5646f36cf230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f938ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 0x5646f37c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f938ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 0x5646f37c1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58610/0 (socket says 172.21.15.45:58610) 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f938ffff640 1 -- 172.21.15.45:0/4073409050 learned_addr learned my addr 172.21.15.45:0/4073409050 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f9394df0640 1 --2- 172.21.15.45:0/4073409050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b8470 0x5646f37c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f938ffff640 1 -- 172.21.15.45:0/4073409050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b8470 msgr2=0x5646f37c2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f938ffff640 1 --2- 172.21.15.45:0/4073409050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b8470 0x5646f37c2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f938ffff640 1 -- 172.21.15.45:0/4073409050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37bd5e0 msgr2=0x5646f37c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f938ffff640 1 --2- 172.21.15.45:0/4073409050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37bd5e0 0x5646f37c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f938ffff640 1 -- 172.21.15.45:0/4073409050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9390109a00 con 0x5646f37b7a80 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f938ffff640 1 --2- 172.21.15.45:0/4073409050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 0x5646f37c1d00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9384009900 tx=0x7f93840058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88447a7a6eacb16 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f936ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f936ffff640 1 -- 172.21.15.45:0/4073409050 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384047040 con 0x5646f37b7a80 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f936ffff640 1 -- 172.21.15.45:0/4073409050 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9384004950 con 0x5646f37b7a80 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.772+0000 7f9399e0f740 1 -- 172.21.15.45:0/4073409050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646f3713e70 con 0x5646f37b7a80 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.773+0000 7f936ffff640 1 -- 172.21.15.45:0/4073409050 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93840035f0 con 0x5646f37b7a80 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.773+0000 7f9399e0f740 10 client.15225 fetch_fsmap learned FSMap version 20 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.773+0000 7f9399e0f740 1 -- 172.21.15.45:0/4073409050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5646f3719660 con 0x5646f37b7a80 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.773+0000 7f936ffff640 1 -- 172.21.15.45:0/4073409050 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f938405d020 con 0x5646f37b7a80 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.773+0000 7f936ffff640 1 -- 172.21.15.45:0/4073409050 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f938404d720 con 0x5646f37b7a80 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.773+0000 7f9399e0f740 10 client.15225 fetch_fsmap finished waiting for FSMap version 20 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.773+0000 7f9399e0f740 10 client.15225 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.773+0000 7f9399e0f740 20 client.15225 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.773+0000 7f9399e0f740 1 --2- 172.21.15.45:0/4073409050 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x5646f37cebd0 0x5646f37cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.773+0000 7f9399e0f740 4 client.15225 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:22:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.773+0000 7f9399e0f740 1 -- 172.21.15.45:0/4073409050 --> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646f3713b90 con 0x5646f37cebd0 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.773+0000 7f9394df0640 1 --2- 172.21.15.45:0/4073409050 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x5646f37cebd0 0x5646f37cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.774+0000 7f9394df0640 1 --2- 172.21.15.45:0/4073409050 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x5646f37cebd0 0x5646f37cd700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=639d3ce2419cedc0 server_cookie=8b69262f9af1b2c6 in_seq=0 out_seq=0 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.774+0000 7f936ffff640 10 client.15225 ms_handle_connect on v2:172.21.15.186:6836/2559285393 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f936ffff640 1 -- 172.21.15.45:0/4073409050 <== mds.2 v2:172.21.15.186:6836/2559285393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5646f3713b90 con 0x5646f37cebd0 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f936ffff640 10 client.15225 handle_command_reply: tid=0 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f9399e0f740 1 client.15225 shutdown 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f9399e0f740 1 -- 172.21.15.45:0/4073409050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 msgr2=0x5646f37c1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f9399e0f740 1 --2- 172.21.15.45:0/4073409050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 0x5646f37c1d00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9384009900 tx=0x7f93840058b0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f9399e0f740 1 -- 172.21.15.45:0/4073409050 shutdown_connections 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f9399e0f740 1 --2- 172.21.15.45:0/4073409050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37bd5e0 0x5646f37c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f9399e0f740 1 --2- 172.21.15.45:0/4073409050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b8470 0x5646f37c2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f9399e0f740 1 --2- 172.21.15.45:0/4073409050 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x5646f37cebd0 0x5646f37cd700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f9399e0f740 1 --2- 172.21.15.45:0/4073409050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b7a80 0x5646f37c1d00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f936ffff640 0 client.15225 ms_handle_reset on v2:172.21.15.186:6836/2559285393 2024-01-31T05:22:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f9399e0f740 1 -- 172.21.15.45:0/4073409050 >> 172.21.15.45:0/4073409050 conn(0x5646f37b2ff0 msgr2=0x5646f37add70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f9399e0f740 1 -- 172.21.15.45:0/4073409050 shutdown_connections 2024-01-31T05:22:27.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f9399e0f740 1 -- 172.21.15.45:0/4073409050 wait complete. 2024-01-31T05:22:27.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.775+0000 7f9399e0f740 20 client.15225 trim_cache size 0 max 16384 2024-01-31T05:22:27.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.790+0000 7f9399e0f740 1 Processor -- start 2024-01-31T05:22:27.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f9399e0f740 1 -- start start 2024-01-31T05:22:27.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f9399e0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 0x5646f37cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f9399e0f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b7a80 0x5646f37c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f9399e0f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b8470 0x5646f37cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f9399e0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f37bff30 con 0x5646f37b2ff0 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f9399e0f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f36c2650 con 0x5646f37b7a80 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f9399e0f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f36beff0 con 0x5646f37b8470 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f938ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 0x5646f37cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f938ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 0x5646f37cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58616/0 (socket says 172.21.15.45:58616) 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f938ffff640 1 -- 172.21.15.45:0/1315487582 learned_addr learned my addr 172.21.15.45:0/1315487582 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f9394df0640 1 --2- 172.21.15.45:0/1315487582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b7a80 0x5646f37c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f938ffff640 1 -- 172.21.15.45:0/1315487582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b8470 msgr2=0x5646f37cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f938ffff640 1 --2- 172.21.15.45:0/1315487582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b8470 0x5646f37cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f938ffff640 1 -- 172.21.15.45:0/1315487582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b7a80 msgr2=0x5646f37c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f938ffff640 1 --2- 172.21.15.45:0/1315487582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b7a80 0x5646f37c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f938ffff640 1 -- 172.21.15.45:0/1315487582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646f36b6360 con 0x5646f37b2ff0 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f9394df0640 1 --2- 172.21.15.45:0/1315487582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b7a80 0x5646f37c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f938ffff640 1 --2- 172.21.15.45:0/1315487582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 0x5646f37cd700 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f938402c170 tx=0x7f938402c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4453905d8308e036 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f93767fc640 1 -- 172.21.15.45:0/1315487582 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384047070 con 0x5646f37b2ff0 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f93767fc640 1 -- 172.21.15.45:0/1315487582 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f938405d030 con 0x5646f37b2ff0 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.791+0000 7f93767fc640 1 -- 172.21.15.45:0/1315487582 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384005500 con 0x5646f37b2ff0 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.792+0000 7f9399e0f740 1 -- 172.21.15.45:0/1315487582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 msgr2=0x5646f37cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.792+0000 7f9399e0f740 1 --2- 172.21.15.45:0/1315487582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 0x5646f37cd700 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f938402c170 tx=0x7f938402c330 comp rx=0 tx=0).stop 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.792+0000 7f9399e0f740 1 -- 172.21.15.45:0/1315487582 shutdown_connections 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.792+0000 7f9399e0f740 1 --2- 172.21.15.45:0/1315487582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b8470 0x5646f37cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.792+0000 7f9399e0f740 1 --2- 172.21.15.45:0/1315487582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b7a80 0x5646f37c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.792+0000 7f9399e0f740 1 --2- 172.21.15.45:0/1315487582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 0x5646f37cd700 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.792+0000 7f9399e0f740 1 -- 172.21.15.45:0/1315487582 >> 172.21.15.45:0/1315487582 conn(0x5646f37c0ac0 msgr2=0x5646f37b9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.792+0000 7f9399e0f740 1 -- 172.21.15.45:0/1315487582 shutdown_connections 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.792+0000 7f9399e0f740 1 -- 172.21.15.45:0/1315487582 wait complete. 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.792+0000 7f9399e0f740 1 -- start start 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.792+0000 7f9399e0f740 1 Processor -- start 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f9399e0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 0x5646f37c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f9399e0f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b7a80 0x5646f37c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f9399e0f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b8470 0x5646f37c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f9399e0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f37c6430 con 0x5646f37b2ff0 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f9399e0f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f37c65a0 con 0x5646f37b8470 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f9399e0f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f3823b10 con 0x5646f37b7a80 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f9399e0f740 10 client.0 _ll_register_callbacks cb 0x7f9384009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f938ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 0x5646f37c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f938ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 0x5646f37c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58622/0 (socket says 172.21.15.45:58622) 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f938ffff640 1 -- 172.21.15.45:0/897231563 learned_addr learned my addr 172.21.15.45:0/897231563 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f9394df0640 1 --2- 172.21.15.45:0/897231563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b7a80 0x5646f37c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f938ffff640 1 -- 172.21.15.45:0/897231563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b7a80 msgr2=0x5646f37c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f938ffff640 1 --2- 172.21.15.45:0/897231563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b7a80 0x5646f37c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f938ffff640 1 -- 172.21.15.45:0/897231563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b8470 msgr2=0x5646f37c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f938ffff640 1 --2- 172.21.15.45:0/897231563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b8470 0x5646f37c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f938ffff640 1 -- 172.21.15.45:0/897231563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646f3719660 con 0x5646f37b2ff0 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f938ffff640 1 --2- 172.21.15.45:0/897231563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 0x5646f37c8e50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f938402c6b0 tx=0x7f9384005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6d0de16fa2202c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f936ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f936ffff640 1 -- 172.21.15.45:0/897231563 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384047040 con 0x5646f37b2ff0 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f936ffff640 1 -- 172.21.15.45:0/897231563 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9384000e40 con 0x5646f37b2ff0 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f9399e0f740 1 -- 172.21.15.45:0/897231563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9384004030 con 0x5646f37b2ff0 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f936ffff640 1 -- 172.21.15.45:0/897231563 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9384005500 con 0x5646f37b2ff0 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f9399e0f740 10 client.15231 fetch_fsmap learned FSMap version 20 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.793+0000 7f9399e0f740 1 -- 172.21.15.45:0/897231563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5646f36dbeb0 con 0x5646f37b2ff0 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.794+0000 7f936ffff640 1 -- 172.21.15.45:0/897231563 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9384004030 con 0x5646f37b2ff0 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.794+0000 7f936ffff640 1 -- 172.21.15.45:0/897231563 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f93840038e0 con 0x5646f37b2ff0 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.794+0000 7f9399e0f740 10 client.15231 fetch_fsmap finished waiting for FSMap version 20 2024-01-31T05:22:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.794+0000 7f9399e0f740 10 client.15231 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.794+0000 7f9399e0f740 20 client.15231 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.794+0000 7f9399e0f740 1 --2- 172.21.15.45:0/897231563 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x5646f37bd5e0 0x5646f3824cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.794+0000 7f9399e0f740 4 client.15231 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.794+0000 7f9399e0f740 1 -- 172.21.15.45:0/897231563 --> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5646f341a6c0 con 0x5646f37bd5e0 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.794+0000 7f9394df0640 1 --2- 172.21.15.45:0/897231563 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x5646f37bd5e0 0x5646f3824cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f936ffff640 10 client.15231 ms_handle_connect on v2:172.21.15.186:6836/2559285393 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9394df0640 1 --2- 172.21.15.45:0/897231563 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x5646f37bd5e0 0x5646f3824cb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97ce82a90e90a71c server_cookie=29978e7713b6c15c in_seq=0 out_seq=0 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f936ffff640 1 -- 172.21.15.45:0/897231563 <== mds.2 v2:172.21.15.186:6836/2559285393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x5646f341a6c0 con 0x5646f37bd5e0 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f936ffff640 10 client.15231 handle_command_reply: tid=0 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9399e0f740 1 client.15231 shutdown 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9399e0f740 1 -- 172.21.15.45:0/897231563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 msgr2=0x5646f37c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9399e0f740 1 --2- 172.21.15.45:0/897231563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 0x5646f37c8e50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f938402c6b0 tx=0x7f9384005200 comp rx=0 tx=0).stop 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9399e0f740 1 -- 172.21.15.45:0/897231563 shutdown_connections 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9399e0f740 1 --2- 172.21.15.45:0/897231563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646f37b8470 0x5646f37c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9399e0f740 1 --2- 172.21.15.45:0/897231563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646f37b7a80 0x5646f37c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9399e0f740 1 --2- 172.21.15.45:0/897231563 >> [v2:172.21.15.186:6836/2559285393,v1:172.21.15.186:6837/2559285393] conn(0x5646f37bd5e0 0x5646f3824cb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9399e0f740 1 --2- 172.21.15.45:0/897231563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646f37b2ff0 0x5646f37c8e50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f936ffff640 0 client.15231 ms_handle_reset on v2:172.21.15.186:6836/2559285393 2024-01-31T05:22:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9399e0f740 1 -- 172.21.15.45:0/897231563 >> 172.21.15.45:0/897231563 conn(0x5646f37c0ac0 msgr2=0x5646f37cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9399e0f740 1 -- 172.21.15.45:0/897231563 shutdown_connections 2024-01-31T05:22:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9399e0f740 1 -- 172.21.15.45:0/897231563 wait complete. 2024-01-31T05:22:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.795+0000 7f9399e0f740 20 client.15231 trim_cache size 0 max 16384 2024-01-31T05:22:27.800 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-01-31T05:22:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "3a1f4122-bff7-11ee-95b3-87774f69a715", 2024-01-31T05:22:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2024-01-31T05:22:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24469, 2024-01-31T05:22:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-01-31T05:22:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-01-31T05:22:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-01-31T05:22:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 104.32284588, 2024-01-31T05:22:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 20, 2024-01-31T05:22:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 88, 2024-01-31T05:22:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 86, 2024-01-31T05:22:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 126.582712051 2024-01-31T05:22:27.801 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-01-31T05:22:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.796+0000 7f9397054640 1 -- 172.21.15.45:0/219521467 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f93780a12e0 msgr2=0x7f93780a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.796+0000 7f9397054640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f93780a12e0 0x7f93780a37a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9380096850 tx=0x7f938008f5a0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.796+0000 7f9397054640 1 -- 172.21.15.45:0/219521467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93901069d0 msgr2=0x7f939019e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.796+0000 7f9397054640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93901069d0 0x7f939019e250 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f938000c970 tx=0x7f938000ce30 comp rx=0 tx=0).stop 2024-01-31T05:22:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.796+0000 7f9397054640 1 -- 172.21.15.45:0/219521467 shutdown_connections 2024-01-31T05:22:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.796+0000 7f9397054640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f93780a12e0 0x7f93780a37a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.796+0000 7f9397054640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93901069d0 0x7f939019e250 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.796+0000 7f9397054640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93901040a0 0x7f93901971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.796+0000 7f9397054640 1 --2- 172.21.15.45:0/219521467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9390101770 0x7f9390196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.796+0000 7f9397054640 1 -- 172.21.15.45:0/219521467 >> 172.21.15.45:0/219521467 conn(0x7f93900fb180 msgr2=0x7f93901020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.796+0000 7f9397054640 1 -- 172.21.15.45:0/219521467 shutdown_connections 2024-01-31T05:22:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.796+0000 7f9397054640 1 -- 172.21.15.45:0/219521467 wait complete. 2024-01-31T05:22:27.805 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-01-31T05:22:27.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:27 smithi186 ceph-mon[35659]: pgmap v338: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:22:27.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:27 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/52823088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:22:27.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:27 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/982551987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:22:27.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:27 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/549556922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:27.877 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:27 smithi156 ceph-mon[35720]: pgmap v338: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-31T05:22:27.878 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:27 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/52823088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:22:27.878 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:27 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/982551987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:22:27.878 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:27 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/549556922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:22:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.909+0000 7f4027b6b640 1 -- 172.21.15.45:0/189834814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40201017d0 msgr2=0x7f4020103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.909+0000 7f4027b6b640 1 --2- 172.21.15.45:0/189834814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40201017d0 0x7f4020103bc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f401c009900 tx=0x7f401c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.909+0000 7f4027b6b640 1 -- 172.21.15.45:0/189834814 shutdown_connections 2024-01-31T05:22:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.909+0000 7f4027b6b640 1 --2- 172.21.15.45:0/189834814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4020106a30 0x7f4020108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.909+0000 7f4027b6b640 1 --2- 172.21.15.45:0/189834814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4020104100 0x7f40201064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.909+0000 7f4027b6b640 1 --2- 172.21.15.45:0/189834814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40201017d0 0x7f4020103bc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.909+0000 7f4027b6b640 1 -- 172.21.15.45:0/189834814 >> 172.21.15.45:0/189834814 conn(0x7f40200fb180 msgr2=0x7f40200fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4027b6b640 1 -- 172.21.15.45:0/189834814 shutdown_connections 2024-01-31T05:22:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4027b6b640 1 -- 172.21.15.45:0/189834814 wait complete. 2024-01-31T05:22:27.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4027b6b640 1 Processor -- start 2024-01-31T05:22:27.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4027b6b640 1 -- start start 2024-01-31T05:22:27.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4027b6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40201017d0 0x7f4020101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4027b6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4020104100 0x7f40200ff780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4027b6b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4020106a30 0x7f40200ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4027b6b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4020100420 con 0x7f40201017d0 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4027b6b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4020100590 con 0x7f4020106a30 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4027b6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40201a0df0 con 0x7f4020104100 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4025907640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40201017d0 0x7f4020101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4025907640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40201017d0 0x7f4020101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58646/0 (socket says 172.21.15.45:58646) 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.910+0000 7f4025907640 1 -- 172.21.15.45:0/4210553291 learned_addr learned my addr 172.21.15.45:0/4210553291 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4026108640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4020106a30 0x7f40200ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4025106640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4020104100 0x7f40200ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4025907640 1 -- 172.21.15.45:0/4210553291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4020104100 msgr2=0x7f40200ff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4025907640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4020104100 0x7f40200ff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4025907640 1 -- 172.21.15.45:0/4210553291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4020106a30 msgr2=0x7f40200ffcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4025907640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4020106a30 0x7f40200ffcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4025907640 1 -- 172.21.15.45:0/4210553291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c009560 con 0x7f40201017d0 2024-01-31T05:22:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4025106640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4020104100 0x7f40200ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:27.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4025907640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40201017d0 0x7f4020101140 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f401c009430 tx=0x7f401c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4016ffd640 1 -- 172.21.15.45:0/4210553291 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0044e0 con 0x7f40201017d0 2024-01-31T05:22:27.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4016ffd640 1 -- 172.21.15.45:0/4210553291 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401c0035f0 con 0x7f40201017d0 2024-01-31T05:22:27.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4027b6b640 1 -- 172.21.15.45:0/4210553291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40201a1040 con 0x7f40201017d0 2024-01-31T05:22:27.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4016ffd640 1 -- 172.21.15.45:0/4210553291 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c03e6b0 con 0x7f40201017d0 2024-01-31T05:22:27.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.911+0000 7f4027b6b640 1 -- 172.21.15.45:0/4210553291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40201a1620 con 0x7f40201017d0 2024-01-31T05:22:27.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.913+0000 7f4016ffd640 1 -- 172.21.15.45:0/4210553291 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f401c0039d0 con 0x7f40201017d0 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.914+0000 7f4016ffd640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f400c0a1360 0x7f400c0a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.914+0000 7f402aa0c740 1 Processor -- start 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.914+0000 7f4025106640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f400c0a1360 0x7f400c0a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.914+0000 7f402aa0c740 1 -- start start 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.914+0000 7f4016ffd640 1 -- 172.21.15.45:0/4210553291 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f401c0ef000 con 0x7f40201017d0 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.914+0000 7f402aa0c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 0x564fcf979a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.914+0000 7f402aa0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 0x564fcf97eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.914+0000 7f402aa0c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97f1d0 0x564fcf981660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.914+0000 7f402aa0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf8e8000 con 0x564fcf97a060 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.914+0000 7f402aa0c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf8e56f0 con 0x564fcf979670 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.914+0000 7f402aa0c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf8e0fe0 con 0x564fcf97f1d0 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f4025106640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 0x564fcf97eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f4025106640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 0x564fcf97eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58648/0 (socket says 172.21.15.45:58648) 2024-01-31T05:22:27.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f4025106640 1 -- 172.21.15.45:0/2508359913 learned_addr learned my addr 172.21.15.45:0/2508359913 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f4025907640 1 --2- 172.21.15.45:0/2508359913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 0x564fcf979a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f4026108640 1 --2- 172.21.15.45:0/2508359913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97f1d0 0x564fcf981660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f4025106640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f400c0a1360 0x7f400c0a3820 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4020100eb0 tx=0x7f4010005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f4025106640 1 -- 172.21.15.45:0/2508359913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97f1d0 msgr2=0x564fcf981660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f4025106640 1 --2- 172.21.15.45:0/2508359913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97f1d0 0x564fcf981660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f4025106640 1 -- 172.21.15.45:0/2508359913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 msgr2=0x564fcf979a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f4025106640 1 --2- 172.21.15.45:0/2508359913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 0x564fcf979a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f4025106640 1 -- 172.21.15.45:0/2508359913 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fcf8a3650 con 0x564fcf97a060 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f4025106640 1 --2- 172.21.15.45:0/2508359913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 0x564fcf97eb60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f401000ddc0 tx=0x7f4010010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9564b250365d218e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f400bfff640 1 -- 172.21.15.45:0/2508359913 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401001c650 con 0x564fcf97a060 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f400bfff640 1 -- 172.21.15.45:0/2508359913 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401001cc70 con 0x564fcf97a060 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.915+0000 7f400bfff640 1 -- 172.21.15.45:0/2508359913 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401001bc70 con 0x564fcf97a060 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.916+0000 7f402aa0c740 1 -- 172.21.15.45:0/2508359913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 msgr2=0x564fcf97eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.916+0000 7f402aa0c740 1 --2- 172.21.15.45:0/2508359913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 0x564fcf97eb60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f401000ddc0 tx=0x7f4010010000 comp rx=0 tx=0).stop 2024-01-31T05:22:27.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.916+0000 7f402aa0c740 1 -- 172.21.15.45:0/2508359913 shutdown_connections 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.916+0000 7f402aa0c740 1 --2- 172.21.15.45:0/2508359913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 0x564fcf97eb60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.916+0000 7f402aa0c740 1 --2- 172.21.15.45:0/2508359913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97f1d0 0x564fcf981660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.916+0000 7f402aa0c740 1 --2- 172.21.15.45:0/2508359913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 0x564fcf979a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.916+0000 7f402aa0c740 1 -- 172.21.15.45:0/2508359913 >> 172.21.15.45:0/2508359913 conn(0x564fcf974be0 msgr2=0x564fcf974fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.916+0000 7f402aa0c740 1 -- 172.21.15.45:0/2508359913 shutdown_connections 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.916+0000 7f402aa0c740 1 -- 172.21.15.45:0/2508359913 wait complete. 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.916+0000 7f402aa0c740 1 -- start start 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f402aa0c740 1 Processor -- start 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f402aa0c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf979670 0x564fcf9837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f402aa0c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf97a060 0x564fcf983d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f402aa0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 0x564fcf98ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f402aa0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf89a5c0 con 0x564fcf97f1d0 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f402aa0c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf89a260 con 0x564fcf97a060 2024-01-31T05:22:27.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f402aa0c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf85f2d0 con 0x564fcf979670 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f402aa0c740 10 client.0 _ll_register_callbacks cb 0x564fcf895230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f4026108640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 0x564fcf98ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f4026108640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 0x564fcf98ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58660/0 (socket says 172.21.15.45:58660) 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f4026108640 1 -- 172.21.15.45:0/3620952550 learned_addr learned my addr 172.21.15.45:0/3620952550 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f4025106640 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf97a060 0x564fcf983d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f4026108640 1 -- 172.21.15.45:0/3620952550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf979670 msgr2=0x564fcf9837f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f4025907640 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf979670 0x564fcf9837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f4026108640 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf979670 0x564fcf9837f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f4026108640 1 -- 172.21.15.45:0/3620952550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf97a060 msgr2=0x564fcf983d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f4026108640 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf97a060 0x564fcf983d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f4026108640 1 -- 172.21.15.45:0/3620952550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401000da70 con 0x564fcf97f1d0 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f4025106640 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf97a060 0x564fcf983d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f4026108640 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 0x564fcf98ad70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f401800be50 tx=0x7f4018007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165d2d1f2506408d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.917+0000 7f40097fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f40097fa640 1 -- 172.21.15.45:0/3620952550 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018009e80 con 0x564fcf97f1d0 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f40097fa640 1 -- 172.21.15.45:0/3620952550 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4018018400 con 0x564fcf97f1d0 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f402aa0c740 1 -- 172.21.15.45:0/3620952550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fcf8d9e70 con 0x564fcf97f1d0 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f40097fa640 1 -- 172.21.15.45:0/3620952550 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40180185c0 con 0x564fcf97f1d0 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f402aa0c740 10 client.15243 fetch_fsmap learned FSMap version 20 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f402aa0c740 1 -- 172.21.15.45:0/3620952550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564fcf8df660 con 0x564fcf97f1d0 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f40097fa640 1 -- 172.21.15.45:0/3620952550 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564fcf8d9e70 con 0x564fcf97f1d0 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f40097fa640 1 -- 172.21.15.45:0/3620952550 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4018021020 con 0x564fcf97f1d0 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f402aa0c740 10 client.15243 fetch_fsmap finished waiting for FSMap version 20 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f402aa0c740 10 client.15243 resolve_mds: resolved name 'i' to daemon mds.i 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f402aa0c740 20 client.15243 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f402aa0c740 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x564fcf990aa0 0x564fcf98f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f402aa0c740 4 client.15243 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:22:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.918+0000 7f402aa0c740 1 -- 172.21.15.45:0/3620952550 --> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fcf8d9b90 con 0x564fcf990aa0 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.919+0000 7f4025907640 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x564fcf990aa0 0x564fcf98f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.919+0000 7f4025907640 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x564fcf990aa0 0x564fcf98f4a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db75eafd47591761 server_cookie=183d0e1bbfdec29c in_seq=0 out_seq=0 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.919+0000 7f40097fa640 10 client.15243 ms_handle_connect on v2:172.21.15.186:6838/3821255404 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f40097fa640 1 -- 172.21.15.45:0/3620952550 <== mds.1 v2:172.21.15.186:6838/3821255404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564fcf8d9b90 con 0x564fcf990aa0 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f40097fa640 10 client.15243 handle_command_reply: tid=0 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f402aa0c740 1 client.15243 shutdown 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f402aa0c740 1 -- 172.21.15.45:0/3620952550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 msgr2=0x564fcf98ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f402aa0c740 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 0x564fcf98ad70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f401800be50 tx=0x7f4018007680 comp rx=0 tx=0).stop 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f402aa0c740 1 -- 172.21.15.45:0/3620952550 shutdown_connections 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f402aa0c740 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x564fcf990aa0 0x564fcf98f4a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f402aa0c740 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 0x564fcf98ad70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f402aa0c740 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf97a060 0x564fcf983d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f40097fa640 0 client.15243 ms_handle_reset on v2:172.21.15.186:6838/3821255404 2024-01-31T05:22:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f402aa0c740 1 --2- 172.21.15.45:0/3620952550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf979670 0x564fcf9837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f402aa0c740 1 -- 172.21.15.45:0/3620952550 >> 172.21.15.45:0/3620952550 conn(0x564fcf974be0 msgr2=0x564fcf973d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f402aa0c740 1 -- 172.21.15.45:0/3620952550 shutdown_connections 2024-01-31T05:22:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f402aa0c740 1 -- 172.21.15.45:0/3620952550 wait complete. 2024-01-31T05:22:27.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.920+0000 7f402aa0c740 20 client.15243 trim_cache size 0 max 16384 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f402aa0c740 1 Processor -- start 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f402aa0c740 1 -- start start 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f402aa0c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 0x564fcf9837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f402aa0c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97a060 0x564fcf983d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f402aa0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 0x564fcf98ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f402aa0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf8d9b90 con 0x564fcf97f1d0 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f402aa0c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf8d9e70 con 0x564fcf979670 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f402aa0c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf981b20 con 0x564fcf97a060 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f4026108640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 0x564fcf98ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f4026108640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 0x564fcf98ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58664/0 (socket says 172.21.15.45:58664) 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f4026108640 1 -- 172.21.15.45:0/2375099750 learned_addr learned my addr 172.21.15.45:0/2375099750 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f4025106640 1 --2- 172.21.15.45:0/2375099750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97a060 0x564fcf983d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f4026108640 1 -- 172.21.15.45:0/2375099750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97a060 msgr2=0x564fcf983d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f4026108640 1 --2- 172.21.15.45:0/2375099750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97a060 0x564fcf983d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f4026108640 1 -- 172.21.15.45:0/2375099750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 msgr2=0x564fcf9837f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f4026108640 1 --2- 172.21.15.45:0/2375099750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 0x564fcf9837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f4026108640 1 -- 172.21.15.45:0/2375099750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fcf87c360 con 0x564fcf97f1d0 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.936+0000 7f4025106640 1 --2- 172.21.15.45:0/2375099750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97a060 0x564fcf983d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f4026108640 1 --2- 172.21.15.45:0/2375099750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 0x564fcf98ac90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f401800b830 tx=0x7f40180094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16daa811ff36244c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f400b7fe640 1 -- 172.21.15.45:0/2375099750 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401802f070 con 0x564fcf97f1d0 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f400b7fe640 1 -- 172.21.15.45:0/2375099750 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4018012040 con 0x564fcf97f1d0 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f400b7fe640 1 -- 172.21.15.45:0/2375099750 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018023a40 con 0x564fcf97f1d0 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f402aa0c740 1 -- 172.21.15.45:0/2375099750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 msgr2=0x564fcf98ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f402aa0c740 1 --2- 172.21.15.45:0/2375099750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 0x564fcf98ac90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f401800b830 tx=0x7f40180094b0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f402aa0c740 1 -- 172.21.15.45:0/2375099750 shutdown_connections 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f402aa0c740 1 --2- 172.21.15.45:0/2375099750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97f1d0 0x564fcf98ac90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f402aa0c740 1 --2- 172.21.15.45:0/2375099750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97a060 0x564fcf983d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f402aa0c740 1 --2- 172.21.15.45:0/2375099750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 0x564fcf9837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f402aa0c740 1 -- 172.21.15.45:0/2375099750 >> 172.21.15.45:0/2375099750 conn(0x564fcf974be0 msgr2=0x564fcf97b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f402aa0c740 1 -- 172.21.15.45:0/2375099750 shutdown_connections 2024-01-31T05:22:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.937+0000 7f402aa0c740 1 -- 172.21.15.45:0/2375099750 wait complete. 2024-01-31T05:22:27.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f402aa0c740 1 -- start start 2024-01-31T05:22:27.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f402aa0c740 1 Processor -- start 2024-01-31T05:22:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f402aa0c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 0x564fcf9883e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f402aa0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 0x564fcf988920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f402aa0c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97f1d0 0x564fcf989930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f402aa0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf989e70 con 0x564fcf97a060 2024-01-31T05:22:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f402aa0c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf989fe0 con 0x564fcf979670 2024-01-31T05:22:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f402aa0c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf98a1a0 con 0x564fcf97f1d0 2024-01-31T05:22:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f402aa0c740 10 client.0 _ll_register_callbacks cb 0x7f4010013070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f4025106640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 0x564fcf988920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f4025106640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 0x564fcf988920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58666/0 (socket says 172.21.15.45:58666) 2024-01-31T05:22:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f4025106640 1 -- 172.21.15.45:0/3387176363 learned_addr learned my addr 172.21.15.45:0/3387176363 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f4025907640 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 0x564fcf9883e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f4025106640 1 -- 172.21.15.45:0/3387176363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97f1d0 msgr2=0x564fcf989930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f4026108640 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97f1d0 0x564fcf989930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f4025106640 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97f1d0 0x564fcf989930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f4025106640 1 -- 172.21.15.45:0/3387176363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 msgr2=0x564fcf9883e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f4025106640 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 0x564fcf9883e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f4025106640 1 -- 172.21.15.45:0/3387176363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564fcf8df660 con 0x564fcf97a060 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.938+0000 7f4025907640 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 0x564fcf9883e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f4025106640 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 0x564fcf988920 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f401000dd60 tx=0x7f401001cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b534e0ae70450501 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f40097fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f40097fa640 1 -- 172.21.15.45:0/3387176363 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010025860 con 0x564fcf97a060 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f40097fa640 1 -- 172.21.15.45:0/3387176363 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f40100259c0 con 0x564fcf97a060 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f402aa0c740 1 -- 172.21.15.45:0/3387176363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fcf9907b0 con 0x564fcf97a060 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f40097fa640 1 -- 172.21.15.45:0/3387176363 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4010025b80 con 0x564fcf97a060 2024-01-31T05:22:27.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f402aa0c740 10 client.15249 fetch_fsmap learned FSMap version 20 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f402aa0c740 1 -- 172.21.15.45:0/3387176363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564fcf8a1eb0 con 0x564fcf97a060 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f40097fa640 1 -- 172.21.15.45:0/3387176363 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564fcf9907b0 con 0x564fcf97a060 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f40097fa640 1 -- 172.21.15.45:0/3387176363 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f401000b050 con 0x564fcf97a060 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f402aa0c740 10 client.15249 fetch_fsmap finished waiting for FSMap version 20 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f402aa0c740 10 client.15249 resolve_mds: resolved name 'i' to daemon mds.i 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f402aa0c740 20 client.15249 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f402aa0c740 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x564fcf9eba20 0x564fcf9e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f402aa0c740 4 client.15249 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.939+0000 7f402aa0c740 1 -- 172.21.15.45:0/3387176363 --> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f401001cc70 con 0x564fcf9eba20 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f4025907640 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x564fcf9eba20 0x564fcf9e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f4025907640 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x564fcf9eba20 0x564fcf9e7ec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1576be5177b70fba server_cookie=4675e395318a56f1 in_seq=0 out_seq=0 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f40097fa640 10 client.15249 ms_handle_connect on v2:172.21.15.186:6838/3821255404 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f40097fa640 1 -- 172.21.15.45:0/3387176363 <== mds.1 v2:172.21.15.186:6838/3821255404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7f401001cc70 con 0x564fcf9eba20 2024-01-31T05:22:27.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f40097fa640 10 client.15249 handle_command_reply: tid=0 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f402aa0c740 1 client.15249 shutdown 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f402aa0c740 1 -- 172.21.15.45:0/3387176363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 msgr2=0x564fcf988920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f402aa0c740 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 0x564fcf988920 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f401000dd60 tx=0x7f401001cb30 comp rx=0 tx=0).stop 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f402aa0c740 1 -- 172.21.15.45:0/3387176363 shutdown_connections 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f402aa0c740 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.186:6838/3821255404,v1:172.21.15.186:6839/3821255404] conn(0x564fcf9eba20 0x564fcf9e7ec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f402aa0c740 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fcf97a060 0x564fcf988920 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f402aa0c740 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564fcf97f1d0 0x564fcf989930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f402aa0c740 1 --2- 172.21.15.45:0/3387176363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fcf979670 0x564fcf9883e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f40097fa640 0 client.15249 ms_handle_reset on v2:172.21.15.186:6838/3821255404 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.940+0000 7f402aa0c740 1 -- 172.21.15.45:0/3387176363 >> 172.21.15.45:0/3387176363 conn(0x564fcf974be0 msgr2=0x564fcf97b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f402aa0c740 1 -- 172.21.15.45:0/3387176363 shutdown_connections 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f402aa0c740 1 -- 172.21.15.45:0/3387176363 wait complete. 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f402aa0c740 20 client.15249 trim_cache size 0 max 16384 2024-01-31T05:22:27.945 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-01-31T05:22:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "3a1f4122-bff7-11ee-95b3-87774f69a715", 2024-01-31T05:22:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2024-01-31T05:22:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24490, 2024-01-31T05:22:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-01-31T05:22:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-01-31T05:22:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-01-31T05:22:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 105.485006117, 2024-01-31T05:22:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 20, 2024-01-31T05:22:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 88, 2024-01-31T05:22:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 86, 2024-01-31T05:22:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 121.125064714 2024-01-31T05:22:27.946 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-01-31T05:22:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f4027b6b640 1 -- 172.21.15.45:0/4210553291 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f400c0a1360 msgr2=0x7f400c0a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f4027b6b640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f400c0a1360 0x7f400c0a3820 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4020100eb0 tx=0x7f4010005d30 comp rx=0 tx=0).stop 2024-01-31T05:22:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f4027b6b640 1 -- 172.21.15.45:0/4210553291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40201017d0 msgr2=0x7f4020101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f4027b6b640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40201017d0 0x7f4020101140 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f401c009430 tx=0x7f401c0042f0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f4027b6b640 1 -- 172.21.15.45:0/4210553291 shutdown_connections 2024-01-31T05:22:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f4027b6b640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f400c0a1360 0x7f400c0a3820 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f4027b6b640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4020106a30 0x7f40200ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f4027b6b640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4020104100 0x7f40200ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f4027b6b640 1 --2- 172.21.15.45:0/4210553291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40201017d0 0x7f4020101140 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f4027b6b640 1 -- 172.21.15.45:0/4210553291 >> 172.21.15.45:0/4210553291 conn(0x7f40200fb180 msgr2=0x7f4020102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:27.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f4027b6b640 1 -- 172.21.15.45:0/4210553291 shutdown_connections 2024-01-31T05:22:27.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:27.941+0000 7f4027b6b640 1 -- 172.21.15.45:0/4210553291 wait complete. 2024-01-31T05:22:27.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-31T05:22:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.055+0000 7ff290b5e640 1 -- 172.21.15.45:0/1057381752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28c1069f0 msgr2=0x7ff28c108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.055+0000 7ff290b5e640 1 --2- 172.21.15.45:0/1057381752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28c1069f0 0x7ff28c108e40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff284009ca0 tx=0x7ff28402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.055+0000 7ff290b5e640 1 -- 172.21.15.45:0/1057381752 shutdown_connections 2024-01-31T05:22:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.055+0000 7ff290b5e640 1 --2- 172.21.15.45:0/1057381752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28c1069f0 0x7ff28c108e40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.055+0000 7ff290b5e640 1 --2- 172.21.15.45:0/1057381752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff28c1040c0 0x7ff28c1064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.055+0000 7ff290b5e640 1 --2- 172.21.15.45:0/1057381752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff28c101790 0x7ff28c103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.055+0000 7ff290b5e640 1 -- 172.21.15.45:0/1057381752 >> 172.21.15.45:0/1057381752 conn(0x7ff28c0fb1a0 msgr2=0x7ff28c0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.055+0000 7ff290b5e640 1 -- 172.21.15.45:0/1057381752 shutdown_connections 2024-01-31T05:22:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.055+0000 7ff290b5e640 1 -- 172.21.15.45:0/1057381752 wait complete. 2024-01-31T05:22:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.055+0000 7ff290b5e640 1 Processor -- start 2024-01-31T05:22:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff290b5e640 1 -- start start 2024-01-31T05:22:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff290b5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28c101790 0x7ff28c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff290b5e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff28c1040c0 0x7ff28c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff290b5e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff28c1069f0 0x7ff28c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff290b5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c19e700 con 0x7ff28c101790 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff290b5e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c19e870 con 0x7ff28c1069f0 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff290b5e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c19ea30 con 0x7ff28c1040c0 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff28a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28c101790 0x7ff28c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff28a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28c101790 0x7ff28c196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58676/0 (socket says 172.21.15.45:58676) 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff28a59c640 1 -- 172.21.15.45:0/1342353620 learned_addr learned my addr 172.21.15.45:0/1342353620 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff281d9b640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff28c1040c0 0x7ff28c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff28a59c640 1 -- 172.21.15.45:0/1342353620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff28c1040c0 msgr2=0x7ff28c197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff28a59c640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff28c1040c0 0x7ff28c197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff28ad9d640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff28c1069f0 0x7ff28c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff28a59c640 1 -- 172.21.15.45:0/1342353620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff28c1069f0 msgr2=0x7ff28c19e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff28a59c640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff28c1069f0 0x7ff28c19e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff28a59c640 1 -- 172.21.15.45:0/1342353620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff284009900 con 0x7ff28c101790 2024-01-31T05:22:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.056+0000 7ff28a59c640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28c101790 0x7ff28c196c20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00c920 tx=0x7ff27c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.057+0000 7ff283fff640 1 -- 172.21.15.45:0/1342353620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c007c70 con 0x7ff28c101790 2024-01-31T05:22:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.057+0000 7ff290b5e640 1 -- 172.21.15.45:0/1342353620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28c19ed10 con 0x7ff28c101790 2024-01-31T05:22:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.057+0000 7ff283fff640 1 -- 172.21.15.45:0/1342353620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff27c007dd0 con 0x7ff28c101790 2024-01-31T05:22:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.057+0000 7ff283fff640 1 -- 172.21.15.45:0/1342353620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c0187c0 con 0x7ff28c101790 2024-01-31T05:22:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.057+0000 7ff290b5e640 1 -- 172.21.15.45:0/1342353620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28c19f1c0 con 0x7ff28c101790 2024-01-31T05:22:28.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.059+0000 7ff283fff640 1 -- 172.21.15.45:0/1342353620 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7ff27c00f450 con 0x7ff28c101790 2024-01-31T05:22:28.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.059+0000 7ff293a03740 1 Processor -- start 2024-01-31T05:22:28.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.059+0000 7ff293a03740 1 -- start start 2024-01-31T05:22:28.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff293a03740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e3670 0x5641aa0e3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff293a03740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e4060 0x5641aa0e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff293a03740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e91d0 0x5641aa0eb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff293a03740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641aa052000 con 0x5641aa0e91d0 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff293a03740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641aa04f6f0 con 0x5641aa0e3670 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff293a03740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641aa04afe0 con 0x5641aa0e4060 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff281d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e91d0 0x5641aa0eb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff283fff640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ff2780a12e0 0x7ff2780a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff281d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e91d0 0x5641aa0eb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58686/0 (socket says 172.21.15.45:58686) 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff281d9b640 1 -- 172.21.15.45:0/2456817106 learned_addr learned my addr 172.21.15.45:0/2456817106 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff28a59c640 1 --2- 172.21.15.45:0/2456817106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e4060 0x5641aa0e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff28ad9d640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ff2780a12e0 0x7ff2780a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff28ad9d640 1 --2- 172.21.15.45:0/2456817106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e3670 0x5641aa0e3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff281d9b640 1 -- 172.21.15.45:0/2456817106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e4060 msgr2=0x5641aa0e8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff281d9b640 1 --2- 172.21.15.45:0/2456817106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e4060 0x5641aa0e8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff283fff640 1 -- 172.21.15.45:0/1342353620 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff27c0cce20 con 0x7ff28c101790 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff281d9b640 1 -- 172.21.15.45:0/2456817106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e3670 msgr2=0x5641aa0e3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff281d9b640 1 --2- 172.21.15.45:0/2456817106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e3670 0x5641aa0e3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff281d9b640 1 -- 172.21.15.45:0/2456817106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641aa00d650 con 0x5641aa0e91d0 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff281d9b640 1 --2- 172.21.15.45:0/2456817106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e91d0 0x5641aa0eb660 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff27400bc80 tx=0x7ff27400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400c40f3c8c8a92e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff28ad9d640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ff2780a12e0 0x7ff2780a37a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5641aa0e48c0 tx=0x7ff284009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.060+0000 7ff280d99640 1 -- 172.21.15.45:0/2456817106 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27401a460 con 0x5641aa0e91d0 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.061+0000 7ff280d99640 1 -- 172.21.15.45:0/2456817106 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff274011900 con 0x5641aa0e91d0 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.061+0000 7ff280d99640 1 -- 172.21.15.45:0/2456817106 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274010770 con 0x5641aa0e91d0 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.061+0000 7ff293a03740 1 -- 172.21.15.45:0/2456817106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e91d0 msgr2=0x5641aa0eb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.061+0000 7ff293a03740 1 --2- 172.21.15.45:0/2456817106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e91d0 0x5641aa0eb660 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff27400bc80 tx=0x7ff27400f730 comp rx=0 tx=0).stop 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.061+0000 7ff293a03740 1 -- 172.21.15.45:0/2456817106 shutdown_connections 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.061+0000 7ff293a03740 1 --2- 172.21.15.45:0/2456817106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e91d0 0x5641aa0eb660 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.061+0000 7ff293a03740 1 --2- 172.21.15.45:0/2456817106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e4060 0x5641aa0e8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.061+0000 7ff293a03740 1 --2- 172.21.15.45:0/2456817106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e3670 0x5641aa0e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.061+0000 7ff293a03740 1 -- 172.21.15.45:0/2456817106 >> 172.21.15.45:0/2456817106 conn(0x5641aa0debe0 msgr2=0x5641aa0defe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.061+0000 7ff293a03740 1 -- 172.21.15.45:0/2456817106 shutdown_connections 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.061+0000 7ff293a03740 1 -- 172.21.15.45:0/2456817106 wait complete. 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff293a03740 1 -- start start 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff293a03740 1 Processor -- start 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff293a03740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e3670 0x5641aa0ed8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff293a03740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e4060 0x5641aa0ede00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff293a03740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e91d0 0x5641aa0f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff293a03740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641aa0045c0 con 0x5641aa0e4060 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff293a03740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641aa004260 con 0x5641aa0e91d0 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff293a03740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a9fc92d0 con 0x5641aa0e3670 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff293a03740 10 client.0 _ll_register_callbacks cb 0x5641a9fff230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff281d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e4060 0x5641aa0ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff281d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e4060 0x5641aa0ede00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58700/0 (socket says 172.21.15.45:58700) 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff281d9b640 1 -- 172.21.15.45:0/3477461650 learned_addr learned my addr 172.21.15.45:0/3477461650 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff28ad9d640 1 --2- 172.21.15.45:0/3477461650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e91d0 0x5641aa0f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff281d9b640 1 -- 172.21.15.45:0/3477461650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e3670 msgr2=0x5641aa0ed8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff281d9b640 1 --2- 172.21.15.45:0/3477461650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e3670 0x5641aa0ed8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff281d9b640 1 -- 172.21.15.45:0/3477461650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e91d0 msgr2=0x5641aa0f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff281d9b640 1 --2- 172.21.15.45:0/3477461650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e91d0 0x5641aa0f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff281d9b640 1 -- 172.21.15.45:0/3477461650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27400b8e0 con 0x5641aa0e4060 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff281d9b640 1 --2- 172.21.15.45:0/3477461650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e4060 0x5641aa0ede00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff274005780 tx=0x7ff274011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c56d61340a751dd server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.062+0000 7ff271ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff271ffb640 1 -- 172.21.15.45:0/3477461650 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274020070 con 0x5641aa0e4060 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff271ffb640 1 -- 172.21.15.45:0/3477461650 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff274011c80 con 0x5641aa0e4060 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff293a03740 1 -- 172.21.15.45:0/3477461650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641aa043e70 con 0x5641aa0e4060 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff271ffb640 1 -- 172.21.15.45:0/3477461650 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff274004040 con 0x5641aa0e4060 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff293a03740 10 client.15261 fetch_fsmap learned FSMap version 20 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff293a03740 1 -- 172.21.15.45:0/3477461650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5641aa049660 con 0x5641aa0e4060 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff271ffb640 1 -- 172.21.15.45:0/3477461650 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5641aa043e70 con 0x5641aa0e4060 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff271ffb640 1 -- 172.21.15.45:0/3477461650 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff2740104b0 con 0x5641aa0e4060 2024-01-31T05:22:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff293a03740 10 client.15261 fetch_fsmap finished waiting for FSMap version 20 2024-01-31T05:22:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff293a03740 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-31T05:22:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff293a03740 20 client.15261 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff293a03740 1 --2- 172.21.15.45:0/3477461650 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5641aa0fa930 0x5641aa0f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff293a03740 4 client.15261 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:22:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.063+0000 7ff293a03740 1 -- 172.21.15.45:0/3477461650 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641aa043b90 con 0x5641aa0fa930 2024-01-31T05:22:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.064+0000 7ff28a59c640 1 --2- 172.21.15.45:0/3477461650 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5641aa0fa930 0x5641aa0f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.064+0000 7ff28a59c640 1 --2- 172.21.15.45:0/3477461650 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5641aa0fa930 0x5641aa0f93c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38bca49d6ec4eb86 server_cookie=bb833e7fdcee5e88 in_seq=0 out_seq=0 2024-01-31T05:22:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.064+0000 7ff271ffb640 10 client.15261 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:22:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff271ffb640 1 -- 172.21.15.45:0/3477461650 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5641aa043b90 con 0x5641aa0fa930 2024-01-31T05:22:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff271ffb640 10 client.15261 handle_command_reply: tid=0 2024-01-31T05:22:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff293a03740 1 client.15261 shutdown 2024-01-31T05:22:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff293a03740 1 -- 172.21.15.45:0/3477461650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e4060 msgr2=0x5641aa0ede00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff293a03740 1 --2- 172.21.15.45:0/3477461650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e4060 0x5641aa0ede00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff274005780 tx=0x7ff274011670 comp rx=0 tx=0).stop 2024-01-31T05:22:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff293a03740 1 -- 172.21.15.45:0/3477461650 shutdown_connections 2024-01-31T05:22:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff293a03740 1 --2- 172.21.15.45:0/3477461650 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5641aa0fa930 0x5641aa0f93c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff293a03740 1 --2- 172.21.15.45:0/3477461650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0e4060 0x5641aa0ede00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff293a03740 1 --2- 172.21.15.45:0/3477461650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e91d0 0x5641aa0f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff293a03740 1 --2- 172.21.15.45:0/3477461650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e3670 0x5641aa0ed8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff271ffb640 0 client.15261 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:22:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff293a03740 1 -- 172.21.15.45:0/3477461650 >> 172.21.15.45:0/3477461650 conn(0x5641aa0debe0 msgr2=0x5641aa0ddd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff293a03740 1 -- 172.21.15.45:0/3477461650 shutdown_connections 2024-01-31T05:22:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff293a03740 1 -- 172.21.15.45:0/3477461650 wait complete. 2024-01-31T05:22:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.065+0000 7ff293a03740 20 client.15261 trim_cache size 0 max 16384 2024-01-31T05:22:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff293a03740 1 Processor -- start 2024-01-31T05:22:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff293a03740 1 -- start start 2024-01-31T05:22:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff293a03740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 0x5641aa0f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff293a03740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e3670 0x5641aa0f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff293a03740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e4060 0x5641aa0f9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff293a03740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641aa0e4750 con 0x5641aa0debe0 2024-01-31T05:22:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff293a03740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641aa0ebc90 con 0x5641aa0e4060 2024-01-31T05:22:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff293a03740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a9feeff0 con 0x5641aa0e3670 2024-01-31T05:22:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff28a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 0x5641aa0f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff28a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 0x5641aa0f93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58704/0 (socket says 172.21.15.45:58704) 2024-01-31T05:22:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff28a59c640 1 -- 172.21.15.45:0/1253507326 learned_addr learned my addr 172.21.15.45:0/1253507326 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff28a59c640 1 -- 172.21.15.45:0/1253507326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e3670 msgr2=0x5641aa0f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff28a59c640 1 --2- 172.21.15.45:0/1253507326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e3670 0x5641aa0f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff28a59c640 1 -- 172.21.15.45:0/1253507326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e4060 msgr2=0x5641aa0f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff28a59c640 1 --2- 172.21.15.45:0/1253507326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e4060 0x5641aa0f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff28a59c640 1 -- 172.21.15.45:0/1253507326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641a9fe6360 con 0x5641aa0debe0 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.081+0000 7ff28a59c640 1 --2- 172.21.15.45:0/1253507326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 0x5641aa0f93c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff27c007fd0 tx=0x7ff27c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b2e2a30fb1b3af server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff273fff640 1 -- 172.21.15.45:0/1253507326 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c09a630 con 0x5641aa0debe0 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff273fff640 1 -- 172.21.15.45:0/1253507326 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff27c00de20 con 0x5641aa0debe0 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff273fff640 1 -- 172.21.15.45:0/1253507326 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00f700 con 0x5641aa0debe0 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff293a03740 1 -- 172.21.15.45:0/1253507326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 msgr2=0x5641aa0f93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff293a03740 1 --2- 172.21.15.45:0/1253507326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 0x5641aa0f93c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff27c007fd0 tx=0x7ff27c000bc0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff293a03740 1 -- 172.21.15.45:0/1253507326 shutdown_connections 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff293a03740 1 --2- 172.21.15.45:0/1253507326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e4060 0x5641aa0f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff293a03740 1 --2- 172.21.15.45:0/1253507326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e3670 0x5641aa0f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff293a03740 1 --2- 172.21.15.45:0/1253507326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 0x5641aa0f93c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff293a03740 1 -- 172.21.15.45:0/1253507326 >> 172.21.15.45:0/1253507326 conn(0x5641aa0df330 msgr2=0x5641aa0e5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff293a03740 1 -- 172.21.15.45:0/1253507326 shutdown_connections 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff293a03740 1 -- 172.21.15.45:0/1253507326 wait complete. 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff293a03740 1 -- start start 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.082+0000 7ff293a03740 1 Processor -- start 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff293a03740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 0x5641aa0f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff293a03740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e3670 0x5641aa0f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff293a03740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e4060 0x5641aa14f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff293a03740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641aa14fdb0 con 0x5641aa0debe0 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff293a03740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641aa14fef0 con 0x5641aa0e3670 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff293a03740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641aa150030 con 0x5641aa0e4060 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff293a03740 10 client.0 _ll_register_callbacks cb 0x7ff27400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff28a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 0x5641aa0f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff28a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 0x5641aa0f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58706/0 (socket says 172.21.15.45:58706) 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff28a59c640 1 -- 172.21.15.45:0/767918778 learned_addr learned my addr 172.21.15.45:0/767918778 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff28a59c640 1 -- 172.21.15.45:0/767918778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e4060 msgr2=0x5641aa14f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff28a59c640 1 --2- 172.21.15.45:0/767918778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e4060 0x5641aa14f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff28a59c640 1 -- 172.21.15.45:0/767918778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e3670 msgr2=0x5641aa0f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff28a59c640 1 --2- 172.21.15.45:0/767918778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e3670 0x5641aa0f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff28a59c640 1 -- 172.21.15.45:0/767918778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28c19f1c0 con 0x5641aa0debe0 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff28a59c640 1 --2- 172.21.15.45:0/767918778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 0x5641aa0f1db0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00c8f0 tx=0x7ff27c0a22e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47624ffbf15b695 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff271ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff271ffb640 1 -- 172.21.15.45:0/767918778 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c09a630 con 0x5641aa0debe0 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff271ffb640 1 -- 172.21.15.45:0/767918778 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff27c099f30 con 0x5641aa0debe0 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff293a03740 1 -- 172.21.15.45:0/767918778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff274011900 con 0x5641aa0debe0 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff271ffb640 1 -- 172.21.15.45:0/767918778 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff27c0965b0 con 0x5641aa0debe0 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff293a03740 10 client.15267 fetch_fsmap learned FSMap version 20 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.083+0000 7ff293a03740 1 -- 172.21.15.45:0/767918778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5641aa00beb0 con 0x5641aa0debe0 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.084+0000 7ff271ffb640 1 -- 172.21.15.45:0/767918778 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff274011900 con 0x5641aa0debe0 2024-01-31T05:22:28.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.084+0000 7ff271ffb640 1 -- 172.21.15.45:0/767918778 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff27c016020 con 0x5641aa0debe0 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.084+0000 7ff293a03740 10 client.15267 fetch_fsmap finished waiting for FSMap version 20 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.084+0000 7ff293a03740 10 client.15267 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.084+0000 7ff293a03740 20 client.15267 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.084+0000 7ff293a03740 1 --2- 172.21.15.45:0/767918778 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5641aa0e91d0 0x5641aa151320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.084+0000 7ff293a03740 4 client.15267 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.084+0000 7ff293a03740 1 -- 172.21.15.45:0/767918778 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5641a9d4a6c0 con 0x5641aa0e91d0 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.084+0000 7ff281d9b640 1 --2- 172.21.15.45:0/767918778 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5641aa0e91d0 0x5641aa151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff271ffb640 10 client.15267 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff281d9b640 1 --2- 172.21.15.45:0/767918778 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5641aa0e91d0 0x5641aa151320 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ad013ba9a46a4f server_cookie=9e4c2f9cbe65884d in_seq=0 out_seq=0 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff271ffb640 1 -- 172.21.15.45:0/767918778 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5641a9d4a6c0 con 0x5641aa0e91d0 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff271ffb640 10 client.15267 handle_command_reply: tid=0 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff293a03740 1 client.15267 shutdown 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff293a03740 1 -- 172.21.15.45:0/767918778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 msgr2=0x5641aa0f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:28.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff293a03740 1 --2- 172.21.15.45:0/767918778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 0x5641aa0f1db0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00c8f0 tx=0x7ff27c0a22e0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff293a03740 1 -- 172.21.15.45:0/767918778 shutdown_connections 2024-01-31T05:22:28.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff293a03740 1 --2- 172.21.15.45:0/767918778 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5641aa0e91d0 0x5641aa151320 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff293a03740 1 --2- 172.21.15.45:0/767918778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5641aa0e4060 0x5641aa14f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff293a03740 1 --2- 172.21.15.45:0/767918778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641aa0e3670 0x5641aa0f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff293a03740 1 --2- 172.21.15.45:0/767918778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641aa0debe0 0x5641aa0f1db0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff271ffb640 0 client.15267 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:22:28.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff293a03740 1 -- 172.21.15.45:0/767918778 >> 172.21.15.45:0/767918778 conn(0x5641aa0df330 msgr2=0x5641aa0f93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:28.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff293a03740 1 -- 172.21.15.45:0/767918778 shutdown_connections 2024-01-31T05:22:28.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff293a03740 1 -- 172.21.15.45:0/767918778 wait complete. 2024-01-31T05:22:28.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.085+0000 7ff293a03740 20 client.15267 trim_cache size 0 max 16384 2024-01-31T05:22:28.090 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-01-31T05:22:28.091 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "3a1f4122-bff7-11ee-95b3-87774f69a715", 2024-01-31T05:22:28.091 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-01-31T05:22:28.091 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24503, 2024-01-31T05:22:28.091 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-01-31T05:22:28.091 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-01-31T05:22:28.091 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-01-31T05:22:28.091 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 107.388474035, 2024-01-31T05:22:28.091 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 20, 2024-01-31T05:22:28.091 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 88, 2024-01-31T05:22:28.091 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 86, 2024-01-31T05:22:28.091 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 113.08860995800001 2024-01-31T05:22:28.091 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-01-31T05:22:28.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.086+0000 7ff290b5e640 1 -- 172.21.15.45:0/1342353620 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ff2780a12e0 msgr2=0x7ff2780a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:28.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.086+0000 7ff290b5e640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ff2780a12e0 0x7ff2780a37a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5641aa0e48c0 tx=0x7ff284009360 comp rx=0 tx=0).stop 2024-01-31T05:22:28.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.086+0000 7ff290b5e640 1 -- 172.21.15.45:0/1342353620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28c101790 msgr2=0x7ff28c196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:28.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.086+0000 7ff290b5e640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28c101790 0x7ff28c196c20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00c920 tx=0x7ff27c00cde0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.086+0000 7ff290b5e640 1 -- 172.21.15.45:0/1342353620 shutdown_connections 2024-01-31T05:22:28.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.086+0000 7ff290b5e640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7ff2780a12e0 0x7ff2780a37a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.086+0000 7ff290b5e640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff28c1069f0 0x7ff28c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.086+0000 7ff290b5e640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff28c1040c0 0x7ff28c197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.086+0000 7ff290b5e640 1 --2- 172.21.15.45:0/1342353620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28c101790 0x7ff28c196c20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.086+0000 7ff290b5e640 1 -- 172.21.15.45:0/1342353620 >> 172.21.15.45:0/1342353620 conn(0x7ff28c0fb1a0 msgr2=0x7ff28c1020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.086+0000 7ff290b5e640 1 -- 172.21.15.45:0/1342353620 shutdown_connections 2024-01-31T05:22:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:28.086+0000 7ff290b5e640 1 -- 172.21.15.45:0/1342353620 wait complete. 2024-01-31T05:22:28.095 INFO:teuthology.run_tasks:Running task kclient... 2024-01-31T05:22:28.116 INFO:tasks.kclient:Mounting kernel clients... 2024-01-31T05:22:28.116 INFO:tasks.kclient:config is {'client.0': {}} 2024-01-31T05:22:28.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:28.116 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-01-31T05:22:28.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:28.131 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-01-31T05:22:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-01-31T05:22:28.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:22:28.200 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-01-31T05:22:28.201 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-31T05:22:28.201 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-31T05:22:28.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph fs ls 2024-01-31T05:22:28.481 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:22:28.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:28 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1143987580' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:28.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:28 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1143987580' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:28 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1143987580' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:22:29.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:29 smithi045 ceph-mon[31585]: pgmap v339: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:22:29.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:29 smithi186 ceph-mon[35659]: pgmap v339: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:22:29.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:29 smithi156 ceph-mon[35720]: pgmap v339: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-31T05:22:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.139+0000 7fe986214700 1 -- 172.21.15.45:0/4064551982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe980069ee0 msgr2=0x7fe98006a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.139+0000 7fe986214700 1 --2- 172.21.15.45:0/4064551982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe980069ee0 0x7fe98006a360 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe9740099c0 tx=0x7fe974009cd0 comp rx=0 tx=0).stop 2024-01-31T05:22:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.139+0000 7fe986214700 1 -- 172.21.15.45:0/4064551982 shutdown_connections 2024-01-31T05:22:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.139+0000 7fe986214700 1 --2- 172.21.15.45:0/4064551982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe98006a8a0 0x7fe98010e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.139+0000 7fe986214700 1 --2- 172.21.15.45:0/4064551982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe980107790 0x7fe980107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.139+0000 7fe986214700 1 --2- 172.21.15.45:0/4064551982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe980069ee0 0x7fe98006a360 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.139+0000 7fe986214700 1 -- 172.21.15.45:0/4064551982 >> 172.21.15.45:0/4064551982 conn(0x7fe980076700 msgr2=0x7fe980076b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.139+0000 7fe986214700 1 -- 172.21.15.45:0/4064551982 shutdown_connections 2024-01-31T05:22:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.139+0000 7fe986214700 1 -- 172.21.15.45:0/4064551982 wait complete. 2024-01-31T05:22:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.140+0000 7fe986214700 1 Processor -- start 2024-01-31T05:22:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.140+0000 7fe986214700 1 -- start start 2024-01-31T05:22:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.140+0000 7fe986214700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe980069ee0 0x7fe98019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.140+0000 7fe986214700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe98006a8a0 0x7fe98019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.140+0000 7fe986214700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980107790 0x7fe980197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.140+0000 7fe986214700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9801980a0 con 0x7fe980069ee0 2024-01-31T05:22:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.140+0000 7fe986214700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9801981e0 con 0x7fe980107790 2024-01-31T05:22:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.140+0000 7fe986214700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9801983a0 con 0x7fe98006a8a0 2024-01-31T05:22:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.140+0000 7fe97f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe980069ee0 0x7fe98019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.140+0000 7fe97f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe980069ee0 0x7fe98019d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58724/0 (socket says 172.21.15.45:58724) 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.140+0000 7fe97f7fe700 1 -- 172.21.15.45:0/1917119466 learned_addr learned my addr 172.21.15.45:0/1917119466 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.141+0000 7fe97effd700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe98006a8a0 0x7fe98019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.141+0000 7fe97ffff700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980107790 0x7fe980197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.141+0000 7fe97f7fe700 1 -- 172.21.15.45:0/1917119466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe98006a8a0 msgr2=0x7fe98019ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.141+0000 7fe97f7fe700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe98006a8a0 0x7fe98019ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.141+0000 7fe97f7fe700 1 -- 172.21.15.45:0/1917119466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980107790 msgr2=0x7fe980197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.141+0000 7fe97f7fe700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980107790 0x7fe980197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.141+0000 7fe97f7fe700 1 -- 172.21.15.45:0/1917119466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9740095f0 con 0x7fe980069ee0 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.141+0000 7fe97f7fe700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe980069ee0 0x7fe98019d880 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe97000b530 tx=0x7fe97000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.141+0000 7fe97cff9700 1 -- 172.21.15.45:0/1917119466 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970011830 con 0x7fe980069ee0 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.141+0000 7fe986214700 1 -- 172.21.15.45:0/1917119466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe980198680 con 0x7fe980069ee0 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.141+0000 7fe97cff9700 1 -- 172.21.15.45:0/1917119466 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe970011e50 con 0x7fe980069ee0 2024-01-31T05:22:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.141+0000 7fe97cff9700 1 -- 172.21.15.45:0/1917119466 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97000f2e0 con 0x7fe980069ee0 2024-01-31T05:22:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.142+0000 7fe986214700 1 -- 172.21.15.45:0/1917119466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9801a4920 con 0x7fe980069ee0 2024-01-31T05:22:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.143+0000 7fe986214700 1 -- 172.21.15.45:0/1917119466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe980109d20 con 0x7fe980069ee0 2024-01-31T05:22:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.143+0000 7fe97cff9700 1 -- 172.21.15.45:0/1917119466 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe9700105a0 con 0x7fe980069ee0 2024-01-31T05:22:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.144+0000 7fe97cff9700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9680a1370 0x7fe9680a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.144+0000 7fe97effd700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9680a1370 0x7fe9680a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.144+0000 7fe97cff9700 1 -- 172.21.15.45:0/1917119466 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe9700cc4c0 con 0x7fe980069ee0 2024-01-31T05:22:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.144+0000 7fe97effd700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9680a1370 0x7fe9680a3830 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe974009990 tx=0x7fe974003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.149+0000 7fe97cff9700 1 -- 172.21.15.45:0/1917119466 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe970095970 con 0x7fe980069ee0 2024-01-31T05:22:30.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.393+0000 7fe986214700 1 -- 172.21.15.45:0/1917119466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe980198f70 con 0x7fe980069ee0 2024-01-31T05:22:30.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.394+0000 7fe97cff9700 1 -- 172.21.15.45:0/1917119466 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe970098f90 con 0x7fe980069ee0 2024-01-31T05:22:30.393 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-31T05:22:30.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.396+0000 7fe986214700 1 -- 172.21.15.45:0/1917119466 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9680a1370 msgr2=0x7fe9680a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:30.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.396+0000 7fe986214700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9680a1370 0x7fe9680a3830 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe974009990 tx=0x7fe974003680 comp rx=0 tx=0).stop 2024-01-31T05:22:30.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.396+0000 7fe986214700 1 -- 172.21.15.45:0/1917119466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe980069ee0 msgr2=0x7fe98019d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:30.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.396+0000 7fe986214700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe980069ee0 0x7fe98019d880 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe97000b530 tx=0x7fe97000b840 comp rx=0 tx=0).stop 2024-01-31T05:22:30.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.396+0000 7fe986214700 1 -- 172.21.15.45:0/1917119466 shutdown_connections 2024-01-31T05:22:30.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.396+0000 7fe986214700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980107790 0x7fe980197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:30.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.396+0000 7fe986214700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9680a1370 0x7fe9680a3830 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:30.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.396+0000 7fe986214700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe980069ee0 0x7fe98019d880 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:30.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.396+0000 7fe986214700 1 --2- 172.21.15.45:0/1917119466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe98006a8a0 0x7fe98019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:30.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.396+0000 7fe986214700 1 -- 172.21.15.45:0/1917119466 >> 172.21.15.45:0/1917119466 conn(0x7fe980076700 msgr2=0x7fe9800ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:30.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.396+0000 7fe986214700 1 -- 172.21.15.45:0/1917119466 shutdown_connections 2024-01-31T05:22:30.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:30.396+0000 7fe986214700 1 -- 172.21.15.45:0/1917119466 wait complete. 2024-01-31T05:22:30.826 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:30 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1917119466' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-01-31T05:22:30.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:30 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1917119466' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-01-31T05:22:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:30 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1917119466' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-01-31T05:22:31.008 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-31T05:22:31.008 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-31T05:22:31.008 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-01-31T05:22:31.008 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-31T05:22:31.008 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-31T05:22:31.008 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-31T05:22:31.008 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-31T05:22:31.008 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-31T05:22:31.008 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:22:31.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:31.008 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-01-31T05:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-01-31T05:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-01-31T05:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-01-31T05:22:31.026 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-31T05:22:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-01-31T05:22:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-01-31T05:22:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-31T05:22:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42133sec preferred_lft 42133sec 2024-01-31T05:22:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-01-31T05:22:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:22:31.026 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:22:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-01-31T05:22:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-01-31T05:22:31.026 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-31T05:22:31.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:31.027 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-01-31T05:22:31.027 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-01-31T05:22:31.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-01-31T05:22:31.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-01-31T05:22:31.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-01-31T05:22:31.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-31T05:22:31.027 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-01-31T05:22:31.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:31.190 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-31T05:22:31.217 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-01-31T05:22:31.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:31.219 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-01-31T05:22:31.291 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-01-31T05:22:31.291 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-31T05:22:31.291 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-31T05:22:31.292 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-31T05:22:31.292 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-31T05:22:31.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:31.292 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-01-31T05:22:31.292 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-01-31T05:22:31.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-31T05:22:31.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-31T05:22:31.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-31T05:22:31.292 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-01-31T05:22:31.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:31.565 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-01-31T05:22:31.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:31.579 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-01-31T05:22:31.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:31.634 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-01-31T05:22:31.634 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-01-31T05:22:31.634 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:22:31.634 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-31T05:22:31.634 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-01-31T05:22:31.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:31 smithi045 ceph-mon[31585]: pgmap v340: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:22:31.683 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-31T05:22:31.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:31.684 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-01-31T05:22:31.684 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-01-31T05:22:31.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-31T05:22:31.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-31T05:22:31.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-31T05:22:31.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-31T05:22:31.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-31T05:22:31.684 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-01-31T05:22:31.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:31.817 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-01-31T05:22:31.817 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-01-31T05:22:31.818 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-01-31T05:22:31.818 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-01-31T05:22:31.818 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-01-31T05:22:31.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:31 smithi186 ceph-mon[35659]: pgmap v340: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:22:31.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:22:31.859 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-31T05:22:31.917 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:22:31.917 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:22:31.917 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:22:31.972 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-01-31T05:22:31.972 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-01-31T05:22:31.972 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/0677e5c0-3673-41ea-a0ca-4130f847896f 2024-01-31T05:22:31.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:31.972 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/0677e5c0-3673-41ea-a0ca-4130f847896f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-01-31T05:22:31.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:31 smithi156 ceph-mon[35720]: pgmap v340: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:22:32.077 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-01-31T05:22:32.078 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-01-31T05:22:32.188 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-01-31T05:22:32.188 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-01-31T05:22:32.188 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.156:3300,172.21.15.186:3300" 2024-01-31T05:22:32.188 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@3a1f4122-bff7-11ee-95b3-87774f69a715.cephfs=/volumes/qa/sv_1/0677e5c0-3673-41ea-a0ca-4130f847896f 2024-01-31T05:22:32.188 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.45:3300/172.21.15.156:3300/172.21.15.186:3300" will pass to kernel 2024-01-31T05:22:32.189 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-01-31T05:22:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-01-31T05:22:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-01-31T05:22:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-01-31T05:22:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-01-31T05:22:32.190 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-31T05:22:32.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:32.191 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:22:32.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:32.219 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-31T05:22:32.275 INFO:teuthology.orchestra.run.smithi045.stdout:client24839 2024-01-31T05:22:32.275 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:22:32.275 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:22:32.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:32.331 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-31T05:22:32.387 INFO:teuthology.orchestra.run.smithi045.stdout:client24839 2024-01-31T05:22:32.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:32.388 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-31T05:22:32.443 INFO:teuthology.orchestra.run.smithi045.stdout:client24839 2024-01-31T05:22:32.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:22:32.444 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715.client24839/status') 2024-01-31T05:22:32.510 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.24839 (3)172.21.15.45:0/3345941179 2024-01-31T05:22:32.510 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2024-01-31T05:22:32.510 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2024-01-31T05:22:32.510 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2024-01-31T05:22:32.510 INFO:teuthology.orchestra.run.smithi045.stderr:71 bytes copied, 2.1151e-05 s, 3.4 MB/s 2024-01-31T05:22:32.511 INFO:teuthology.run_tasks:Running task exec... 2024-01-31T05:22:32.522 INFO:teuthology.task.exec:Executing custom commands... 2024-01-31T05:22:32.522 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-01-31T05:22:32.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-01-31T05:22:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.656+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1113325325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b8100d20 msgr2=0x7f05b8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.656+0000 7f05bdcf1640 1 --2- 172.21.15.45:0/1113325325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b8100d20 0x7f05b8101140 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009900 tx=0x7f05a802bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.656+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1113325325 shutdown_connections 2024-01-31T05:22:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.656+0000 7f05bdcf1640 1 --2- 172.21.15.45:0/1113325325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05b81028d0 0x7f05b8109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.656+0000 7f05bdcf1640 1 --2- 172.21.15.45:0/1113325325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05b8101f10 0x7f05b8102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.656+0000 7f05bdcf1640 1 --2- 172.21.15.45:0/1113325325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b8100d20 0x7f05b8101140 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.656+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1113325325 >> 172.21.15.45:0/1113325325 conn(0x7f05b80fc2b0 msgr2=0x7f05b80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.656+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1113325325 shutdown_connections 2024-01-31T05:22:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.656+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1113325325 wait complete. 2024-01-31T05:22:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05bdcf1640 1 Processor -- start 2024-01-31T05:22:32.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05bdcf1640 1 -- start start 2024-01-31T05:22:32.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05bdcf1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05b8100d20 0x7f05b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:32.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05bdcf1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05b8101f10 0x7f05b80730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:32.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05bdcf1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b81028d0 0x7f05b8073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:32.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05bdcf1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b8073d80 con 0x7f05b81028d0 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05bdcf1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b8073ef0 con 0x7f05b8101f10 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05bdcf1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b81a0e00 con 0x7f05b8100d20 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05b7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b81028d0 0x7f05b8073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05b7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b81028d0 0x7f05b8073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58754/0 (socket says 172.21.15.45:58754) 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05b7fff640 1 -- 172.21.15.45:0/1610069206 learned_addr learned my addr 172.21.15.45:0/1610069206 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05b77fe640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05b8100d20 0x7f05b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05aeffd640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05b8101f10 0x7f05b80730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05b7fff640 1 -- 172.21.15.45:0/1610069206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05b8100d20 msgr2=0x7f05b8074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05b7fff640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05b8100d20 0x7f05b8074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05b7fff640 1 -- 172.21.15.45:0/1610069206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05b8101f10 msgr2=0x7f05b80730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05b7fff640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05b8101f10 0x7f05b80730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.657+0000 7f05b7fff640 1 -- 172.21.15.45:0/1610069206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a8009560 con 0x7f05b81028d0 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.658+0000 7f05b7fff640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b81028d0 0x7f05b8073620 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f05b000b9a0 tx=0x7f05b000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.658+0000 7f05b57fa640 1 -- 172.21.15.45:0/1610069206 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b000cba0 con 0x7f05b81028d0 2024-01-31T05:22:32.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.658+0000 7f05b57fa640 1 -- 172.21.15.45:0/1610069206 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05b000cd00 con 0x7f05b81028d0 2024-01-31T05:22:32.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.658+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1610069206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b81a10b0 con 0x7f05b81028d0 2024-01-31T05:22:32.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.658+0000 7f05b57fa640 1 -- 172.21.15.45:0/1610069206 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0007640 con 0x7f05b81028d0 2024-01-31T05:22:32.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.658+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1610069206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b81a15f0 con 0x7f05b81028d0 2024-01-31T05:22:32.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.659+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1610069206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b8108090 con 0x7f05b81028d0 2024-01-31T05:22:32.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.660+0000 7f05b57fa640 1 -- 172.21.15.45:0/1610069206 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f05b0007800 con 0x7f05b81028d0 2024-01-31T05:22:32.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.661+0000 7f05b57fa640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f05a40a1380 0x7f05a40a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:32.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.661+0000 7f05b77fe640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f05a40a1380 0x7f05a40a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:32.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.661+0000 7f05b57fa640 1 -- 172.21.15.45:0/1610069206 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f05b00cc830 con 0x7f05b81028d0 2024-01-31T05:22:32.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.661+0000 7f05b77fe640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f05a40a1380 0x7f05a40a3840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f05a8000c00 tx=0x7f05a802c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.664+0000 7f05b57fa640 1 -- 172.21.15.45:0/1610069206 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f05b0095d90 con 0x7f05b81028d0 2024-01-31T05:22:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:32.886+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1610069206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f05b810c6b0 con 0x7f05b81028d0 2024-01-31T05:22:33.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.488+0000 7f05b57fa640 1 -- 172.21.15.45:0/1610069206 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v21) v1 ==== 118+0+0 (secure 0 0 0) 0x7f05b00993b0 con 0x7f05b81028d0 2024-01-31T05:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.492+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1610069206 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f05a40a1380 msgr2=0x7f05a40a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.492+0000 7f05bdcf1640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f05a40a1380 0x7f05a40a3840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f05a8000c00 tx=0x7f05a802c5d0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.492+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1610069206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b81028d0 msgr2=0x7f05b8073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.492+0000 7f05bdcf1640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b81028d0 0x7f05b8073620 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f05b000b9a0 tx=0x7f05b000be60 comp rx=0 tx=0).stop 2024-01-31T05:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.492+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1610069206 shutdown_connections 2024-01-31T05:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.492+0000 7f05bdcf1640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f05a40a1380 0x7f05a40a3840 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.492+0000 7f05bdcf1640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b81028d0 0x7f05b8073620 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.492+0000 7f05bdcf1640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05b8101f10 0x7f05b80730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.492+0000 7f05bdcf1640 1 --2- 172.21.15.45:0/1610069206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05b8100d20 0x7f05b8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.492+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1610069206 >> 172.21.15.45:0/1610069206 conn(0x7f05b80fc2b0 msgr2=0x7f05b8105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.492+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1610069206 shutdown_connections 2024-01-31T05:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.493+0000 7f05bdcf1640 1 -- 172.21.15.45:0/1610069206 wait complete. 2024-01-31T05:22:33.505 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-01-31T05:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.617+0000 7fe924cd6640 1 -- 172.21.15.45:0/2183144873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9201028d0 msgr2=0x7fe920109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.617+0000 7fe924cd6640 1 --2- 172.21.15.45:0/2183144873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9201028d0 0x7fe920109160 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe918009ca0 tx=0x7fe91802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.617+0000 7fe924cd6640 1 -- 172.21.15.45:0/2183144873 shutdown_connections 2024-01-31T05:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.617+0000 7fe924cd6640 1 --2- 172.21.15.45:0/2183144873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9201028d0 0x7fe920109160 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.617+0000 7fe924cd6640 1 --2- 172.21.15.45:0/2183144873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe920101f10 0x7fe920102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.617+0000 7fe924cd6640 1 --2- 172.21.15.45:0/2183144873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe920100d20 0x7fe920101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.617+0000 7fe924cd6640 1 -- 172.21.15.45:0/2183144873 >> 172.21.15.45:0/2183144873 conn(0x7fe9200fc2b0 msgr2=0x7fe9200fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.617+0000 7fe924cd6640 1 -- 172.21.15.45:0/2183144873 shutdown_connections 2024-01-31T05:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.617+0000 7fe924cd6640 1 -- 172.21.15.45:0/2183144873 wait complete. 2024-01-31T05:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.618+0000 7fe924cd6640 1 Processor -- start 2024-01-31T05:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.618+0000 7fe924cd6640 1 -- start start 2024-01-31T05:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.618+0000 7fe924cd6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe920100d20 0x7fe920196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.618+0000 7fe924cd6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe920101f10 0x7fe920196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.618+0000 7fe924cd6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9201028d0 0x7fe92019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.618+0000 7fe924cd6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92019e590 con 0x7fe9201028d0 2024-01-31T05:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.618+0000 7fe924cd6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92019e700 con 0x7fe920100d20 2024-01-31T05:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.618+0000 7fe924cd6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92019e8c0 con 0x7fe920101f10 2024-01-31T05:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.618+0000 7fe91ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9201028d0 0x7fe92019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.618+0000 7fe91ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9201028d0 0x7fe92019e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58758/0 (socket says 172.21.15.45:58758) 2024-01-31T05:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.618+0000 7fe91ed9d640 1 -- 172.21.15.45:0/1252252240 learned_addr learned my addr 172.21.15.45:0/1252252240 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe91e59c640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe920100d20 0x7fe920196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe91dd9b640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe920101f10 0x7fe920196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe91ed9d640 1 -- 172.21.15.45:0/1252252240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe920101f10 msgr2=0x7fe920196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe91ed9d640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe920101f10 0x7fe920196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe91ed9d640 1 -- 172.21.15.45:0/1252252240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe920100d20 msgr2=0x7fe920196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe91ed9d640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe920100d20 0x7fe920196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe91ed9d640 1 -- 172.21.15.45:0/1252252240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe918009900 con 0x7fe9201028d0 2024-01-31T05:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe91dd9b640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe920101f10 0x7fe920196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe91ed9d640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9201028d0 0x7fe92019e050 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe918037040 tx=0x7fe918032a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe90f7fe640 1 -- 172.21.15.45:0/1252252240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918003970 con 0x7fe9201028d0 2024-01-31T05:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe90f7fe640 1 -- 172.21.15.45:0/1252252240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe918004040 con 0x7fe9201028d0 2024-01-31T05:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe924cd6640 1 -- 172.21.15.45:0/1252252240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92019eb40 con 0x7fe9201028d0 2024-01-31T05:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe90f7fe640 1 -- 172.21.15.45:0/1252252240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91803d650 con 0x7fe9201028d0 2024-01-31T05:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.619+0000 7fe924cd6640 1 -- 172.21.15.45:0/1252252240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92019f040 con 0x7fe9201028d0 2024-01-31T05:22:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.620+0000 7fe924cd6640 1 -- 172.21.15.45:0/1252252240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe920107f70 con 0x7fe9201028d0 2024-01-31T05:22:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.621+0000 7fe90f7fe640 1 -- 172.21.15.45:0/1252252240 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe91803b040 con 0x7fe9201028d0 2024-01-31T05:22:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.622+0000 7fe90f7fe640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9080a1400 0x7fe9080a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.622+0000 7fe90f7fe640 1 -- 172.21.15.45:0/1252252240 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe9180ef3a0 con 0x7fe9201028d0 2024-01-31T05:22:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.622+0000 7fe91e59c640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9080a1400 0x7fe9080a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.622+0000 7fe91e59c640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9080a1400 0x7fe9080a38c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe920101d70 tx=0x7fe910005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.626+0000 7fe90f7fe640 1 -- 172.21.15.45:0/1252252240 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe9180b8850 con 0x7fe9201028d0 2024-01-31T05:22:33.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.832+0000 7fe924cd6640 1 -- 172.21.15.45:0/1252252240 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7fe92002cc20 con 0x7fe9080a1400 2024-01-31T05:22:33.836 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:33 smithi045 ceph-mon[31585]: pgmap v341: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:22:33.836 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:33 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1610069206' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-01-31T05:22:33.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:33 smithi186 ceph-mon[35659]: pgmap v341: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:22:33.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:33 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1610069206' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-01-31T05:22:33.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.846+0000 7fe90f7fe640 1 -- 172.21.15.45:0/1252252240 <== mgr.14152 v2:172.21.15.45:6800/2788245525 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7fe92002cc20 con 0x7fe9080a1400 2024-01-31T05:22:33.845 INFO:teuthology.orchestra.run.smithi045.stdout:{} 2024-01-31T05:22:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.848+0000 7fe924cd6640 1 -- 172.21.15.45:0/1252252240 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9080a1400 msgr2=0x7fe9080a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.848+0000 7fe924cd6640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9080a1400 0x7fe9080a38c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe920101d70 tx=0x7fe910005f90 comp rx=0 tx=0).stop 2024-01-31T05:22:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.848+0000 7fe924cd6640 1 -- 172.21.15.45:0/1252252240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9201028d0 msgr2=0x7fe92019e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.848+0000 7fe924cd6640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9201028d0 0x7fe92019e050 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe918037040 tx=0x7fe918032a40 comp rx=0 tx=0).stop 2024-01-31T05:22:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.848+0000 7fe924cd6640 1 -- 172.21.15.45:0/1252252240 shutdown_connections 2024-01-31T05:22:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.848+0000 7fe924cd6640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fe9080a1400 0x7fe9080a38c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.848+0000 7fe924cd6640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9201028d0 0x7fe92019e050 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.848+0000 7fe924cd6640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe920101f10 0x7fe920196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.848+0000 7fe924cd6640 1 --2- 172.21.15.45:0/1252252240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe920100d20 0x7fe920196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.848+0000 7fe924cd6640 1 -- 172.21.15.45:0/1252252240 >> 172.21.15.45:0/1252252240 conn(0x7fe9200fc2b0 msgr2=0x7fe920105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.848+0000 7fe924cd6640 1 -- 172.21.15.45:0/1252252240 shutdown_connections 2024-01-31T05:22:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:33.848+0000 7fe924cd6640 1 -- 172.21.15.45:0/1252252240 wait complete. 2024-01-31T05:22:33.858 INFO:teuthology.run_tasks:Running task check-counter... 2024-01-31T05:22:33.868 INFO:teuthology.run_tasks:Running task exec... 2024-01-31T05:22:33.897 INFO:teuthology.task.exec:Executing custom commands... 2024-01-31T05:22:33.897 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-01-31T05:22:33.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-01-31T05:22:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:33 smithi156 ceph-mon[35720]: pgmap v341: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-31T05:22:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:33 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1610069206' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-01-31T05:22:34.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.000+0000 7f08a162a640 1 -- 172.21.15.45:0/739475534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f089c1017c0 msgr2=0x7f089c108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:34.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.000+0000 7f08a162a640 1 --2- 172.21.15.45:0/739475534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f089c1017c0 0x7f089c108050 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f088c009ca0 tx=0x7f088c02c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.000+0000 7f08a162a640 1 -- 172.21.15.45:0/739475534 shutdown_connections 2024-01-31T05:22:34.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.000+0000 7f08a162a640 1 --2- 172.21.15.45:0/739475534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f089c1017c0 0x7f089c108050 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.000+0000 7f08a162a640 1 --2- 172.21.15.45:0/739475534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f089c100e00 0x7f089c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.000+0000 7f08a162a640 1 --2- 172.21.15.45:0/739475534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f089c0ffc10 0x7f089c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.000+0000 7f08a162a640 1 -- 172.21.15.45:0/739475534 >> 172.21.15.45:0/739475534 conn(0x7f089c0fb1c0 msgr2=0x7f089c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.000+0000 7f08a162a640 1 -- 172.21.15.45:0/739475534 shutdown_connections 2024-01-31T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.000+0000 7f08a162a640 1 -- 172.21.15.45:0/739475534 wait complete. 2024-01-31T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f08a162a640 1 Processor -- start 2024-01-31T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f08a162a640 1 -- start start 2024-01-31T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f08a162a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f089c0ffc10 0x7f089c103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f08a162a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f089c100e00 0x7f089c1021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f08a162a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f089c1017c0 0x7f089c102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f08a162a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c102e80 con 0x7f089c0ffc10 2024-01-31T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f08a162a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c102ff0 con 0x7f089c1017c0 2024-01-31T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f08a162a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c1a0c60 con 0x7f089c100e00 2024-01-31T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f089affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f089c0ffc10 0x7f089c103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f089affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f089c0ffc10 0x7f089c103ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58810/0 (socket says 172.21.15.45:58810) 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f089affd640 1 -- 172.21.15.45:0/645415607 learned_addr learned my addr 172.21.15.45:0/645415607 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f089a7fc640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f089c100e00 0x7f089c1021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.001+0000 7f089b7fe640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f089c1017c0 0x7f089c102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.002+0000 7f089affd640 1 -- 172.21.15.45:0/645415607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f089c100e00 msgr2=0x7f089c1021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.002+0000 7f089affd640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f089c100e00 0x7f089c1021e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.002+0000 7f089affd640 1 -- 172.21.15.45:0/645415607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f089c1017c0 msgr2=0x7f089c102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.002+0000 7f089affd640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f089c1017c0 0x7f089c102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.002+0000 7f089affd640 1 -- 172.21.15.45:0/645415607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c009900 con 0x7f089c0ffc10 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.002+0000 7f089b7fe640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f089c1017c0 0x7f089c102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.002+0000 7f089affd640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f089c0ffc10 0x7f089c103ba0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f089400e9f0 tx=0x7f089400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.002+0000 7f0893fff640 1 -- 172.21.15.45:0/645415607 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894009700 con 0x7f089c0ffc10 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.002+0000 7f0893fff640 1 -- 172.21.15.45:0/645415607 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08940044e0 con 0x7f089c0ffc10 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.002+0000 7f08a162a640 1 -- 172.21.15.45:0/645415607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089c1a0f10 con 0x7f089c0ffc10 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.002+0000 7f0893fff640 1 -- 172.21.15.45:0/645415607 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894010430 con 0x7f089c0ffc10 2024-01-31T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.002+0000 7f08a162a640 1 -- 172.21.15.45:0/645415607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c1a1330 con 0x7f089c0ffc10 2024-01-31T05:22:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.003+0000 7f08a162a640 1 -- 172.21.15.45:0/645415607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089c10b4d0 con 0x7f089c0ffc10 2024-01-31T05:22:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.004+0000 7f0893fff640 1 -- 172.21.15.45:0/645415607 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f089400cd80 con 0x7f089c0ffc10 2024-01-31T05:22:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.004+0000 7f0893fff640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f08880a1380 0x7f08880a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.005+0000 7f089a7fc640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f08880a1380 0x7f08880a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.005+0000 7f0893fff640 1 -- 172.21.15.45:0/645415607 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0894014070 con 0x7f089c0ffc10 2024-01-31T05:22:34.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.005+0000 7f089a7fc640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f08880a1380 0x7f08880a3840 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f089c103910 tx=0x7f0884005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.009+0000 7f0893fff640 1 -- 172.21.15.45:0/645415607 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f0894096370 con 0x7f089c0ffc10 2024-01-31T05:22:34.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:33 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:33.836+0000 7f4ecee5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:34.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:33 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:33.836+0000 7f4ecee5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:34.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:33 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:33.836+0000 7f4ecee5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:34.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:33 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:33.836+0000 7f4ecee5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:34.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:33 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:33.836+0000 7f4ecee5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.235+0000 7f08a162a640 1 -- 172.21.15.45:0/645415607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f089c10b690 con 0x7f089c0ffc10 2024-01-31T05:22:34.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.492+0000 7f0893fff640 1 -- 172.21.15.45:0/645415607 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0894099990 con 0x7f089c0ffc10 2024-01-31T05:22:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.494+0000 7f08a162a640 1 -- 172.21.15.45:0/645415607 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f08880a1380 msgr2=0x7f08880a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.494+0000 7f08a162a640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f08880a1380 0x7f08880a3840 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f089c103910 tx=0x7f0884005f90 comp rx=0 tx=0).stop 2024-01-31T05:22:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.494+0000 7f08a162a640 1 -- 172.21.15.45:0/645415607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f089c0ffc10 msgr2=0x7f089c103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.494+0000 7f08a162a640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f089c0ffc10 0x7f089c103ba0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f089400e9f0 tx=0x7f089400eeb0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.494+0000 7f08a162a640 1 -- 172.21.15.45:0/645415607 shutdown_connections 2024-01-31T05:22:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.494+0000 7f08a162a640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f08880a1380 0x7f08880a3840 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.494+0000 7f08a162a640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f089c1017c0 0x7f089c102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.494+0000 7f08a162a640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f089c100e00 0x7f089c1021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.494+0000 7f08a162a640 1 --2- 172.21.15.45:0/645415607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f089c0ffc10 0x7f089c103ba0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.494+0000 7f08a162a640 1 -- 172.21.15.45:0/645415607 >> 172.21.15.45:0/645415607 conn(0x7f089c0fb1c0 msgr2=0x7f089c0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.494+0000 7f08a162a640 1 -- 172.21.15.45:0/645415607 shutdown_connections 2024-01-31T05:22:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.495+0000 7f08a162a640 1 -- 172.21.15.45:0/645415607 wait complete. 2024-01-31T05:22:34.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-01-31T05:22:34.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.621+0000 7f0265c2a640 1 -- 172.21.15.45:0/3274424319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02601017c0 msgr2=0x7f0260108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:34.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.621+0000 7f0265c2a640 1 --2- 172.21.15.45:0/3274424319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02601017c0 0x7f0260108050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0258009ca0 tx=0x7f025802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.621+0000 7f0265c2a640 1 -- 172.21.15.45:0/3274424319 shutdown_connections 2024-01-31T05:22:34.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.621+0000 7f0265c2a640 1 --2- 172.21.15.45:0/3274424319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02601017c0 0x7f0260108050 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.621+0000 7f0265c2a640 1 --2- 172.21.15.45:0/3274424319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0260100e00 0x7f0260101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.621+0000 7f0265c2a640 1 --2- 172.21.15.45:0/3274424319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02600ffc10 0x7f0260100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.621+0000 7f0265c2a640 1 -- 172.21.15.45:0/3274424319 >> 172.21.15.45:0/3274424319 conn(0x7f02600fb180 msgr2=0x7f02600fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.621+0000 7f0265c2a640 1 -- 172.21.15.45:0/3274424319 shutdown_connections 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.621+0000 7f0265c2a640 1 -- 172.21.15.45:0/3274424319 wait complete. 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f0265c2a640 1 Processor -- start 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f0265c2a640 1 -- start start 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f0265c2a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02600ffc10 0x7f0260195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f0265c2a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0260100e00 0x7f0260196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f0265c2a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02601017c0 0x7f026019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f0265c2a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026019d5d0 con 0x7f02601017c0 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f0265c2a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026019d740 con 0x7f0260100e00 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f0265c2a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026019d900 con 0x7f02600ffc10 2024-01-31T05:22:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f025ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02601017c0 0x7f026019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:34.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f025ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02601017c0 0x7f026019d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58858/0 (socket says 172.21.15.45:58858) 2024-01-31T05:22:34.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f025ffff640 1 -- 172.21.15.45:0/11154125 learned_addr learned my addr 172.21.15.45:0/11154125 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:34.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f025f7fe640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02600ffc10 0x7f0260195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:34.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f025effd640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0260100e00 0x7f0260196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:34.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f025ffff640 1 -- 172.21.15.45:0/11154125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02600ffc10 msgr2=0x7f0260195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:34.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f025ffff640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02600ffc10 0x7f0260195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f025ffff640 1 -- 172.21.15.45:0/11154125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0260100e00 msgr2=0x7f0260196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:34.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f025ffff640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0260100e00 0x7f0260196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.622+0000 7f025ffff640 1 -- 172.21.15.45:0/11154125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0258009900 con 0x7f02601017c0 2024-01-31T05:22:34.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.623+0000 7f025ffff640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02601017c0 0x7f026019d090 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0258037040 tx=0x7f0258032790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:34.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.623+0000 7f025cff9640 1 -- 172.21.15.45:0/11154125 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258034c70 con 0x7f02601017c0 2024-01-31T05:22:34.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.623+0000 7f0265c2a640 1 -- 172.21.15.45:0/11154125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026019db80 con 0x7f02601017c0 2024-01-31T05:22:34.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.623+0000 7f025cff9640 1 -- 172.21.15.45:0/11154125 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0258003e30 con 0x7f02601017c0 2024-01-31T05:22:34.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.623+0000 7f025cff9640 1 -- 172.21.15.45:0/11154125 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025803d3f0 con 0x7f02601017c0 2024-01-31T05:22:34.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.623+0000 7f0265c2a640 1 -- 172.21.15.45:0/11154125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026019e080 con 0x7f02601017c0 2024-01-31T05:22:34.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.624+0000 7f0265c2a640 1 -- 172.21.15.45:0/11154125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0260106f30 con 0x7f02601017c0 2024-01-31T05:22:34.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.625+0000 7f025cff9640 1 -- 172.21.15.45:0/11154125 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) v1 ==== 135912+0+0 (secure 0 0 0) 0x7f025803b040 con 0x7f02601017c0 2024-01-31T05:22:34.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.626+0000 7f025cff9640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f02480a13d0 0x7f02480a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:34.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.626+0000 7f025f7fe640 1 -- 172.21.15.45:0/11154125 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f02480a13d0 msgr2=0x7f02480a3890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2788245525 2024-01-31T05:22:34.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.626+0000 7f025cff9640 1 -- 172.21.15.45:0/11154125 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f02580ef520 con 0x7f02601017c0 2024-01-31T05:22:34.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.626+0000 7f025f7fe640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f02480a13d0 0x7f02480a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-31T05:22:34.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.634+0000 7f025cff9640 1 -- 172.21.15.45:0/11154125 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f02580b8a80 con 0x7f02601017c0 2024-01-31T05:22:34.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.818+0000 7f0265c2a640 1 -- 172.21.15.45:0/11154125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f026002cb10 con 0x7f02601017c0 2024-01-31T05:22:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.821+0000 7f025cff9640 1 -- 172.21.15.45:0/11154125 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f02580bc0a0 con 0x7f02601017c0 2024-01-31T05:22:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.824+0000 7f0265c2a640 1 -- 172.21.15.45:0/11154125 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f02480a13d0 msgr2=0x7f02480a3890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:22:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.824+0000 7f0265c2a640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f02480a13d0 0x7f02480a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.824+0000 7f0265c2a640 1 -- 172.21.15.45:0/11154125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02601017c0 msgr2=0x7f026019d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.824+0000 7f0265c2a640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02601017c0 0x7f026019d090 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0258037040 tx=0x7f0258032790 comp rx=0 tx=0).stop 2024-01-31T05:22:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.824+0000 7f0265c2a640 1 -- 172.21.15.45:0/11154125 shutdown_connections 2024-01-31T05:22:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.824+0000 7f0265c2a640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f02480a13d0 0x7f02480a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.824+0000 7f0265c2a640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02601017c0 0x7f026019d090 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.824+0000 7f0265c2a640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0260100e00 0x7f0260196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.824+0000 7f0265c2a640 1 --2- 172.21.15.45:0/11154125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02600ffc10 0x7f0260195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.824+0000 7f0265c2a640 1 -- 172.21.15.45:0/11154125 >> 172.21.15.45:0/11154125 conn(0x7f02600fb180 msgr2=0x7f0260104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:34.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.824+0000 7f0265c2a640 1 -- 172.21.15.45:0/11154125 shutdown_connections 2024-01-31T05:22:34.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.824+0000 7f0265c2a640 1 -- 172.21.15.45:0/11154125 wait complete. 2024-01-31T05:22:34.834 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:34 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1610069206' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-01-31T05:22:34.834 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:34 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:22:34.834 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:34 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/645415607' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-01-31T05:22:34.834 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:34 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: ignoring --setuser ceph since I am not root 2024-01-31T05:22:34.834 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:34 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: ignoring --setgroup ceph since I am not root 2024-01-31T05:22:34.835 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-01-31T05:22:34.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:34 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1610069206' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-01-31T05:22:34.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:34 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:22:34.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:34 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/645415607' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-01-31T05:22:34.841 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:34 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: ignoring --setuser ceph since I am not root 2024-01-31T05:22:34.841 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:34 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: ignoring --setgroup ceph since I am not root 2024-01-31T05:22:34.841 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:34 smithi186 ceph-mgr[37521]: -- 172.21.15.186:0/281968063 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56171c623520 con 0x56171c5f8c00 2024-01-31T05:22:34.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.940+0000 7f1f6c292640 1 -- 172.21.15.45:0/3549750011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f641017c0 msgr2=0x7f1f64108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:34.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.940+0000 7f1f6c292640 1 --2- 172.21.15.45:0/3549750011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f641017c0 0x7f1f64108050 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009ca0 tx=0x7f1f5c02c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.940+0000 7f1f6c292640 1 -- 172.21.15.45:0/3549750011 shutdown_connections 2024-01-31T05:22:34.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.940+0000 7f1f6c292640 1 --2- 172.21.15.45:0/3549750011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f641017c0 0x7f1f64108050 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.940+0000 7f1f6c292640 1 --2- 172.21.15.45:0/3549750011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f64100e00 0x7f1f64101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.940+0000 7f1f6c292640 1 --2- 172.21.15.45:0/3549750011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f640ffc10 0x7f1f64100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.940+0000 7f1f6c292640 1 -- 172.21.15.45:0/3549750011 >> 172.21.15.45:0/3549750011 conn(0x7f1f640fb180 msgr2=0x7f1f640fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:34.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.940+0000 7f1f6c292640 1 -- 172.21.15.45:0/3549750011 shutdown_connections 2024-01-31T05:22:34.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.940+0000 7f1f6c292640 1 -- 172.21.15.45:0/3549750011 wait complete. 2024-01-31T05:22:34.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6c292640 1 Processor -- start 2024-01-31T05:22:34.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6c292640 1 -- start start 2024-01-31T05:22:34.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6c292640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f640ffc10 0x7f1f64103c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:34.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6c292640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f64100e00 0x7f1f64102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:34.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6c292640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f641017c0 0x7f1f64102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:34.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6c292640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64102ef0 con 0x7f1f64100e00 2024-01-31T05:22:34.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6c292640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64103060 con 0x7f1f640ffc10 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6c292640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f641a0ca0 con 0x7f1f641017c0 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6982d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f64100e00 0x7f1f64102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6982d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f64100e00 0x7f1f64102250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58878/0 (socket says 172.21.15.45:58878) 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6982d640 1 -- 172.21.15.45:0/3760365115 learned_addr learned my addr 172.21.15.45:0/3760365115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6a02e640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f640ffc10 0x7f1f64103c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6a82f640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f641017c0 0x7f1f64102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6982d640 1 -- 172.21.15.45:0/3760365115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f641017c0 msgr2=0x7f1f64102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6982d640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f641017c0 0x7f1f64102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6982d640 1 -- 172.21.15.45:0/3760365115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f640ffc10 msgr2=0x7f1f64103c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6982d640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f640ffc10 0x7f1f64103c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6982d640 1 -- 172.21.15.45:0/3760365115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c009900 con 0x7f1f64100e00 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.941+0000 7f1f6982d640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f64100e00 0x7f1f64102250 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400d980 tx=0x7f1f5400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.942+0000 7f1f5b7fe640 1 -- 172.21.15.45:0/3760365115 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54009c30 con 0x7f1f64100e00 2024-01-31T05:22:34.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.942+0000 7f1f5b7fe640 1 -- 172.21.15.45:0/3760365115 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f5400b660 con 0x7f1f64100e00 2024-01-31T05:22:34.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.942+0000 7f1f6c292640 1 -- 172.21.15.45:0/3760365115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f641a0f50 con 0x7f1f64100e00 2024-01-31T05:22:34.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.942+0000 7f1f5b7fe640 1 -- 172.21.15.45:0/3760365115 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400f620 con 0x7f1f64100e00 2024-01-31T05:22:34.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.942+0000 7f1f6c292640 1 -- 172.21.15.45:0/3760365115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f641a13d0 con 0x7f1f64100e00 2024-01-31T05:22:34.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.943+0000 7f1f6c292640 1 -- 172.21.15.45:0/3760365115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6410b4b0 con 0x7f1f64100e00 2024-01-31T05:22:34.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.944+0000 7f1f5b7fe640 1 -- 172.21.15.45:0/3760365115 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) v1 ==== 135912+0+0 (secure 0 0 0) 0x7f1f54010460 con 0x7f1f64100e00 2024-01-31T05:22:34.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.944+0000 7f1f5b7fe640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1f500a13d0 0x7f1f500a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:34.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.944+0000 7f1f6a02e640 1 -- 172.21.15.45:0/3760365115 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1f500a13d0 msgr2=0x7f1f500a3890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2788245525 2024-01-31T05:22:34.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.945+0000 7f1f5b7fe640 1 -- 172.21.15.45:0/3760365115 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1f540cc620 con 0x7f1f64100e00 2024-01-31T05:22:34.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.945+0000 7f1f6a02e640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1f500a13d0 0x7f1f500a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-31T05:22:34.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:34.949+0000 7f1f5b7fe640 1 -- 172.21.15.45:0/3760365115 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1f54095b80 con 0x7f1f64100e00 2024-01-31T05:22:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:34 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1610069206' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-01-31T05:22:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:34 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:22:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:34 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/645415607' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-01-31T05:22:34.976 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:34 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: ignoring --setuser ceph since I am not root 2024-01-31T05:22:34.976 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:34 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: ignoring --setgroup ceph since I am not root 2024-01-31T05:22:34.976 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:34 smithi156 ceph-mgr[37104]: -- 172.21.15.156:0/196545921 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561911323520 con 0x5619112f8c00 2024-01-31T05:22:35.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.130+0000 7f1f6c292640 1 -- 172.21.15.45:0/3760365115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f1f641a15c0 con 0x7f1f64100e00 2024-01-31T05:22:35.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.134+0000 7f1f5b7fe640 1 -- 172.21.15.45:0/3760365115 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1f540991a0 con 0x7f1f64100e00 2024-01-31T05:22:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.136+0000 7f1f6c292640 1 -- 172.21.15.45:0/3760365115 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1f500a13d0 msgr2=0x7f1f500a3890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:22:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.136+0000 7f1f6c292640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1f500a13d0 0x7f1f500a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.136+0000 7f1f6c292640 1 -- 172.21.15.45:0/3760365115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f64100e00 msgr2=0x7f1f64102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.136+0000 7f1f6c292640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f64100e00 0x7f1f64102250 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400d980 tx=0x7f1f5400de40 comp rx=0 tx=0).stop 2024-01-31T05:22:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.136+0000 7f1f6c292640 1 -- 172.21.15.45:0/3760365115 shutdown_connections 2024-01-31T05:22:35.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.136+0000 7f1f6c292640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7f1f500a13d0 0x7f1f500a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:35.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.136+0000 7f1f6c292640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f64100e00 0x7f1f64102250 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:35.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.136+0000 7f1f6c292640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f641017c0 0x7f1f64102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:35.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.136+0000 7f1f6c292640 1 --2- 172.21.15.45:0/3760365115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f640ffc10 0x7f1f64103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:35.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.136+0000 7f1f6c292640 1 -- 172.21.15.45:0/3760365115 >> 172.21.15.45:0/3760365115 conn(0x7f1f640fb180 msgr2=0x7f1f64068370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:35.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.136+0000 7f1f6c292640 1 -- 172.21.15.45:0/3760365115 shutdown_connections 2024-01-31T05:22:35.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.136+0000 7f1f6c292640 1 -- 172.21.15.45:0/3760365115 wait complete. 2024-01-31T05:22:35.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:35 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:35.074+0000 7f0b391c9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-31T05:22:35.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-01-31T05:22:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.255+0000 7fbf714b1640 1 -- 172.21.15.45:0/3751941104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf6c104120 msgr2=0x7fbf6c106510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.255+0000 7fbf714b1640 1 --2- 172.21.15.45:0/3751941104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf6c104120 0x7fbf6c106510 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009900 tx=0x7fbf6002bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.255+0000 7fbf714b1640 1 -- 172.21.15.45:0/3751941104 shutdown_connections 2024-01-31T05:22:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.255+0000 7fbf714b1640 1 --2- 172.21.15.45:0/3751941104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf6c104120 0x7fbf6c106510 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.255+0000 7fbf714b1640 1 --2- 172.21.15.45:0/3751941104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf6c106a50 0x7fbf6c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.255+0000 7fbf714b1640 1 --2- 172.21.15.45:0/3751941104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf6c1017f0 0x7fbf6c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.255+0000 7fbf714b1640 1 -- 172.21.15.45:0/3751941104 >> 172.21.15.45:0/3751941104 conn(0x7fbf6c0fb180 msgr2=0x7fbf6c0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.255+0000 7fbf714b1640 1 -- 172.21.15.45:0/3751941104 shutdown_connections 2024-01-31T05:22:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.255+0000 7fbf714b1640 1 -- 172.21.15.45:0/3751941104 wait complete. 2024-01-31T05:22:35.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.255+0000 7fbf714b1640 1 Processor -- start 2024-01-31T05:22:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.255+0000 7fbf714b1640 1 -- start start 2024-01-31T05:22:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf714b1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf6c1017f0 0x7fbf6c101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf714b1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf6c104120 0x7fbf6c0ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf714b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf6c106a50 0x7fbf6c0ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf714b1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c100440 con 0x7fbf6c104120 2024-01-31T05:22:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf714b1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c1005b0 con 0x7fbf6c106a50 2024-01-31T05:22:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf714b1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c1a0c80 con 0x7fbf6c1017f0 2024-01-31T05:22:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf6a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf6c104120 0x7fbf6c0ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf6a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf6c104120 0x7fbf6c0ff7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58900/0 (socket says 172.21.15.45:58900) 2024-01-31T05:22:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf6a7fc640 1 -- 172.21.15.45:0/672039066 learned_addr learned my addr 172.21.15.45:0/672039066 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf6affd640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf6c1017f0 0x7fbf6c101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf6b7fe640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf6c106a50 0x7fbf6c0ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf6a7fc640 1 -- 172.21.15.45:0/672039066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf6c1017f0 msgr2=0x7fbf6c101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf6a7fc640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf6c1017f0 0x7fbf6c101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf6a7fc640 1 -- 172.21.15.45:0/672039066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf6c106a50 msgr2=0x7fbf6c0ffce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf6a7fc640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf6c106a50 0x7fbf6c0ffce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf6a7fc640 1 -- 172.21.15.45:0/672039066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf60009560 con 0x7fbf6c104120 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf6a7fc640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf6c104120 0x7fbf6c0ff7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbf60036040 tx=0x7fbf60004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf53fff640 1 -- 172.21.15.45:0/672039066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600043d0 con 0x7fbf6c104120 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.256+0000 7fbf714b1640 1 -- 172.21.15.45:0/672039066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6c1a0ed0 con 0x7fbf6c104120 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.257+0000 7fbf714b1640 1 -- 172.21.15.45:0/672039066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6c1a13a0 con 0x7fbf6c104120 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.257+0000 7fbf53fff640 1 -- 172.21.15.45:0/672039066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf600054c0 con 0x7fbf6c104120 2024-01-31T05:22:35.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.257+0000 7fbf53fff640 1 -- 172.21.15.45:0/672039066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60003da0 con 0x7fbf6c104120 2024-01-31T05:22:35.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.258+0000 7fbf714b1640 1 -- 172.21.15.45:0/672039066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf580052c0 con 0x7fbf6c104120 2024-01-31T05:22:35.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.259+0000 7fbf53fff640 1 -- 172.21.15.45:0/672039066 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) v1 ==== 135912+0+0 (secure 0 0 0) 0x7fbf600035f0 con 0x7fbf6c104120 2024-01-31T05:22:35.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.260+0000 7fbf53fff640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 0x7fbf540a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:35.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.260+0000 7fbf6affd640 1 -- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 msgr2=0x7fbf540a3890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2788245525 2024-01-31T05:22:35.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.260+0000 7fbf53fff640 1 -- 172.21.15.45:0/672039066 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbf600eea90 con 0x7fbf6c104120 2024-01-31T05:22:35.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.260+0000 7fbf6affd640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 0x7fbf540a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-31T05:22:35.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.264+0000 7fbf53fff640 1 -- 172.21.15.45:0/672039066 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fbf600b7f40 con 0x7fbf6c104120 2024-01-31T05:22:35.341 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:35 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:35.044+0000 7f84d57e5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-31T05:22:35.341 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:35 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:35.137+0000 7f84d57e5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-31T05:22:35.434 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:35 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:35.173+0000 7f0b391c9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-31T05:22:35.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.461+0000 7fbf6affd640 1 -- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 msgr2=0x7fbf540a3890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2788245525 2024-01-31T05:22:35.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.461+0000 7fbf6affd640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 0x7fbf540a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-31T05:22:35.476 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:35 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:35.036+0000 7fc366e47200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-31T05:22:35.476 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:35 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:35.127+0000 7fc366e47200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-31T05:22:35.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.485+0000 7fbf714b1640 1 -- 172.21.15.45:0/672039066 --> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf58000bf0 con 0x7fbf540a13d0 2024-01-31T05:22:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:35 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/645415607' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-01-31T05:22:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:35 smithi186 ceph-mon[35659]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-01-31T05:22:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:35 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/11154125' entity='client.admin' 2024-01-31T05:22:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:35 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3760365115' entity='client.admin' 2024-01-31T05:22:35.840 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:35 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:35.752+0000 7fc366e47200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-31T05:22:35.841 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:35 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/645415607' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-01-31T05:22:35.841 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:35 smithi156 ceph-mon[35720]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-01-31T05:22:35.841 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:35 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/11154125' entity='client.admin' 2024-01-31T05:22:35.841 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:35 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3760365115' entity='client.admin' 2024-01-31T05:22:35.846 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:35 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/645415607' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-01-31T05:22:35.846 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:35 smithi045 ceph-mon[31585]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-01-31T05:22:35.846 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:35 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/11154125' entity='client.admin' 2024-01-31T05:22:35.846 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:35 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3760365115' entity='client.admin' 2024-01-31T05:22:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.862+0000 7fbf6affd640 1 -- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 msgr2=0x7fbf540a3890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2788245525 2024-01-31T05:22:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:35.862+0000 7fbf6affd640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 0x7fbf540a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-31T05:22:36.091 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:35 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:35.775+0000 7f84d57e5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-31T05:22:36.091 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:35 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:35.866+0000 7f84d57e5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-31T05:22:36.091 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:35 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:35.953+0000 7f84d57e5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-31T05:22:36.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:35 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:35.847+0000 7f0b391c9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-31T05:22:36.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:35 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:35.944+0000 7f0b391c9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-31T05:22:36.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:36 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:36.033+0000 7f0b391c9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-31T05:22:36.155 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:35 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:35.841+0000 7fc366e47200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-31T05:22:36.155 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:35 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:35.925+0000 7fc366e47200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-31T05:22:36.155 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:36 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:36.075+0000 7fc366e47200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-31T05:22:36.155 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:36 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:36.156+0000 7fc366e47200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-31T05:22:36.364 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:36 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:36.106+0000 7f84d57e5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-31T05:22:36.364 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:36 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:36.188+0000 7f84d57e5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-31T05:22:36.364 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:36 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:36.365+0000 7f84d57e5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-31T05:22:36.455 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:36 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:36.188+0000 7f0b391c9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-31T05:22:36.456 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:36 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:36.273+0000 7f0b391c9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-31T05:22:36.476 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:36 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:36.330+0000 7fc366e47200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-31T05:22:36.634 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:36 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:36.548+0000 7f84d57e5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-31T05:22:36.634 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:36 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:36.634+0000 7f84d57e5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-31T05:22:36.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:36.663+0000 7fbf6affd640 1 -- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 msgr2=0x7fbf540a3890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2788245525 2024-01-31T05:22:36.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:36.663+0000 7fbf6affd640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 0x7fbf540a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-31T05:22:36.727 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:36 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:36.456+0000 7f0b391c9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-31T05:22:36.727 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:36 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:36.639+0000 7f0b391c9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-31T05:22:36.771 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:36 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:36.509+0000 7fc366e47200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-31T05:22:36.771 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:36 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:36.594+0000 7fc366e47200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-31T05:22:36.771 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:36 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:36.685+0000 7fc366e47200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-31T05:22:37.091 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:36 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:36.729+0000 7f84d57e5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-31T05:22:37.091 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:36 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:36.817+0000 7f84d57e5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-31T05:22:37.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:36 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:36.727+0000 7f0b391c9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-31T05:22:37.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:36 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:36.824+0000 7f0b391c9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-31T05:22:37.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:36 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:36.914+0000 7f0b391c9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-31T05:22:37.225 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:36 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:36.772+0000 7fc366e47200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-31T05:22:38.125 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:37 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:37.810+0000 7fc366e47200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-31T05:22:38.125 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:37 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:37.936+0000 7fc366e47200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-31T05:22:38.125 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:38 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:38.125+0000 7fc366e47200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-31T05:22:38.199 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:37 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:37.875+0000 7f84d57e5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-31T05:22:38.199 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:38 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:38.006+0000 7f84d57e5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-31T05:22:38.199 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:38 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:38.200+0000 7f84d57e5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-31T05:22:38.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:38.264+0000 7fbf6affd640 1 -- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 msgr2=0x7fbf540a3890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2788245525 2024-01-31T05:22:38.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:38.264+0000 7fbf6affd640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 0x7fbf540a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-01-31T05:22:38.321 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:37 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:37.988+0000 7f0b391c9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-31T05:22:38.321 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:38 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:38.123+0000 7f0b391c9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-31T05:22:38.476 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:38 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:38.355+0000 7fc366e47200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-31T05:22:38.591 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:38 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:38.431+0000 7f84d57e5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-31T05:22:38.642 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:38 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:38.322+0000 7f0b391c9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-31T05:22:38.642 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:38 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:38.554+0000 7f0b391c9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-31T05:22:38.976 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:38 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:38.533+0000 7fc366e47200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:22:38.976 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:38 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:38.715+0000 7fc366e47200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-31T05:22:39.062 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:38 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:38.611+0000 7f84d57e5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:22:39.063 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:38 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:38.795+0000 7f84d57e5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-31T05:22:39.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:38 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:38.741+0000 7f0b391c9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:22:39.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:38 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:38.933+0000 7f0b391c9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-31T05:22:39.341 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:39 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:39.063+0000 7f84d57e5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:22:39.342 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:39 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:39.151+0000 7f84d57e5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-31T05:22:39.468 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:38 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:38.983+0000 7fc366e47200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:22:39.468 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:39 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:39.071+0000 7fc366e47200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-31T05:22:39.642 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:39 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:39.211+0000 7f0b391c9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:22:39.642 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:39 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:39.301+0000 7f0b391c9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-31T05:22:39.726 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:39 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:39.469+0000 7fc366e47200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-31T05:22:39.841 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:39 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:39.546+0000 7f84d57e5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-31T05:22:40.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:39 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:39.704+0000 7f0b391c9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-31T05:22:40.591 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:40 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:40.341+0000 7f84d57e5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-31T05:22:40.726 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:40 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:40.261+0000 7fc366e47200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-31T05:22:40.892 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:40 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:40.529+0000 7f0b391c9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-31T05:22:41.133 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:40 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:40.840+0000 7fc366e47200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-31T05:22:41.134 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:40 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:40.936+0000 7fc366e47200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-31T05:22:41.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:41.210+0000 7fbf53fff640 1 -- 172.21.15.45:0/672039066 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 20) v1 ==== 135912+0+0 (secure 0 0 0) 0x7fbf600b8310 con 0x7fbf6c104120 2024-01-31T05:22:41.212 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:40 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:40.918+0000 7f84d57e5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-31T05:22:41.212 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:41 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:41.015+0000 7f84d57e5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-31T05:22:41.392 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.114+0000 7f0b391c9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-31T05:22:41.392 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.222+0000 7f0b391c9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-31T05:22:41.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-mon[31585]: Standby manager daemon y restarted 2024-01-31T05:22:41.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-mon[31585]: Standby manager daemon y started 2024-01-31T05:22:41.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.156:0/970907215' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-31T05:22:41.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.156:0/970907215' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-31T05:22:41.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.156:0/970907215' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-31T05:22:41.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.156:0/970907215' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-31T05:22:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:41.456+0000 7fbf53fff640 1 -- 172.21.15.45:0/672039066 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 21) v1 ==== 135580+0+0 (secure 0 0 0) 0x7fbf600ec610 con 0x7fbf6c104120 2024-01-31T05:22:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:41.457+0000 7fbf53fff640 1 -- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 msgr2=0x7fbf540a3890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:22:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:41.457+0000 7fbf53fff640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/2788245525,v1:172.21.15.45:6801/2788245525] conn(0x7fbf540a13d0 0x7fbf540a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:41.476 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:22:41 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y[37100]: 2024-01-31T05:22:41.134+0000 7fc366e47200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-31T05:22:41.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:41 smithi156 ceph-mon[35720]: Standby manager daemon y restarted 2024-01-31T05:22:41.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:41 smithi156 ceph-mon[35720]: Standby manager daemon y started 2024-01-31T05:22:41.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:41 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.156:0/970907215' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-31T05:22:41.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:41 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.156:0/970907215' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-31T05:22:41.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:41 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.156:0/970907215' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-31T05:22:41.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:41 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.156:0/970907215' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-31T05:22:41.591 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:22:41 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z[37517]: 2024-01-31T05:22:41.217+0000 7f84d57e5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-31T05:22:41.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:41 smithi186 ceph-mon[35659]: Standby manager daemon y restarted 2024-01-31T05:22:41.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:41 smithi186 ceph-mon[35659]: Standby manager daemon y started 2024-01-31T05:22:41.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:41 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.156:0/970907215' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-31T05:22:41.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:41 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.156:0/970907215' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-31T05:22:41.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:41 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.156:0/970907215' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-31T05:22:41.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:41 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.156:0/970907215' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-31T05:22:41.686 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.433+0000 7f0b391c9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-31T05:22:42.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.687+0000 7f0a9ba06700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:42.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.687+0000 7f0a9ba06700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:42.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.687+0000 7f0a9ba06700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:42.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.687+0000 7f0a9ba06700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:42.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.687+0000 7f0a9ba06700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:42.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.688+0000 7f0a981ff700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:42.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.688+0000 7f0a981ff700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:42.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.688+0000 7f0a981ff700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:42.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.688+0000 7f0a981ff700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:42.143 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:22:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:22:41.688+0000 7f0a981ff700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:22:42.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.465+0000 7fbf53fff640 1 -- 172.21.15.45:0/672039066 <== mon.0 v2:172.21.15.45:3300/0 9 ==== mgrmap(e 22) v1 ==== 135707+0+0 (secure 0 0 0) 0x7fbf600ac040 con 0x7fbf6c104120 2024-01-31T05:22:42.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.465+0000 7fbf53fff640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fbf54034020 0x7fbf54034400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:42.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.465+0000 7fbf53fff640 1 -- 172.21.15.45:0/672039066 --> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf58000bf0 con 0x7fbf54034020 2024-01-31T05:22:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.467+0000 7fbf6affd640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fbf54034020 0x7fbf54034400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:42.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.470+0000 7fbf6affd640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fbf54034020 0x7fbf54034400 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c005d00 tx=0x7fbf5c00b410 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:42.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-01-31T05:22:42.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: Standby manager daemon z restarted 2024-01-31T05:22:42.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: Standby manager daemon z started 2024-01-31T05:22:42.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.186:0/220183389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-31T05:22:42.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.186:0/220183389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-31T05:22:42.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.186:0/220183389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-31T05:22:42.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.? 172.21.15.186:0/220183389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-31T05:22:42.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: Active manager daemon x restarted 2024-01-31T05:22:42.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: Activating manager daemon x 2024-01-31T05:22:42.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: osdmap e89: 12 total, 12 up, 12 in 2024-01-31T05:22:42.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: mgrmap e21: x(active, starting, since 0.0111621s), standbys: y, z 2024-01-31T05:22:42.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:22:42.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:22:42.478 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:22:42.478 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-31T05:22:42.478 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-31T05:22:42.478 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-31T05:22:42.478 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-31T05:22:42.478 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-31T05:22:42.478 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-31T05:22:42.478 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-31T05:22:42.478 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-31T05:22:42.478 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:22:42.478 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-31T05:22:42.479 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-31T05:22:42.479 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-31T05:22:42.479 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-31T05:22:42.479 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-31T05:22:42.479 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-31T05:22:42.479 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:22:42.479 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:22:42.479 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:22:42.479 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:22:42.479 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:22:42.479 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:22:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:22:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:22:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:22:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:22:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:22:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:22:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-31T05:22:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-31T05:22:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-31T05:22:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: Manager daemon x is now available 2024-01-31T05:22:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:22:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:22:42.481 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-31T05:22:42.481 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:42 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-31T05:22:42.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.533+0000 7fbf53fff640 1 -- 172.21.15.45:0/672039066 <== mgr.15315 v2:172.21.15.45:6800/1984996061 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbf58000bf0 con 0x7fbf54034020 2024-01-31T05:22:42.532 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path / 2024-01-31T05:22:42.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.535+0000 7fbf714b1640 1 -- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fbf54034020 msgr2=0x7fbf54034400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:42.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.535+0000 7fbf714b1640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fbf54034020 0x7fbf54034400 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c005d00 tx=0x7fbf5c00b410 comp rx=0 tx=0).stop 2024-01-31T05:22:42.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.535+0000 7fbf714b1640 1 -- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf6c104120 msgr2=0x7fbf6c0ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:42.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.535+0000 7fbf714b1640 1 --2- 172.21.15.45:0/672039066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf6c104120 0x7fbf6c0ff7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbf60036040 tx=0x7fbf60004ed0 comp rx=0 tx=0).stop 2024-01-31T05:22:42.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.535+0000 7fbf6b7fe640 1 -- 172.21.15.45:0/672039066 reap_dead start 2024-01-31T05:22:42.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.535+0000 7fbf714b1640 1 -- 172.21.15.45:0/672039066 shutdown_connections 2024-01-31T05:22:42.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.535+0000 7fbf714b1640 1 -- 172.21.15.45:0/672039066 >> 172.21.15.45:0/672039066 conn(0x7fbf6c0fb180 msgr2=0x7fbf6c1023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:42.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.536+0000 7fbf714b1640 1 -- 172.21.15.45:0/672039066 shutdown_connections 2024-01-31T05:22:42.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.536+0000 7fbf714b1640 1 -- 172.21.15.45:0/672039066 wait complete. 2024-01-31T05:22:42.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-01-31T05:22:42.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: Standby manager daemon z restarted 2024-01-31T05:22:42.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: Standby manager daemon z started 2024-01-31T05:22:42.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.186:0/220183389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-31T05:22:42.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.186:0/220183389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-31T05:22:42.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.186:0/220183389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.? 172.21.15.186:0/220183389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: Active manager daemon x restarted 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: Activating manager daemon x 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: osdmap e89: 12 total, 12 up, 12 in 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: mgrmap e21: x(active, starting, since 0.0111621s), standbys: y, z 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-31T05:22:42.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:22:42.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: Manager daemon x is now available 2024-01-31T05:22:42.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:22:42.549 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:22:42.549 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-31T05:22:42.549 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:42 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-31T05:22:42.550 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-01-31T05:22:42.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-01-31T05:22:42.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: Standby manager daemon z restarted 2024-01-31T05:22:42.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: Standby manager daemon z started 2024-01-31T05:22:42.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.186:0/220183389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-31T05:22:42.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.186:0/220183389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-31T05:22:42.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.186:0/220183389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-31T05:22:42.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.? 172.21.15.186:0/220183389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-31T05:22:42.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: Active manager daemon x restarted 2024-01-31T05:22:42.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: Activating manager daemon x 2024-01-31T05:22:42.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: osdmap e89: 12 total, 12 up, 12 in 2024-01-31T05:22:42.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: mgrmap e21: x(active, starting, since 0.0111621s), standbys: y, z 2024-01-31T05:22:42.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-31T05:22:42.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-31T05:22:42.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-31T05:22:42.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-31T05:22:42.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-31T05:22:42.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-31T05:22:42.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-31T05:22:42.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-31T05:22:42.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-31T05:22:42.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-31T05:22:42.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-31T05:22:42.593 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-31T05:22:42.594 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: Manager daemon x is now available 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-31T05:22:42.595 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:42 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-31T05:22:42.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.658+0000 7f1e8842a640 1 -- 172.21.15.45:0/2640206043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e801017d0 msgr2=0x7f1e80108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:42.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.658+0000 7f1e8842a640 1 --2- 172.21.15.45:0/2640206043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e801017d0 0x7f1e80108060 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009ca0 tx=0x7f1e7802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:42.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.658+0000 7f1e8842a640 1 -- 172.21.15.45:0/2640206043 shutdown_connections 2024-01-31T05:22:42.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.658+0000 7f1e8842a640 1 --2- 172.21.15.45:0/2640206043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e801017d0 0x7f1e80108060 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:42.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.658+0000 7f1e8842a640 1 --2- 172.21.15.45:0/2640206043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e80100e10 0x7f1e80101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:42.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.658+0000 7f1e8842a640 1 --2- 172.21.15.45:0/2640206043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e800ffc20 0x7f1e80100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:42.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.658+0000 7f1e8842a640 1 -- 172.21.15.45:0/2640206043 >> 172.21.15.45:0/2640206043 conn(0x7f1e800fb190 msgr2=0x7f1e800fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:42.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.658+0000 7f1e8842a640 1 -- 172.21.15.45:0/2640206043 shutdown_connections 2024-01-31T05:22:42.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.658+0000 7f1e8842a640 1 -- 172.21.15.45:0/2640206043 wait complete. 2024-01-31T05:22:42.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.658+0000 7f1e8842a640 1 Processor -- start 2024-01-31T05:22:42.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e8842a640 1 -- start start 2024-01-31T05:22:42.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e8842a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e800ffc20 0x7f1e80103bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e8842a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e80100e10 0x7f1e801021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e8842a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e801017d0 0x7f1e80102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e8842a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e80102e90 con 0x7f1e80100e10 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e8842a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e80103000 con 0x7f1e800ffc20 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e8842a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e801a0c20 con 0x7f1e801017d0 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e859c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e80100e10 0x7f1e801021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e859c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e80100e10 0x7f1e801021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43984/0 (socket says 172.21.15.45:43984) 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e859c5640 1 -- 172.21.15.45:0/1058820030 learned_addr learned my addr 172.21.15.45:0/1058820030 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e861c6640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e800ffc20 0x7f1e80103bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e869c7640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e801017d0 0x7f1e80102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e859c5640 1 -- 172.21.15.45:0/1058820030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e801017d0 msgr2=0x7f1e80102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e859c5640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e801017d0 0x7f1e80102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:42.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e859c5640 1 -- 172.21.15.45:0/1058820030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e800ffc20 msgr2=0x7f1e80103bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:42.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e859c5640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e800ffc20 0x7f1e80103bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:42.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e859c5640 1 -- 172.21.15.45:0/1058820030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e78009900 con 0x7f1e80100e10 2024-01-31T05:22:42.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e869c7640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e801017d0 0x7f1e80102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:42.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e859c5640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e80100e10 0x7f1e801021f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000d980 tx=0x7f1e7000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:42.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e777fe640 1 -- 172.21.15.45:0/1058820030 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70009a30 con 0x7f1e80100e10 2024-01-31T05:22:42.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.659+0000 7f1e777fe640 1 -- 172.21.15.45:0/1058820030 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e70009b90 con 0x7f1e80100e10 2024-01-31T05:22:42.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.660+0000 7f1e8842a640 1 -- 172.21.15.45:0/1058820030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e801a0ed0 con 0x7f1e80100e10 2024-01-31T05:22:42.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.660+0000 7f1e777fe640 1 -- 172.21.15.45:0/1058820030 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e700047a0 con 0x7f1e80100e10 2024-01-31T05:22:42.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.660+0000 7f1e8842a640 1 -- 172.21.15.45:0/1058820030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e801a1350 con 0x7f1e80100e10 2024-01-31T05:22:42.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.661+0000 7f1e8842a640 1 -- 172.21.15.45:0/1058820030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8010b4e0 con 0x7f1e80100e10 2024-01-31T05:22:42.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.662+0000 7f1e777fe640 1 -- 172.21.15.45:0/1058820030 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 22) v1 ==== 135707+0+0 (secure 0 0 0) 0x7f1e70004020 con 0x7f1e80100e10 2024-01-31T05:22:42.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.662+0000 7f1e777fe640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1e6c0a1190 0x7f1e6c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:42.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.662+0000 7f1e777fe640 1 -- 172.21.15.45:0/1058820030 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1e700cc7f0 con 0x7f1e80100e10 2024-01-31T05:22:42.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.662+0000 7f1e861c6640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1e6c0a1190 0x7f1e6c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:42.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.663+0000 7f1e861c6640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1e6c0a1190 0x7f1e6c0a3650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0058c0 tx=0x7f1e7c00a2d0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:42.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.666+0000 7f1e777fe640 1 -- 172.21.15.45:0/1058820030 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1e70095e90 con 0x7f1e80100e10 2024-01-31T05:22:42.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.877+0000 7f1e8842a640 1 -- 172.21.15.45:0/1058820030 --> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e800fd2b0 con 0x7f1e6c0a1190 2024-01-31T05:22:42.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.930+0000 7f1e777fe640 1 -- 172.21.15.45:0/1058820030 <== mgr.15315 v2:172.21.15.45:6800/1984996061 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f1e800fd2b0 con 0x7f1e6c0a1190 2024-01-31T05:22:42.930 INFO:teuthology.orchestra.run.smithi045.stdout:Retention added to path / 2024-01-31T05:22:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.933+0000 7f1e8842a640 1 -- 172.21.15.45:0/1058820030 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1e6c0a1190 msgr2=0x7f1e6c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.933+0000 7f1e8842a640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1e6c0a1190 0x7f1e6c0a3650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0058c0 tx=0x7f1e7c00a2d0 comp rx=0 tx=0).stop 2024-01-31T05:22:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.933+0000 7f1e8842a640 1 -- 172.21.15.45:0/1058820030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e80100e10 msgr2=0x7f1e801021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.933+0000 7f1e8842a640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e80100e10 0x7f1e801021f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000d980 tx=0x7f1e7000de40 comp rx=0 tx=0).stop 2024-01-31T05:22:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.933+0000 7f1e8842a640 1 -- 172.21.15.45:0/1058820030 shutdown_connections 2024-01-31T05:22:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.933+0000 7f1e8842a640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1e6c0a1190 0x7f1e6c0a3650 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.933+0000 7f1e8842a640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e80100e10 0x7f1e801021f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.933+0000 7f1e8842a640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e801017d0 0x7f1e80102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.933+0000 7f1e8842a640 1 --2- 172.21.15.45:0/1058820030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e800ffc20 0x7f1e80103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.933+0000 7f1e8842a640 1 -- 172.21.15.45:0/1058820030 >> 172.21.15.45:0/1058820030 conn(0x7f1e800fb190 msgr2=0x7f1e800fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.933+0000 7f1e8842a640 1 -- 172.21.15.45:0/1058820030 shutdown_connections 2024-01-31T05:22:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:42.933+0000 7f1e8842a640 1 -- 172.21.15.45:0/1058820030 wait complete. 2024-01-31T05:22:42.942 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-01-31T05:22:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.048+0000 7f958ed8d640 1 -- 172.21.15.45:0/2622850281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588102320 msgr2=0x7f9588102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.048+0000 7f958ed8d640 1 --2- 172.21.15.45:0/2622850281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588102320 0x7f9588102740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f957c009900 tx=0x7f957c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 -- 172.21.15.45:0/2622850281 shutdown_connections 2024-01-31T05:22:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 --2- 172.21.15.45:0/2622850281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95880ff750 0x7f9588109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 --2- 172.21.15.45:0/2622850281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95880fed90 0x7f95880ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 --2- 172.21.15.45:0/2622850281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588102320 0x7f9588102740 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 -- 172.21.15.45:0/2622850281 >> 172.21.15.45:0/2622850281 conn(0x7f95880fa950 msgr2=0x7f95880fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 -- 172.21.15.45:0/2622850281 shutdown_connections 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 -- 172.21.15.45:0/2622850281 wait complete. 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 Processor -- start 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 -- start start 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880fed90 0x7f9588196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95880ff750 0x7f95881971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9588102320 0x7f958819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958819e780 con 0x7f95880fed90 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958819e8f0 con 0x7f9588102320 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958ed8d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958819eab0 con 0x7f95880ff750 2024-01-31T05:22:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958cb29640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880fed90 0x7f9588196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958cb29640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880fed90 0x7f9588196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44002/0 (socket says 172.21.15.45:44002) 2024-01-31T05:22:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f958cb29640 1 -- 172.21.15.45:0/2473971718 learned_addr learned my addr 172.21.15.45:0/2473971718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.049+0000 7f9587fff640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95880ff750 0x7f95881971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f958d32a640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9588102320 0x7f958819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f958cb29640 1 -- 172.21.15.45:0/2473971718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95880ff750 msgr2=0x7f95881971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f958cb29640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95880ff750 0x7f95881971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f958cb29640 1 -- 172.21.15.45:0/2473971718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9588102320 msgr2=0x7f958819e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f958cb29640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9588102320 0x7f958819e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f958cb29640 1 -- 172.21.15.45:0/2473971718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c009560 con 0x7f95880fed90 2024-01-31T05:22:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f958d32a640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9588102320 0x7f958819e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:22:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f958cb29640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880fed90 0x7f9588196c70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f957c000c00 tx=0x7f957c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:43.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f9585ffb640 1 -- 172.21.15.45:0/2473971718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c004ac0 con 0x7f95880fed90 2024-01-31T05:22:43.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f958ed8d640 1 -- 172.21.15.45:0/2473971718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958819ed30 con 0x7f95880fed90 2024-01-31T05:22:43.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f9585ffb640 1 -- 172.21.15.45:0/2473971718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f957c0044e0 con 0x7f95880fed90 2024-01-31T05:22:43.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f9585ffb640 1 -- 172.21.15.45:0/2473971718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c03e450 con 0x7f95880fed90 2024-01-31T05:22:43.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.050+0000 7f958ed8d640 1 -- 172.21.15.45:0/2473971718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958819f1b0 con 0x7f95880fed90 2024-01-31T05:22:43.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.051+0000 7f958ed8d640 1 -- 172.21.15.45:0/2473971718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9588108180 con 0x7f95880fed90 2024-01-31T05:22:43.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.052+0000 7f9585ffb640 1 -- 172.21.15.45:0/2473971718 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 22) v1 ==== 135707+0+0 (secure 0 0 0) 0x7f957c004dd0 con 0x7f95880fed90 2024-01-31T05:22:43.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.053+0000 7f9585ffb640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f95780a1190 0x7f95780a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:43.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.053+0000 7f9587fff640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f95780a1190 0x7f95780a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:43.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.053+0000 7f9585ffb640 1 -- 172.21.15.45:0/2473971718 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f957c0ef180 con 0x7f95880fed90 2024-01-31T05:22:43.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.053+0000 7f9587fff640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f95780a1190 0x7f95780a3650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9588198170 tx=0x7f9574005f90 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:43.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.057+0000 7f9585ffb640 1 -- 172.21.15.45:0/2473971718 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f957c0b87e0 con 0x7f95880fed90 2024-01-31T05:22:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.267+0000 7f958ed8d640 1 -- 172.21.15.45:0/2473971718 --> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f958802cc20 con 0x7f95780a1190 2024-01-31T05:22:43.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.275+0000 7f9585ffb640 1 -- 172.21.15.45:0/2473971718 <== mgr.15315 v2:172.21.15.45:6800/1984996061 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f958802cc20 con 0x7f95780a1190 2024-01-31T05:22:43.275 INFO:teuthology.orchestra.run.smithi045.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-01-31T00:00:00", "created": "2024-01-31T05:22:42", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-01-31T05:22:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.277+0000 7f958ed8d640 1 -- 172.21.15.45:0/2473971718 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f95780a1190 msgr2=0x7f95780a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.277+0000 7f958ed8d640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f95780a1190 0x7f95780a3650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9588198170 tx=0x7f9574005f90 comp rx=0 tx=0).stop 2024-01-31T05:22:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.277+0000 7f958ed8d640 1 -- 172.21.15.45:0/2473971718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880fed90 msgr2=0x7f9588196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.277+0000 7f958ed8d640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880fed90 0x7f9588196c70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f957c000c00 tx=0x7f957c0042f0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.278+0000 7f958ed8d640 1 -- 172.21.15.45:0/2473971718 shutdown_connections 2024-01-31T05:22:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.278+0000 7f958ed8d640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f95780a1190 0x7f95780a3650 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.278+0000 7f958ed8d640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9588102320 0x7f958819e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.278+0000 7f958ed8d640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95880ff750 0x7f95881971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.278+0000 7f958ed8d640 1 --2- 172.21.15.45:0/2473971718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880fed90 0x7f9588196c70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.278+0000 7f958ed8d640 1 -- 172.21.15.45:0/2473971718 >> 172.21.15.45:0/2473971718 conn(0x7f95880fa950 msgr2=0x7f9588105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.278+0000 7f958ed8d640 1 -- 172.21.15.45:0/2473971718 shutdown_connections 2024-01-31T05:22:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.278+0000 7f958ed8d640 1 -- 172.21.15.45:0/2473971718 wait complete. 2024-01-31T05:22:43.289 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-01-31T05:22:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.399+0000 7fb2df99d640 1 -- 172.21.15.45:0/248801064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d81017c0 msgr2=0x7fb2d8108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.399+0000 7fb2df99d640 1 --2- 172.21.15.45:0/248801064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d81017c0 0x7fb2d8108050 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009ca0 tx=0x7fb2d002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.399+0000 7fb2df99d640 1 -- 172.21.15.45:0/248801064 shutdown_connections 2024-01-31T05:22:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.399+0000 7fb2df99d640 1 --2- 172.21.15.45:0/248801064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d81017c0 0x7fb2d8108050 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.399+0000 7fb2df99d640 1 --2- 172.21.15.45:0/248801064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d8100e00 0x7fb2d8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.399+0000 7fb2df99d640 1 --2- 172.21.15.45:0/248801064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d80ffc10 0x7fb2d8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.399+0000 7fb2df99d640 1 -- 172.21.15.45:0/248801064 >> 172.21.15.45:0/248801064 conn(0x7fb2d80fb180 msgr2=0x7fb2d80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.399+0000 7fb2df99d640 1 -- 172.21.15.45:0/248801064 shutdown_connections 2024-01-31T05:22:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.399+0000 7fb2df99d640 1 -- 172.21.15.45:0/248801064 wait complete. 2024-01-31T05:22:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2df99d640 1 Processor -- start 2024-01-31T05:22:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2df99d640 1 -- start start 2024-01-31T05:22:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2df99d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d80ffc10 0x7fb2d8068f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2df99d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8100e00 0x7fb2d819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2df99d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d81017c0 0x7fb2d819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2df99d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d819e6b0 con 0x7fb2d8100e00 2024-01-31T05:22:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2df99d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d819e820 con 0x7fb2d81017c0 2024-01-31T05:22:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2df99d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d819e9e0 con 0x7fb2d80ffc10 2024-01-31T05:22:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2dcf38640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8100e00 0x7fb2d819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2dcf38640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8100e00 0x7fb2d819b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44028/0 (socket says 172.21.15.45:44028) 2024-01-31T05:22:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2dcf38640 1 -- 172.21.15.45:0/795818275 learned_addr learned my addr 172.21.15.45:0/795818275 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2ddf3a640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d81017c0 0x7fb2d819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.400+0000 7fb2dd739640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d80ffc10 0x7fb2d8068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.401+0000 7fb2dcf38640 1 -- 172.21.15.45:0/795818275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d80ffc10 msgr2=0x7fb2d8068f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.401+0000 7fb2dcf38640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d80ffc10 0x7fb2d8068f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.401+0000 7fb2dcf38640 1 -- 172.21.15.45:0/795818275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d81017c0 msgr2=0x7fb2d819df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.401+0000 7fb2dcf38640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d81017c0 0x7fb2d819df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.401+0000 7fb2dcf38640 1 -- 172.21.15.45:0/795818275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d0009900 con 0x7fb2d8100e00 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.401+0000 7fb2ddf3a640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d81017c0 0x7fb2d819df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.401+0000 7fb2dcf38640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8100e00 0x7fb2d819b8b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800d950 tx=0x7fb2c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.401+0000 7fb2ce7fc640 1 -- 172.21.15.45:0/795818275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8009a30 con 0x7fb2d8100e00 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.401+0000 7fb2ce7fc640 1 -- 172.21.15.45:0/795818275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2c8009b90 con 0x7fb2d8100e00 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.401+0000 7fb2ce7fc640 1 -- 172.21.15.45:0/795818275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80047a0 con 0x7fb2d8100e00 2024-01-31T05:22:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.401+0000 7fb2df99d640 1 -- 172.21.15.45:0/795818275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d819ecc0 con 0x7fb2d8100e00 2024-01-31T05:22:43.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.402+0000 7fb2df99d640 1 -- 172.21.15.45:0/795818275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d819f1d0 con 0x7fb2d8100e00 2024-01-31T05:22:43.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.402+0000 7fb2df99d640 1 -- 172.21.15.45:0/795818275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d8106f30 con 0x7fb2d8100e00 2024-01-31T05:22:43.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.404+0000 7fb2ce7fc640 1 -- 172.21.15.45:0/795818275 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 22) v1 ==== 135707+0+0 (secure 0 0 0) 0x7fb2c8009d00 con 0x7fb2d8100e00 2024-01-31T05:22:43.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.404+0000 7fb2ce7fc640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb2c40a1210 0x7fb2c40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:43.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.405+0000 7fb2dd739640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb2c40a1210 0x7fb2c40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:43.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.405+0000 7fb2ce7fc640 1 -- 172.21.15.45:0/795818275 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb2c80cc070 con 0x7fb2d8100e00 2024-01-31T05:22:43.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.405+0000 7fb2dd739640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb2c40a1210 0x7fb2c40a36d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4005fd0 tx=0x7fb2d400b410 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.409+0000 7fb2ce7fc640 1 -- 172.21.15.45:0/795818275 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb2c80957e0 con 0x7fb2d8100e00 2024-01-31T05:22:43.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:43 smithi045 ceph-mon[31585]: [31/Jan/2024:05:22:41] ENGINE Bus STARTING 2024-01-31T05:22:43.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:43 smithi045 ceph-mon[31585]: [31/Jan/2024:05:22:42] ENGINE Serving on https://172.21.15.45:7150 2024-01-31T05:22:43.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:43 smithi045 ceph-mon[31585]: [31/Jan/2024:05:22:42] ENGINE Serving on http://172.21.15.45:8765 2024-01-31T05:22:43.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:43 smithi045 ceph-mon[31585]: [31/Jan/2024:05:22:42] ENGINE Bus STARTED 2024-01-31T05:22:43.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:43 smithi045 ceph-mon[31585]: mgrmap e22: x(active, since 1.02307s), standbys: y, z 2024-01-31T05:22:43.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:43 smithi045 ceph-mon[31585]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:43.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:43 smithi045 ceph-mon[31585]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:22:43.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.643+0000 7fb2df99d640 1 -- 172.21.15.45:0/795818275 --> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb2d802cbe0 con 0x7fb2c40a1210 2024-01-31T05:22:43.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.650+0000 7fb2ce7fc640 1 -- 172.21.15.45:0/795818275 <== mgr.15315 v2:172.21.15.45:6800/1984996061 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fb2d802cbe0 con 0x7fb2c40a1210 2024-01-31T05:22:43.649 INFO:teuthology.orchestra.run.smithi045.stdout:/ 1m 6m3h 2024-01-31T05:22:43.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.651+0000 7fb2df99d640 1 -- 172.21.15.45:0/795818275 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb2c40a1210 msgr2=0x7fb2c40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:43.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.651+0000 7fb2df99d640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb2c40a1210 0x7fb2c40a36d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4005fd0 tx=0x7fb2d400b410 comp rx=0 tx=0).stop 2024-01-31T05:22:43.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.651+0000 7fb2df99d640 1 -- 172.21.15.45:0/795818275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8100e00 msgr2=0x7fb2d819b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:43.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.652+0000 7fb2df99d640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8100e00 0x7fb2d819b8b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800d950 tx=0x7fb2c800de10 comp rx=0 tx=0).stop 2024-01-31T05:22:43.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.652+0000 7fb2df99d640 1 -- 172.21.15.45:0/795818275 shutdown_connections 2024-01-31T05:22:43.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.652+0000 7fb2df99d640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb2c40a1210 0x7fb2c40a36d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.652+0000 7fb2df99d640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8100e00 0x7fb2d819b8b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.652+0000 7fb2df99d640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d81017c0 0x7fb2d819df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.652+0000 7fb2df99d640 1 --2- 172.21.15.45:0/795818275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d80ffc10 0x7fb2d8068f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:43.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.652+0000 7fb2df99d640 1 -- 172.21.15.45:0/795818275 >> 172.21.15.45:0/795818275 conn(0x7fb2d80fb180 msgr2=0x7fb2d8104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:43.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.652+0000 7fb2df99d640 1 -- 172.21.15.45:0/795818275 shutdown_connections 2024-01-31T05:22:43.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:22:43.652+0000 7fb2df99d640 1 -- 172.21.15.45:0/795818275 wait complete. 2024-01-31T05:22:43.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-01-31T05:22:43.690 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-01-31T05:22:43.726 INFO:teuthology.run_tasks:Running task background_exec... 2024-01-31T05:22:43.739 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-01-31T05:22:43.739 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-01-31T05:22:43.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:43 smithi156 ceph-mon[35720]: [31/Jan/2024:05:22:41] ENGINE Bus STARTING 2024-01-31T05:22:43.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:43 smithi156 ceph-mon[35720]: [31/Jan/2024:05:22:42] ENGINE Serving on https://172.21.15.45:7150 2024-01-31T05:22:43.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:43 smithi156 ceph-mon[35720]: [31/Jan/2024:05:22:42] ENGINE Serving on http://172.21.15.45:8765 2024-01-31T05:22:43.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:43 smithi156 ceph-mon[35720]: [31/Jan/2024:05:22:42] ENGINE Bus STARTED 2024-01-31T05:22:43.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:43 smithi156 ceph-mon[35720]: mgrmap e22: x(active, since 1.02307s), standbys: y, z 2024-01-31T05:22:43.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:43 smithi156 ceph-mon[35720]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:43.740 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:43 smithi156 ceph-mon[35720]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:22:43.744 INFO:teuthology.run_tasks:Running task workunit... 2024-01-31T05:22:43.775 INFO:tasks.workunit:Pulling workunits from ref 661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:22:43.776 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-01-31T05:22:43.776 INFO:tasks.workunit:timeout=3h 2024-01-31T05:22:43.776 INFO:tasks.workunit:cleanup=True 2024-01-31T05:22:43.776 DEBUG:teuthology.orchestra.run.smithi045:> stat -- /home/ubuntu/cephtest/mnt.0 2024-01-31T05:22:43.792 INFO:teuthology.orchestra.run.smithi045.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-01-31T05:22:43.792 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-01-31T05:22:43.792 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-01-31T05:22:43.792 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T05:22:43.792 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-01-31T05:22:43.793 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-01-31 05:20:45.879852334 +0000 2024-01-31T05:22:43.793 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-01-31 05:20:45.879852334 +0000 2024-01-31T05:22:43.793 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-01-31 05:22:32.217506450 +0000 2024-01-31T05:22:43.793 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-01-31 05:20:45.879852334 +0000 2024-01-31T05:22:43.793 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-01-31T05:22:43.793 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-01-31T05:22:43.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:43 smithi186 ceph-mon[35659]: [31/Jan/2024:05:22:41] ENGINE Bus STARTING 2024-01-31T05:22:43.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:43 smithi186 ceph-mon[35659]: [31/Jan/2024:05:22:42] ENGINE Serving on https://172.21.15.45:7150 2024-01-31T05:22:43.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:43 smithi186 ceph-mon[35659]: [31/Jan/2024:05:22:42] ENGINE Serving on http://172.21.15.45:8765 2024-01-31T05:22:43.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:43 smithi186 ceph-mon[35659]: [31/Jan/2024:05:22:42] ENGINE Bus STARTED 2024-01-31T05:22:43.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:43 smithi186 ceph-mon[35659]: mgrmap e22: x(active, since 1.02307s), standbys: y, z 2024-01-31T05:22:43.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:43 smithi186 ceph-mon[35659]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:43.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:43 smithi186 ceph-mon[35659]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:22:43.869 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 661946e7044723130c03627323f6a879a9848cb5 2024-01-31T05:22:43.925 INFO:tasks.workunit.client.0.smithi045.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-01-31T05:22:44.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:44 smithi186 ceph-mon[35659]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:44.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:44 smithi186 ceph-mon[35659]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:44.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:44 smithi186 ceph-mon[35659]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:22:44.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:44 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:44 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:44 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:44 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:44 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:44 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:44 smithi186 ceph-mon[35659]: mgrmap e23: x(active, since 3s), standbys: y, z 2024-01-31T05:22:44.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:44 smithi045 ceph-mon[31585]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:44.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:44 smithi045 ceph-mon[31585]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:44.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:44 smithi045 ceph-mon[31585]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:22:44.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:44 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:44 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:44 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:44 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:44 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:44 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:44 smithi045 ceph-mon[31585]: mgrmap e23: x(active, since 3s), standbys: y, z 2024-01-31T05:22:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:44 smithi156 ceph-mon[35720]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:44 smithi156 ceph-mon[35720]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:44 smithi156 ceph-mon[35720]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:22:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:44 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:44 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:44 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:44 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:44 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:44 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:44 smithi156 ceph-mon[35720]: mgrmap e23: x(active, since 3s), standbys: y, z 2024-01-31T05:22:45.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:22:45.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:45 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:22:45.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:45.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.876 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:22:45.876 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:45 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:22:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:22:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:22:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:22:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:45 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:22:46.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-01-31T05:22:46.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-01-31T05:22:46.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi186 to 174.1M 2024-01-31T05:22:46.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Unable to set osd_memory_target on smithi186 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-01-31T05:22:46.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-01-31T05:22:46.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Unable to set osd_memory_target on smithi045 to 185902796: error parsing value: Value '185902796' is below minimum 939524096 2024-01-31T05:22:46.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Updating smithi045:/etc/ceph/ceph.conf 2024-01-31T05:22:46.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Updating smithi156:/etc/ceph/ceph.conf 2024-01-31T05:22:46.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Updating smithi186:/etc/ceph/ceph.conf 2024-01-31T05:22:46.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Updating smithi045:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:22:46.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:22:46.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:22:46.844 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi186 to 174.1M 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Unable to set osd_memory_target on smithi186 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Unable to set osd_memory_target on smithi045 to 185902796: error parsing value: Value '185902796' is below minimum 939524096 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Updating smithi045:/etc/ceph/ceph.conf 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Updating smithi156:/etc/ceph/ceph.conf 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Updating smithi186:/etc/ceph/ceph.conf 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Updating smithi045:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:22:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:22:46.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-01-31T05:22:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi186 to 174.1M 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Unable to set osd_memory_target on smithi186 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Unable to set osd_memory_target on smithi045 to 185902796: error parsing value: Value '185902796' is below minimum 939524096 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Updating smithi045:/etc/ceph/ceph.conf 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Updating smithi156:/etc/ceph/ceph.conf 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Updating smithi186:/etc/ceph/ceph.conf 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Updating smithi045:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.conf 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:22:46.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-01-31T05:22:46.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:46.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:22:46.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:22:46.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:22:46.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:22:47.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:47 smithi186 ceph-mon[35659]: Updating smithi045:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:22:47.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:47 smithi186 ceph-mon[35659]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:22:47.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:47 smithi186 ceph-mon[35659]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:22:47.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:47 smithi045 ceph-mon[31585]: Updating smithi045:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:22:47.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:47 smithi045 ceph-mon[31585]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:22:47.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:47 smithi045 ceph-mon[31585]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:22:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:47 smithi156 ceph-mon[35720]: Updating smithi045:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:22:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:47 smithi156 ceph-mon[35720]: Updating smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:22:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:47 smithi156 ceph-mon[35720]: Updating smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/config/ceph.client.admin.keyring 2024-01-31T05:22:48.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:48 smithi186 ceph-mon[35659]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 90 KiB/s wr, 67 op/s 2024-01-31T05:22:48.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:48 smithi045 ceph-mon[31585]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 90 KiB/s wr, 67 op/s 2024-01-31T05:22:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:48 smithi156 ceph-mon[35720]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 90 KiB/s wr, 67 op/s 2024-01-31T05:22:50.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:50 smithi186 ceph-mon[35659]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-01-31T05:22:50.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:50 smithi045 ceph-mon[31585]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-01-31T05:22:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:50 smithi156 ceph-mon[35720]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-01-31T05:22:52.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:52 smithi186 ceph-mon[35659]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-01-31T05:22:52.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:52 smithi045 ceph-mon[31585]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-01-31T05:22:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:52 smithi156 ceph-mon[35720]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-01-31T05:22:54.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:54 smithi186 ceph-mon[35659]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-01-31T05:22:54.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:54 smithi045 ceph-mon[31585]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-01-31T05:22:54.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:54 smithi156 ceph-mon[35720]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-01-31T05:22:56.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:56 smithi186 ceph-mon[35659]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-01-31T05:22:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.886+0000 7f55d9889640 1 -- 172.21.15.45:0/3548312369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d4100f30 msgr2=0x7f55d4101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.886+0000 7f55d9889640 1 --2- 172.21.15.45:0/3548312369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d4100f30 0x7f55d4101310 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009930 tx=0x7f55c402bc30 comp rx=0 tx=0).stop 2024-01-31T05:22:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 -- 172.21.15.45:0/3548312369 shutdown_connections 2024-01-31T05:22:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 --2- 172.21.15.45:0/3548312369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4106000 0x7f55d410dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 --2- 172.21.15.45:0/3548312369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d41018e0 0x7f55d4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 --2- 172.21.15.45:0/3548312369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d4100f30 0x7f55d4101310 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 -- 172.21.15.45:0/3548312369 >> 172.21.15.45:0/3548312369 conn(0x7f55d40fc7d0 msgr2=0x7f55d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 -- 172.21.15.45:0/3548312369 shutdown_connections 2024-01-31T05:22:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 -- 172.21.15.45:0/3548312369 wait complete. 2024-01-31T05:22:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 Processor -- start 2024-01-31T05:22:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 -- start start 2024-01-31T05:22:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d4100f30 0x7f55d4072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4106000 0x7f55d4073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d4073a70 0x7f55d41a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.887+0000 7f55d9889640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d41a59a0 con 0x7f55d4073a70 2024-01-31T05:22:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d9889640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d41a5b10 con 0x7f55d4100f30 2024-01-31T05:22:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d9889640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d41a5cd0 con 0x7f55d4106000 2024-01-31T05:22:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d4073a70 0x7f55d41a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d4073a70 0x7f55d41a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35022/0 (socket says 172.21.15.45:35022) 2024-01-31T05:22:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d37fe640 1 -- 172.21.15.45:0/3253884590 learned_addr learned my addr 172.21.15.45:0/3253884590 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55cbfff640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4106000 0x7f55d4073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d2ffd640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d4100f30 0x7f55d4072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d37fe640 1 -- 172.21.15.45:0/3253884590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4106000 msgr2=0x7f55d4073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d37fe640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4106000 0x7f55d4073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d37fe640 1 -- 172.21.15.45:0/3253884590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d4100f30 msgr2=0x7f55d4072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d37fe640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d4100f30 0x7f55d4072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d37fe640 1 -- 172.21.15.45:0/3253884590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c4009560 con 0x7f55d4073a70 2024-01-31T05:22:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55cbfff640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4106000 0x7f55d4073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:22:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d37fe640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d4073a70 0x7f55d41a53d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00eb10 tx=0x7f55cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d8887640 1 -- 172.21.15.45:0/3253884590 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc004b20 con 0x7f55d4073a70 2024-01-31T05:22:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d8887640 1 -- 172.21.15.45:0/3253884590 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55cc004c80 con 0x7f55d4073a70 2024-01-31T05:22:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.888+0000 7f55d9889640 1 -- 172.21.15.45:0/3253884590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d41a5fb0 con 0x7f55d4073a70 2024-01-31T05:22:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.889+0000 7f55d8887640 1 -- 172.21.15.45:0/3253884590 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc010690 con 0x7f55d4073a70 2024-01-31T05:22:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.889+0000 7f55d9889640 1 -- 172.21.15.45:0/3253884590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d41a64e0 con 0x7f55d4073a70 2024-01-31T05:22:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55d8887640 1 -- 172.21.15.45:0/3253884590 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f55cc005020 con 0x7f55d4073a70 2024-01-31T05:22:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55dc64a740 1 Processor -- start 2024-01-31T05:22:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55dc64a740 1 -- start start 2024-01-31T05:22:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55dc64a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 0x55c93d19a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55dc64a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a980 0x55c93d1a3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55dc64a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 0x55c93d1a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55dc64a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93d109040 con 0x55c93d19a060 2024-01-31T05:22:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55dc64a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93d106730 con 0x55c93d19a980 2024-01-31T05:22:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55dc64a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93d102020 con 0x55c93d1a3750 2024-01-31T05:22:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55d8887640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f55c00a1640 0x7f55c00a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 0x55c93d19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 0x55c93d19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35028/0 (socket says 172.21.15.45:35028) 2024-01-31T05:22:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.891+0000 7f55cbfff640 1 -- 172.21.15.45:0/3732551828 learned_addr learned my addr 172.21.15.45:0/3732551828 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55d37fe640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f55c00a1640 0x7f55c00a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55cbfff640 1 --2- 172.21.15.45:0/3732551828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 0x55c93d1a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55d2ffd640 1 --2- 172.21.15.45:0/3732551828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a980 0x55c93d1a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55d8887640 1 -- 172.21.15.45:0/3253884590 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f55cc014070 con 0x7f55d4073a70 2024-01-31T05:22:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55cbfff640 1 -- 172.21.15.45:0/3732551828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 msgr2=0x55c93d1a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55cbfff640 1 --2- 172.21.15.45:0/3732551828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 0x55c93d1a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55cbfff640 1 -- 172.21.15.45:0/3732551828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a980 msgr2=0x55c93d1a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55cbfff640 1 --2- 172.21.15.45:0/3732551828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a980 0x55c93d1a3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55cbfff640 1 -- 172.21.15.45:0/3732551828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c93d0f9e40 con 0x55c93d19a060 2024-01-31T05:22:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55cbfff640 1 --2- 172.21.15.45:0/3732551828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 0x55c93d19a440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f55bc012b30 tx=0x7f55bc00bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d85e2308b73dbb2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55d37fe640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f55c00a1640 0x7f55c00a3b00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0caba0 tx=0x7f55cc06c1a0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55c9ffb640 1 -- 172.21.15.45:0/3732551828 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0120d0 con 0x55c93d19a060 2024-01-31T05:22:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55c9ffb640 1 -- 172.21.15.45:0/3732551828 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55bc012230 con 0x55c93d19a060 2024-01-31T05:22:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.892+0000 7f55c9ffb640 1 -- 172.21.15.45:0/3732551828 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00e810 con 0x55c93d19a060 2024-01-31T05:22:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.893+0000 7f55dc64a740 1 -- 172.21.15.45:0/3732551828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 msgr2=0x55c93d19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.893+0000 7f55dc64a740 1 --2- 172.21.15.45:0/3732551828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 0x55c93d19a440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f55bc012b30 tx=0x7f55bc00bae0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.893+0000 7f55dc64a740 1 -- 172.21.15.45:0/3732551828 shutdown_connections 2024-01-31T05:22:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.893+0000 7f55dc64a740 1 --2- 172.21.15.45:0/3732551828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 0x55c93d1a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.893+0000 7f55dc64a740 1 --2- 172.21.15.45:0/3732551828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a980 0x55c93d1a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.893+0000 7f55dc64a740 1 --2- 172.21.15.45:0/3732551828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 0x55c93d19a440 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.893+0000 7f55dc64a740 1 -- 172.21.15.45:0/3732551828 >> 172.21.15.45:0/3732551828 conn(0x55c93d194980 msgr2=0x55c93d194d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.893+0000 7f55dc64a740 1 -- 172.21.15.45:0/3732551828 shutdown_connections 2024-01-31T05:22:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.893+0000 7f55dc64a740 1 -- 172.21.15.45:0/3732551828 wait complete. 2024-01-31T05:22:56.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:56 smithi045 ceph-mon[31585]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-01-31T05:22:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.893+0000 7f55dc64a740 1 -- start start 2024-01-31T05:22:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55dc64a740 1 Processor -- start 2024-01-31T05:22:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55dc64a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d19a060 0x55c93d1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55dc64a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 0x55c93d1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55dc64a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d1a3750 0x55c93d1a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55dc64a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93d0bb1b0 con 0x55c93d19a980 2024-01-31T05:22:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55dc64a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93d0bae50 con 0x55c93d1a3750 2024-01-31T05:22:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55dc64a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93d080090 con 0x55c93d19a060 2024-01-31T05:22:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55dc64a740 10 client.0 _ll_register_callbacks cb 0x55c93d0a5a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55d2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 0x55c93d1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55d2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 0x55c93d1a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35042/0 (socket says 172.21.15.45:35042) 2024-01-31T05:22:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55d2ffd640 1 -- 172.21.15.45:0/1204217935 learned_addr learned my addr 172.21.15.45:0/1204217935 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55cbfff640 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d19a060 0x55c93d1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55cbfff640 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d1a3750 0x55c93d1a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55d2ffd640 1 -- 172.21.15.45:0/1204217935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d19a060 msgr2=0x55c93d1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55d2ffd640 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d19a060 0x55c93d1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55d2ffd640 1 -- 172.21.15.45:0/1204217935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d1a3750 msgr2=0x55c93d1a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55d2ffd640 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d1a3750 0x55c93d1a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.894+0000 7f55d2ffd640 1 -- 172.21.15.45:0/1204217935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc01a040 con 0x55c93d19a980 2024-01-31T05:22:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55d2ffd640 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 0x55c93d1a0b80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f55c4036040 tx=0x7f55c402c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6663785882cf90 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55b2ffd640 1 -- 172.21.15.45:0/1204217935 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c4004040 con 0x55c93d19a980 2024-01-31T05:22:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55b2ffd640 1 -- 172.21.15.45:0/1204217935 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f55c4046070 con 0x55c93d19a980 2024-01-31T05:22:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55dc64a740 1 -- 172.21.15.45:0/1204217935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c93d0fb010 con 0x55c93d19a980 2024-01-31T05:22:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55b2ffd640 1 -- 172.21.15.45:0/1204217935 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55c40041a0 con 0x55c93d19a980 2024-01-31T05:22:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55dc64a740 10 client.15399 fetch_fsmap learned FSMap version 21 2024-01-31T05:22:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55dc64a740 1 -- 172.21.15.45:0/1204217935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55c93d1006a0 con 0x55c93d19a980 2024-01-31T05:22:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55b2ffd640 1 -- 172.21.15.45:0/1204217935 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c93d0fb010 con 0x55c93d19a980 2024-01-31T05:22:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55b2ffd640 1 -- 172.21.15.45:0/1204217935 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f55c4052050 con 0x55c93d19a980 2024-01-31T05:22:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55dc64a740 10 client.15399 fetch_fsmap finished waiting for FSMap version 21 2024-01-31T05:22:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55dc64a740 10 client.15399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:22:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55dc64a740 20 client.15399 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55dc64a740 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c93d1bbc90 0x55c93d1be0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55dc64a740 4 client.15399 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:22:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.895+0000 7f55dc64a740 1 -- 172.21.15.45:0/1204217935 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c93d0fad30 con 0x55c93d1bbc90 2024-01-31T05:22:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.896+0000 7f55cbfff640 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c93d1bbc90 0x55c93d1be0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.896+0000 7f55b2ffd640 10 client.15399 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:22:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.896+0000 7f55cbfff640 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c93d1bbc90 0x55c93d1be0b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e68664e96a4c74 server_cookie=2c89cf5064eccfc1 in_seq=0 out_seq=0 2024-01-31T05:22:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55b2ffd640 1 -- 172.21.15.45:0/1204217935 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c93d0fad30 con 0x55c93d1bbc90 2024-01-31T05:22:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55b2ffd640 10 client.15399 handle_command_reply: tid=0 2024-01-31T05:22:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55dc64a740 1 client.15399 shutdown 2024-01-31T05:22:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55dc64a740 1 -- 172.21.15.45:0/1204217935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 msgr2=0x55c93d1a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55dc64a740 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 0x55c93d1a0b80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f55c4036040 tx=0x7f55c402c9b0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55dc64a740 1 -- 172.21.15.45:0/1204217935 shutdown_connections 2024-01-31T05:22:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55dc64a740 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c93d1bbc90 0x55c93d1be0b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55dc64a740 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 0x55c93d1a0b80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55dc64a740 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d1a3750 0x55c93d1a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55b2ffd640 0 client.15399 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:22:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55dc64a740 1 --2- 172.21.15.45:0/1204217935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d19a060 0x55c93d1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55dc64a740 1 -- 172.21.15.45:0/1204217935 >> 172.21.15.45:0/1204217935 conn(0x55c93d194980 msgr2=0x55c93d1966d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55dc64a740 1 -- 172.21.15.45:0/1204217935 shutdown_connections 2024-01-31T05:22:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55dc64a740 1 -- 172.21.15.45:0/1204217935 wait complete. 2024-01-31T05:22:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.897+0000 7f55dc64a740 20 client.15399 trim_cache size 0 max 16384 2024-01-31T05:22:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.910+0000 7f55dc64a740 1 Processor -- start 2024-01-31T05:22:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.910+0000 7f55dc64a740 1 -- start start 2024-01-31T05:22:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55dc64a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a060 0x55c93d1a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55dc64a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 0x55c93d1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55dc64a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 0x55c93d1c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55dc64a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93d19b5b0 con 0x55c93d19a980 2024-01-31T05:22:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55dc64a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93d194810 con 0x55c93d19a060 2024-01-31T05:22:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55dc64a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93d195780 con 0x55c93d1a3750 2024-01-31T05:22:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55d2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 0x55c93d1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55d2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 0x55c93d1a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35046/0 (socket says 172.21.15.45:35046) 2024-01-31T05:22:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55d2ffd640 1 -- 172.21.15.45:0/933366720 learned_addr learned my addr 172.21.15.45:0/933366720 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55cbfff640 1 --2- 172.21.15.45:0/933366720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 0x55c93d1c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55cbfff640 1 --2- 172.21.15.45:0/933366720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a060 0x55c93d1a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55d2ffd640 1 -- 172.21.15.45:0/933366720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 msgr2=0x55c93d1c2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55d2ffd640 1 --2- 172.21.15.45:0/933366720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 0x55c93d1c2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55d2ffd640 1 -- 172.21.15.45:0/933366720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a060 msgr2=0x55c93d1a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55d2ffd640 1 --2- 172.21.15.45:0/933366720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a060 0x55c93d1a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55d2ffd640 1 -- 172.21.15.45:0/933366720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c93d09d030 con 0x55c93d19a980 2024-01-31T05:22:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55d2ffd640 1 --2- 172.21.15.45:0/933366720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 0x55c93d1a0ee0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f55c4005a20 tx=0x7f55c402cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75111a17974d519e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55c97fa640 1 -- 172.21.15.45:0/933366720 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c4003a90 con 0x55c93d19a980 2024-01-31T05:22:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55c97fa640 1 -- 172.21.15.45:0/933366720 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55c4044070 con 0x55c93d19a980 2024-01-31T05:22:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.911+0000 7f55c97fa640 1 -- 172.21.15.45:0/933366720 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c40040d0 con 0x55c93d19a980 2024-01-31T05:22:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.912+0000 7f55dc64a740 1 -- 172.21.15.45:0/933366720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 msgr2=0x55c93d1a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.912+0000 7f55dc64a740 1 --2- 172.21.15.45:0/933366720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 0x55c93d1a0ee0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f55c4005a20 tx=0x7f55c402cd30 comp rx=0 tx=0).stop 2024-01-31T05:22:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.912+0000 7f55dc64a740 1 -- 172.21.15.45:0/933366720 shutdown_connections 2024-01-31T05:22:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.912+0000 7f55dc64a740 1 --2- 172.21.15.45:0/933366720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a980 0x55c93d1a0ee0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.912+0000 7f55dc64a740 1 --2- 172.21.15.45:0/933366720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 0x55c93d1c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.912+0000 7f55dc64a740 1 --2- 172.21.15.45:0/933366720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a060 0x55c93d1a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.912+0000 7f55dc64a740 1 -- 172.21.15.45:0/933366720 >> 172.21.15.45:0/933366720 conn(0x55c93d194980 msgr2=0x55c93d194d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.912+0000 7f55dc64a740 1 -- 172.21.15.45:0/933366720 shutdown_connections 2024-01-31T05:22:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.912+0000 7f55dc64a740 1 -- 172.21.15.45:0/933366720 wait complete. 2024-01-31T05:22:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.912+0000 7f55dc64a740 1 -- start start 2024-01-31T05:22:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55dc64a740 1 Processor -- start 2024-01-31T05:22:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55dc64a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 0x55c93d1b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55dc64a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a980 0x55c93d1b1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55dc64a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 0x55c93d1b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55dc64a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93d1b1e30 con 0x55c93d19a060 2024-01-31T05:22:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55dc64a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93d1b1fa0 con 0x55c93d19a980 2024-01-31T05:22:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55dc64a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93d1b2160 con 0x55c93d1a3750 2024-01-31T05:22:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55dc64a740 10 client.0 _ll_register_callbacks cb 0x7f55bc00be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:22:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 0x55c93d1b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 0x55c93d1b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35054/0 (socket says 172.21.15.45:35054) 2024-01-31T05:22:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55cbfff640 1 -- 172.21.15.45:0/4133702873 learned_addr learned my addr 172.21.15.45:0/4133702873 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:22:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55cbfff640 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 0x55c93d1b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55d2ffd640 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a980 0x55c93d1b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55cbfff640 1 -- 172.21.15.45:0/4133702873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 msgr2=0x55c93d1b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55cbfff640 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 0x55c93d1b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55cbfff640 1 -- 172.21.15.45:0/4133702873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a980 msgr2=0x55c93d1b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:22:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55cbfff640 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a980 0x55c93d1b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55cbfff640 1 -- 172.21.15.45:0/4133702873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c93d1006a0 con 0x55c93d19a060 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55d2ffd640 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a980 0x55c93d1b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55cbfff640 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 0x55c93d1b2e90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0054f0 tx=0x7f55bc023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2614226b4f98661 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.913+0000 7f55b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55b2ffd640 1 -- 172.21.15.45:0/4133702873 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc026040 con 0x55c93d19a060 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55b2ffd640 1 -- 172.21.15.45:0/4133702873 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f55bc00e7e0 con 0x55c93d19a060 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55dc64a740 1 -- 172.21.15.45:0/4133702873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55bc012230 con 0x55c93d19a060 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55b2ffd640 1 -- 172.21.15.45:0/4133702873 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55bc029b80 con 0x55c93d19a060 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55dc64a740 10 client.15405 fetch_fsmap learned FSMap version 21 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55dc64a740 1 -- 172.21.15.45:0/4133702873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55c93d0c2a50 con 0x55c93d19a060 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55b2ffd640 1 -- 172.21.15.45:0/4133702873 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55bc033030 con 0x55c93d19a060 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55b2ffd640 1 -- 172.21.15.45:0/4133702873 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f55bc0295c0 con 0x55c93d19a060 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55dc64a740 10 client.15405 fetch_fsmap finished waiting for FSMap version 21 2024-01-31T05:22:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55dc64a740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:22:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55dc64a740 20 client.15405 populate_metadata read hostname 'smithi045' 2024-01-31T05:22:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55dc64a740 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c93d213680 0x55c93d215aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:22:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55dc64a740 4 client.15405 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:22:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55dc64a740 1 -- 172.21.15.45:0/4133702873 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c93ce01600 con 0x55c93d213680 2024-01-31T05:22:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.914+0000 7f55d2ffd640 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c93d213680 0x55c93d215aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:22:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.915+0000 7f55d2ffd640 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c93d213680 0x55c93d215aa0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a13682337e80a3a server_cookie=7d0542ba54ac81e5 in_seq=0 out_seq=0 2024-01-31T05:22:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.915+0000 7f55b2ffd640 10 client.15405 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:22:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55b2ffd640 1 -- 172.21.15.45:0/4133702873 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c93ce01600 con 0x55c93d213680 2024-01-31T05:22:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55b2ffd640 10 client.15405 handle_command_reply: tid=0 2024-01-31T05:22:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55dc64a740 1 client.15405 shutdown 2024-01-31T05:22:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55dc64a740 1 -- 172.21.15.45:0/4133702873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 msgr2=0x55c93d1b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55dc64a740 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 0x55c93d1b2e90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0054f0 tx=0x7f55bc023040 comp rx=0 tx=0).stop 2024-01-31T05:22:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55dc64a740 1 -- 172.21.15.45:0/4133702873 shutdown_connections 2024-01-31T05:22:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55dc64a740 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c93d213680 0x55c93d215aa0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55dc64a740 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c93d1a3750 0x55c93d1b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55dc64a740 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c93d19a980 0x55c93d1b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55b2ffd640 0 client.15405 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:22:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55dc64a740 1 --2- 172.21.15.45:0/4133702873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c93d19a060 0x55c93d1b2e90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55dc64a740 1 -- 172.21.15.45:0/4133702873 >> 172.21.15.45:0/4133702873 conn(0x55c93d194980 msgr2=0x55c93d1c2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55dc64a740 1 -- 172.21.15.45:0/4133702873 shutdown_connections 2024-01-31T05:22:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.926+0000 7f55dc64a740 1 -- 172.21.15.45:0/4133702873 wait complete. 2024-01-31T05:22:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55dc64a740 20 client.15405 trim_cache size 0 max 16384 2024-01-31T05:22:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:22:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:22:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:22:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:22:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55d9889640 1 -- 172.21.15.45:0/3253884590 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f55c00a1640 msgr2=0x7f55c00a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55d9889640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f55c00a1640 0x7f55c00a3b00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0caba0 tx=0x7f55cc06c1a0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55d9889640 1 -- 172.21.15.45:0/3253884590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d4073a70 msgr2=0x7f55d41a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:22:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55d9889640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d4073a70 0x7f55d41a53d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00eb10 tx=0x7f55cc00c2d0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55d9889640 1 -- 172.21.15.45:0/3253884590 shutdown_connections 2024-01-31T05:22:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55d9889640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f55c00a1640 0x7f55c00a3b00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55d9889640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d4073a70 0x7f55d41a53d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55d9889640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4106000 0x7f55d4073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55d9889640 1 --2- 172.21.15.45:0/3253884590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d4100f30 0x7f55d4072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:22:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55d9889640 1 -- 172.21.15.45:0/3253884590 >> 172.21.15.45:0/3253884590 conn(0x7f55d40fc7d0 msgr2=0x7f55d40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:22:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55d9889640 1 -- 172.21.15.45:0/3253884590 shutdown_connections 2024-01-31T05:22:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:22:56.927+0000 7f55d9889640 1 -- 172.21.15.45:0/3253884590 wait complete. 2024-01-31T05:22:56.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:56 smithi156 ceph-mon[35720]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-01-31T05:22:58.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:22:58 smithi186 ceph-mon[35659]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-01-31T05:22:58.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:22:58 smithi045 ceph-mon[31585]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-01-31T05:22:58.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:22:58 smithi156 ceph-mon[35720]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-01-31T05:23:00.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:00 smithi186 ceph-mon[35659]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 KiB/s wr, 26 op/s 2024-01-31T05:23:00.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:00 smithi045 ceph-mon[31585]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 KiB/s wr, 26 op/s 2024-01-31T05:23:00.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:00 smithi156 ceph-mon[35720]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 KiB/s wr, 26 op/s 2024-01-31T05:23:01.392 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:23:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:23:00.942+0000 7f0a79e03700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:23:01.393 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:23:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:23:00.942+0000 7f0a79e03700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:23:01.393 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:23:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:23:00.942+0000 7f0a79e03700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:23:01.393 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:23:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:23:00.942+0000 7f0a79e03700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:23:01.393 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:23:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:23:00.942+0000 7f0a79e03700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-31T05:23:02.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:02 smithi186 ceph-mon[35659]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-01-31T05:23:02.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:02 smithi045 ceph-mon[31585]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-01-31T05:23:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:02 smithi156 ceph-mon[35720]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-01-31T05:23:03.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:03 smithi186 ceph-mon[35659]: mgrmap e25: x(active, since 21s), standbys: y, z 2024-01-31T05:23:03.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:03 smithi186 ceph-mon[35659]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-01-31T05:23:03.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:03 smithi045 ceph-mon[31585]: mgrmap e25: x(active, since 21s), standbys: y, z 2024-01-31T05:23:03.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:03 smithi045 ceph-mon[31585]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-01-31T05:23:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:03 smithi156 ceph-mon[35720]: mgrmap e25: x(active, since 21s), standbys: y, z 2024-01-31T05:23:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:03 smithi156 ceph-mon[35720]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-01-31T05:23:06.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:06 smithi186 ceph-mon[35659]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-01-31T05:23:06.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:06 smithi045 ceph-mon[31585]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-01-31T05:23:06.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:06 smithi156 ceph-mon[35720]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-01-31T05:23:08.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:08 smithi186 ceph-mon[35659]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-01-31T05:23:08.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:08 smithi045 ceph-mon[31585]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-01-31T05:23:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:08 smithi156 ceph-mon[35720]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-01-31T05:23:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.025+0000 7fc104d1f640 1 -- 172.21.15.45:0/2702545463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc100105fe0 msgr2=0x7fc10010dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.025+0000 7fc104d1f640 1 --2- 172.21.15.45:0/2702545463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc100105fe0 0x7fc10010dbb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009930 tx=0x7fc0f802bc30 comp rx=0 tx=0).stop 2024-01-31T05:23:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.025+0000 7fc104d1f640 1 -- 172.21.15.45:0/2702545463 shutdown_connections 2024-01-31T05:23:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.025+0000 7fc104d1f640 1 --2- 172.21.15.45:0/2702545463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc100105fe0 0x7fc10010dbb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.025+0000 7fc104d1f640 1 --2- 172.21.15.45:0/2702545463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1001018c0 0x7fc100105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.025+0000 7fc104d1f640 1 --2- 172.21.15.45:0/2702545463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc100100f10 0x7fc1001012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.025+0000 7fc104d1f640 1 -- 172.21.15.45:0/2702545463 >> 172.21.15.45:0/2702545463 conn(0x7fc1000fc7b0 msgr2=0x7fc1000febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.025+0000 7fc104d1f640 1 -- 172.21.15.45:0/2702545463 shutdown_connections 2024-01-31T05:23:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.025+0000 7fc104d1f640 1 -- 172.21.15.45:0/2702545463 wait complete. 2024-01-31T05:23:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.026+0000 7fc104d1f640 1 Processor -- start 2024-01-31T05:23:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.026+0000 7fc104d1f640 1 -- start start 2024-01-31T05:23:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.026+0000 7fc104d1f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc100100f10 0x7fc1001971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.026+0000 7fc104d1f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc100105fe0 0x7fc1001976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.026+0000 7fc104d1f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc10019ba60 0x7fc10019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.026+0000 7fc104d1f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10019c450 con 0x7fc100105fe0 2024-01-31T05:23:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.026+0000 7fc104d1f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10019c5c0 con 0x7fc100100f10 2024-01-31T05:23:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.026+0000 7fc104d1f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10019c730 con 0x7fc10019ba60 2024-01-31T05:23:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.026+0000 7fc0fdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc100105fe0 0x7fc1001976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0fdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc100105fe0 0x7fc1001976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56834/0 (socket says 172.21.15.45:56834) 2024-01-31T05:23:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0fdd9b640 1 -- 172.21.15.45:0/2635295503 learned_addr learned my addr 172.21.15.45:0/2635295503 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc10019ba60 0x7fc10019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0fe59c640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc100100f10 0x7fc1001971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0fdd9b640 1 -- 172.21.15.45:0/2635295503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc10019ba60 msgr2=0x7fc10019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0fdd9b640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc10019ba60 0x7fc10019bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0fdd9b640 1 -- 172.21.15.45:0/2635295503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc100100f10 msgr2=0x7fc1001971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0fdd9b640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc100100f10 0x7fc1001971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0fdd9b640 1 -- 172.21.15.45:0/2635295503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f8009560 con 0x7fc100105fe0 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0fe59c640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc100100f10 0x7fc1001971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc10019ba60 0x7fc10019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0fdd9b640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc100105fe0 0x7fc1001976e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c970 tx=0x7fc0f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0ef7fe640 1 -- 172.21.15.45:0/2635295503 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4007bb0 con 0x7fc100105fe0 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc104d1f640 1 -- 172.21.15.45:0/2635295503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1001a3d10 con 0x7fc100105fe0 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0ef7fe640 1 -- 172.21.15.45:0/2635295503 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f4007d10 con 0x7fc100105fe0 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc0ef7fe640 1 -- 172.21.15.45:0/2635295503 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4004e20 con 0x7fc100105fe0 2024-01-31T05:23:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.027+0000 7fc104d1f640 1 -- 172.21.15.45:0/2635295503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1001a41c0 con 0x7fc100105fe0 2024-01-31T05:23:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.030+0000 7fc0ef7fe640 1 -- 172.21.15.45:0/2635295503 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136128+0+0 (secure 0 0 0) 0x7fc0f4004fc0 con 0x7fc100105fe0 2024-01-31T05:23:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.030+0000 7fc107bb8740 1 Processor -- start 2024-01-31T05:23:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.030+0000 7fc107bb8740 1 -- start start 2024-01-31T05:23:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.030+0000 7fc0ef7fe640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fc0e80a1690 0x7fc0e80a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc107bb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 0x5636d0578f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc107bb8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d05794a0 0x5636d0586620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc107bb8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d0586b60 0x5636d0588f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc107bb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d04e8040 con 0x5636d0578b80 2024-01-31T05:23:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc107bb8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d04e5730 con 0x5636d0586b60 2024-01-31T05:23:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc107bb8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d04e1020 con 0x5636d05794a0 2024-01-31T05:23:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0ef7fe640 1 -- 172.21.15.45:0/2635295503 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc0f40cce00 con 0x7fc100105fe0 2024-01-31T05:23:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fc0e80a1690 0x7fc0e80a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 0x5636d0578f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 0x5636d0578f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56848/0 (socket says 172.21.15.45:56848) 2024-01-31T05:23:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fe59c640 1 -- 172.21.15.45:0/146080961 learned_addr learned my addr 172.21.15.45:0/146080961 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fdd9b640 1 --2- 172.21.15.45:0/146080961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d05794a0 0x5636d0586620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/146080961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d0586b60 0x5636d0588f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fe59c640 1 -- 172.21.15.45:0/146080961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d05794a0 msgr2=0x5636d0586620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fe59c640 1 --2- 172.21.15.45:0/146080961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d05794a0 0x5636d0586620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fe59c640 1 -- 172.21.15.45:0/146080961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d0586b60 msgr2=0x5636d0588f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fe59c640 1 --2- 172.21.15.45:0/146080961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d0586b60 0x5636d0588f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fe59c640 1 -- 172.21.15.45:0/146080961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636d04d8e40 con 0x5636d0578b80 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/146080961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d0586b60 0x5636d0588f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fc0e80a1690 0x7fc0e80a3b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8006010 tx=0x7fc0f80047a0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.031+0000 7fc0fe59c640 1 --2- 172.21.15.45:0/146080961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 0x5636d0578f60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00103a0 tx=0x7fc0f0010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee654106b9a7845 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.032+0000 7fc0ecff9640 1 -- 172.21.15.45:0/146080961 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0012710 con 0x5636d0578b80 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.032+0000 7fc0ecff9640 1 -- 172.21.15.45:0/146080961 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f0012d30 con 0x5636d0578b80 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.032+0000 7fc0ecff9640 1 -- 172.21.15.45:0/146080961 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f001bdc0 con 0x5636d0578b80 2024-01-31T05:23:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.032+0000 7fc107bb8740 1 -- 172.21.15.45:0/146080961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 msgr2=0x5636d0578f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.032+0000 7fc107bb8740 1 --2- 172.21.15.45:0/146080961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 0x5636d0578f60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00103a0 tx=0x7fc0f0010860 comp rx=0 tx=0).stop 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.032+0000 7fc107bb8740 1 -- 172.21.15.45:0/146080961 shutdown_connections 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.032+0000 7fc107bb8740 1 --2- 172.21.15.45:0/146080961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d0586b60 0x5636d0588f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.032+0000 7fc107bb8740 1 --2- 172.21.15.45:0/146080961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d05794a0 0x5636d0586620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.032+0000 7fc107bb8740 1 --2- 172.21.15.45:0/146080961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 0x5636d0578f60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.032+0000 7fc107bb8740 1 -- 172.21.15.45:0/146080961 >> 172.21.15.45:0/146080961 conn(0x5636d0573980 msgr2=0x5636d0573d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.032+0000 7fc107bb8740 1 -- 172.21.15.45:0/146080961 shutdown_connections 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.032+0000 7fc107bb8740 1 -- 172.21.15.45:0/146080961 wait complete. 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc107bb8740 1 -- start start 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc107bb8740 1 Processor -- start 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc107bb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 0x5636d0583a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc107bb8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d05794a0 0x5636d0583fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc107bb8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d0586b60 0x5636d05844e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc107bb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d049a1b0 con 0x5636d0578b80 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc107bb8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d0499e50 con 0x5636d0586b60 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc107bb8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d045f090 con 0x5636d05794a0 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc107bb8740 10 client.0 _ll_register_callbacks cb 0x5636d0484a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc0fe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 0x5636d0583a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc0fe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 0x5636d0583a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56864/0 (socket says 172.21.15.45:56864) 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc0fe59c640 1 -- 172.21.15.45:0/1771541885 learned_addr learned my addr 172.21.15.45:0/1771541885 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d0586b60 0x5636d05844e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.033+0000 7fc0fdd9b640 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d05794a0 0x5636d0583fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc0fe59c640 1 -- 172.21.15.45:0/1771541885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d05794a0 msgr2=0x5636d0583fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc0fe59c640 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d05794a0 0x5636d0583fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc0fe59c640 1 -- 172.21.15.45:0/1771541885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d0586b60 msgr2=0x5636d05844e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc0fe59c640 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d0586b60 0x5636d05844e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc0fe59c640 1 -- 172.21.15.45:0/1771541885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f0009c60 con 0x5636d0578b80 2024-01-31T05:23:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc0fdd9b640 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d05794a0 0x5636d0583fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc0fe59c640 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 0x5636d0583a60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc0f001baa0 tx=0x7fc0f001b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c442f92b00f6f25 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc0e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc0e5ffb640 1 -- 172.21.15.45:0/1771541885 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0021070 con 0x5636d0578b80 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc0e5ffb640 1 -- 172.21.15.45:0/1771541885 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc0f001bd50 con 0x5636d0578b80 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc107bb8740 1 -- 172.21.15.45:0/1771541885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636d04da010 con 0x5636d0578b80 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc0e5ffb640 1 -- 172.21.15.45:0/1771541885 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0f0003970 con 0x5636d0578b80 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc107bb8740 10 client.15423 fetch_fsmap learned FSMap version 21 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc107bb8740 1 -- 172.21.15.45:0/1771541885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5636d04df6a0 con 0x5636d0578b80 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.034+0000 7fc0e5ffb640 1 -- 172.21.15.45:0/1771541885 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f0034020 con 0x5636d0578b80 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.035+0000 7fc0e5ffb640 1 -- 172.21.15.45:0/1771541885 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc0f0004040 con 0x5636d0578b80 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.035+0000 7fc107bb8740 10 client.15423 fetch_fsmap finished waiting for FSMap version 21 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.035+0000 7fc107bb8740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.035+0000 7fc107bb8740 20 client.15423 populate_metadata read hostname 'smithi045' 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.035+0000 7fc107bb8740 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5636d0596750 0x5636d0598b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.035+0000 7fc107bb8740 4 client.15423 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.035+0000 7fc107bb8740 1 -- 172.21.15.45:0/1771541885 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636d04d9d30 con 0x5636d0596750 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.035+0000 7fc0fdd9b640 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5636d0596750 0x5636d0598b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.035+0000 7fc0e5ffb640 10 client.15423 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.035+0000 7fc0fdd9b640 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5636d0596750 0x5636d0598b40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea46bc86eaef5678 server_cookie=7e3cdc4f22562737 in_seq=0 out_seq=0 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc0e5ffb640 1 -- 172.21.15.45:0/1771541885 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5636d04d9d30 con 0x5636d0596750 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc0e5ffb640 10 client.15423 handle_command_reply: tid=0 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc107bb8740 1 client.15423 shutdown 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc107bb8740 1 -- 172.21.15.45:0/1771541885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 msgr2=0x5636d0583a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc107bb8740 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 0x5636d0583a60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc0f001baa0 tx=0x7fc0f001b690 comp rx=0 tx=0).stop 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc107bb8740 1 -- 172.21.15.45:0/1771541885 shutdown_connections 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc107bb8740 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5636d0596750 0x5636d0598b40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc107bb8740 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d0586b60 0x5636d05844e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc107bb8740 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d05794a0 0x5636d0583fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc0e5ffb640 0 client.15423 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc107bb8740 1 --2- 172.21.15.45:0/1771541885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0578b80 0x5636d0583a60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc107bb8740 1 -- 172.21.15.45:0/1771541885 >> 172.21.15.45:0/1771541885 conn(0x5636d0573980 msgr2=0x5636d05756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc107bb8740 1 -- 172.21.15.45:0/1771541885 shutdown_connections 2024-01-31T05:23:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc107bb8740 1 -- 172.21.15.45:0/1771541885 wait complete. 2024-01-31T05:23:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.036+0000 7fc107bb8740 20 client.15423 trim_cache size 0 max 16384 2024-01-31T05:23:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc107bb8740 1 Processor -- start 2024-01-31T05:23:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc107bb8740 1 -- start start 2024-01-31T05:23:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc107bb8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d0578b80 0x5636d05754e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc107bb8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d05794a0 0x5636d058d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc107bb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 0x5636d058dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc107bb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d056f0b0 con 0x5636d0586b60 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc107bb8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d0584db0 con 0x5636d05794a0 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc107bb8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d059dd60 con 0x5636d0578b80 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc0fed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 0x5636d058dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc0fed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 0x5636d058dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56870/0 (socket says 172.21.15.45:56870) 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc0fed9d640 1 -- 172.21.15.45:0/2713967440 learned_addr learned my addr 172.21.15.45:0/2713967440 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc0fdd9b640 1 --2- 172.21.15.45:0/2713967440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d05794a0 0x5636d058d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.050+0000 7fc0fe59c640 1 --2- 172.21.15.45:0/2713967440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d0578b80 0x5636d05754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.051+0000 7fc0fed9d640 1 -- 172.21.15.45:0/2713967440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d0578b80 msgr2=0x5636d05754e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.051+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/2713967440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d0578b80 0x5636d05754e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.051+0000 7fc0fed9d640 1 -- 172.21.15.45:0/2713967440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d05794a0 msgr2=0x5636d058d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.051+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/2713967440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d05794a0 0x5636d058d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.051+0000 7fc0fed9d640 1 -- 172.21.15.45:0/2713967440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636d047c030 con 0x5636d0586b60 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.051+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/2713967440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 0x5636d058dd60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009930 tx=0x7fc0f8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee312b05833e068d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.051+0000 7fc0e7fff640 1 -- 172.21.15.45:0/2713967440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8004300 con 0x5636d0586b60 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.051+0000 7fc0e7fff640 1 -- 172.21.15.45:0/2713967440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f8004460 con 0x5636d0586b60 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.051+0000 7fc0e7fff640 1 -- 172.21.15.45:0/2713967440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8003d30 con 0x5636d0586b60 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.051+0000 7fc107bb8740 1 -- 172.21.15.45:0/2713967440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 msgr2=0x5636d058dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.051+0000 7fc107bb8740 1 --2- 172.21.15.45:0/2713967440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 0x5636d058dd60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009930 tx=0x7fc0f8005a20 comp rx=0 tx=0).stop 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 -- 172.21.15.45:0/2713967440 shutdown_connections 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 --2- 172.21.15.45:0/2713967440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 0x5636d058dd60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 --2- 172.21.15.45:0/2713967440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d05794a0 0x5636d058d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 --2- 172.21.15.45:0/2713967440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d0578b80 0x5636d05754e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 -- 172.21.15.45:0/2713967440 >> 172.21.15.45:0/2713967440 conn(0x5636d0573980 msgr2=0x5636d0574dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 -- 172.21.15.45:0/2713967440 shutdown_connections 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 -- 172.21.15.45:0/2713967440 wait complete. 2024-01-31T05:23:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 -- start start 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 Processor -- start 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d0578b80 0x5636d05e6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d05794a0 0x5636d05e6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 0x5636d05e0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d05e1320 con 0x5636d0586b60 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d05e1490 con 0x5636d05794a0 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d05e1650 con 0x5636d0578b80 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc107bb8740 10 client.0 _ll_register_callbacks cb 0x7fc0f00097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc0fed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 0x5636d05e0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc0fed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 0x5636d05e0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56884/0 (socket says 172.21.15.45:56884) 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc0fed9d640 1 -- 172.21.15.45:0/2946296653 learned_addr learned my addr 172.21.15.45:0/2946296653 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc0fe59c640 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d0578b80 0x5636d05e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.052+0000 7fc0fdd9b640 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d05794a0 0x5636d05e6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc0fed9d640 1 -- 172.21.15.45:0/2946296653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d0578b80 msgr2=0x5636d05e6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d0578b80 0x5636d05e6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc0fed9d640 1 -- 172.21.15.45:0/2946296653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d05794a0 msgr2=0x5636d05e6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d05794a0 0x5636d05e6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc0fed9d640 1 -- 172.21.15.45:0/2946296653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10010e3c0 con 0x5636d0586b60 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc0fed9d640 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 0x5636d05e0d60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8005ab0 tx=0x7fc0f8004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7a816fc9a61c51 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc0e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc0e5ffb640 1 -- 172.21.15.45:0/2946296653 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8003e80 con 0x5636d0586b60 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc0e5ffb640 1 -- 172.21.15.45:0/2946296653 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc0f8005000 con 0x5636d0586b60 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc107bb8740 1 -- 172.21.15.45:0/2946296653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0f0012d30 con 0x5636d0586b60 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc0e5ffb640 1 -- 172.21.15.45:0/2946296653 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0f80051e0 con 0x5636d0586b60 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc107bb8740 10 client.15429 fetch_fsmap learned FSMap version 21 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc107bb8740 1 -- 172.21.15.45:0/2946296653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5636d04a1a50 con 0x5636d0586b60 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.053+0000 7fc0e5ffb640 1 -- 172.21.15.45:0/2946296653 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f0012d30 con 0x5636d0586b60 2024-01-31T05:23:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.054+0000 7fc0e5ffb640 1 -- 172.21.15.45:0/2946296653 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc0f803c040 con 0x5636d0586b60 2024-01-31T05:23:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.054+0000 7fc107bb8740 10 client.15429 fetch_fsmap finished waiting for FSMap version 21 2024-01-31T05:23:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.054+0000 7fc107bb8740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:23:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.054+0000 7fc107bb8740 20 client.15429 populate_metadata read hostname 'smithi045' 2024-01-31T05:23:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.054+0000 7fc107bb8740 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5636d05ee420 0x5636d05f0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.054+0000 7fc107bb8740 4 client.15429 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:23:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.054+0000 7fc107bb8740 1 -- 172.21.15.45:0/2946296653 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5636d01e0600 con 0x5636d05ee420 2024-01-31T05:23:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.054+0000 7fc0fe59c640 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5636d05ee420 0x5636d05f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.054+0000 7fc0e5ffb640 10 client.15429 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.054+0000 7fc0fe59c640 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5636d05ee420 0x5636d05f0840 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6fcc50b1de84ea8 server_cookie=9359dcf3140c99ee in_seq=0 out_seq=0 2024-01-31T05:23:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc0e5ffb640 1 -- 172.21.15.45:0/2946296653 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5636d01e0600 con 0x5636d05ee420 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc0e5ffb640 10 client.15429 handle_command_reply: tid=0 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc107bb8740 1 client.15429 shutdown 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc107bb8740 1 -- 172.21.15.45:0/2946296653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 msgr2=0x5636d05e0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc107bb8740 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 0x5636d05e0d60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8005ab0 tx=0x7fc0f8004e90 comp rx=0 tx=0).stop 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc107bb8740 1 -- 172.21.15.45:0/2946296653 shutdown_connections 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc107bb8740 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5636d05ee420 0x5636d05f0840 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc107bb8740 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d0586b60 0x5636d05e0d60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc107bb8740 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636d05794a0 0x5636d05e6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc107bb8740 1 --2- 172.21.15.45:0/2946296653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5636d0578b80 0x5636d05e6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc0e5ffb640 0 client.15429 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc107bb8740 1 -- 172.21.15.45:0/2946296653 >> 172.21.15.45:0/2946296653 conn(0x5636d0573980 msgr2=0x5636d0572a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc107bb8740 1 -- 172.21.15.45:0/2946296653 shutdown_connections 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc107bb8740 1 -- 172.21.15.45:0/2946296653 wait complete. 2024-01-31T05:23:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.061+0000 7fc107bb8740 20 client.15429 trim_cache size 0 max 16384 2024-01-31T05:23:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:23:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:23:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:23:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:23:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.062+0000 7fc104d1f640 1 -- 172.21.15.45:0/2635295503 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fc0e80a1690 msgr2=0x7fc0e80a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.062+0000 7fc104d1f640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fc0e80a1690 0x7fc0e80a3b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8006010 tx=0x7fc0f80047a0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.062+0000 7fc104d1f640 1 -- 172.21.15.45:0/2635295503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc100105fe0 msgr2=0x7fc1001976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.062+0000 7fc104d1f640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc100105fe0 0x7fc1001976e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c970 tx=0x7fc0f400ce30 comp rx=0 tx=0).stop 2024-01-31T05:23:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.062+0000 7fc104d1f640 1 -- 172.21.15.45:0/2635295503 shutdown_connections 2024-01-31T05:23:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.062+0000 7fc104d1f640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fc0e80a1690 0x7fc0e80a3b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.062+0000 7fc104d1f640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc100105fe0 0x7fc1001976e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.062+0000 7fc104d1f640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc10019ba60 0x7fc10019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.062+0000 7fc104d1f640 1 --2- 172.21.15.45:0/2635295503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc100100f10 0x7fc1001971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.062+0000 7fc104d1f640 1 -- 172.21.15.45:0/2635295503 >> 172.21.15.45:0/2635295503 conn(0x7fc1000fc7b0 msgr2=0x7fc1000fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.062+0000 7fc104d1f640 1 -- 172.21.15.45:0/2635295503 shutdown_connections 2024-01-31T05:23:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:10.062+0000 7fc104d1f640 1 -- 172.21.15.45:0/2635295503 wait complete. 2024-01-31T05:23:10.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:10 smithi186 ceph-mon[35659]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-01-31T05:23:10.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:10 smithi045 ceph-mon[31585]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-01-31T05:23:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:10 smithi156 ceph-mon[35720]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-01-31T05:23:12.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:12 smithi186 ceph-mon[35659]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-01-31T05:23:12.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:12 smithi045 ceph-mon[31585]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-01-31T05:23:12.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:12 smithi156 ceph-mon[35720]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-01-31T05:23:14.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:14 smithi186 ceph-mon[35659]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-01-31T05:23:14.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:14 smithi045 ceph-mon[31585]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-01-31T05:23:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:14 smithi156 ceph-mon[35720]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-01-31T05:23:16.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:16 smithi186 ceph-mon[35659]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-01-31T05:23:16.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:16 smithi045 ceph-mon[31585]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-01-31T05:23:16.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:16 smithi156 ceph-mon[35720]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-01-31T05:23:18.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:18 smithi186 ceph-mon[35659]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-01-31T05:23:18.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:18 smithi045 ceph-mon[31585]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-01-31T05:23:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:18 smithi156 ceph-mon[35720]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-01-31T05:23:20.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:20 smithi186 ceph-mon[35659]: pgmap v22: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-01-31T05:23:20.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:20 smithi045 ceph-mon[31585]: pgmap v22: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-01-31T05:23:20.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:20 smithi156 ceph-mon[35720]: pgmap v22: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-01-31T05:23:22.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:22 smithi186 ceph-mon[35659]: pgmap v23: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-01-31T05:23:22.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:22 smithi045 ceph-mon[31585]: pgmap v23: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-01-31T05:23:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:22 smithi156 ceph-mon[35720]: pgmap v23: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-01-31T05:23:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.151+0000 7fd7adfc9640 1 -- 172.21.15.45:0/1388804963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a81018c0 msgr2=0x7fd7a8105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.151+0000 7fd7adfc9640 1 --2- 172.21.15.45:0/1388804963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a81018c0 0x7fd7a8105900 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009930 tx=0x7fd79c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.151+0000 7fd7adfc9640 1 -- 172.21.15.45:0/1388804963 shutdown_connections 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.151+0000 7fd7adfc9640 1 --2- 172.21.15.45:0/1388804963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a81018c0 0x7fd7a8105900 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.151+0000 7fd7adfc9640 1 --2- 172.21.15.45:0/1388804963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a8105fe0 0x7fd7a810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.151+0000 7fd7adfc9640 1 --2- 172.21.15.45:0/1388804963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a8100f10 0x7fd7a81012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.151+0000 7fd7adfc9640 1 -- 172.21.15.45:0/1388804963 >> 172.21.15.45:0/1388804963 conn(0x7fd7a80fc7b0 msgr2=0x7fd7a80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.151+0000 7fd7adfc9640 1 -- 172.21.15.45:0/1388804963 shutdown_connections 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.151+0000 7fd7adfc9640 1 -- 172.21.15.45:0/1388804963 wait complete. 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7adfc9640 1 Processor -- start 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7adfc9640 1 -- start start 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7adfc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a8100f10 0x7fd7a81971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7adfc9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a8105fe0 0x7fd7a8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7adfc9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a819ba80 0x7fd7a819bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7adfc9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a819c470 con 0x7fd7a8100f10 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7adfc9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a819c5e0 con 0x7fd7a819ba80 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7adfc9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a819c750 con 0x7fd7a8105fe0 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a8100f10 0x7fd7a81971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a8100f10 0x7fd7a81971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52692/0 (socket says 172.21.15.45:52692) 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7a77fe640 1 -- 172.21.15.45:0/48244423 learned_addr learned my addr 172.21.15.45:0/48244423 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7a6ffd640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a8105fe0 0x7fd7a8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7a7fff640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a819ba80 0x7fd7a819bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7a77fe640 1 -- 172.21.15.45:0/48244423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a8105fe0 msgr2=0x7fd7a8197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a8105fe0 0x7fd7a8197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7a77fe640 1 -- 172.21.15.45:0/48244423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a819ba80 msgr2=0x7fd7a819bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.152+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a819ba80 0x7fd7a819bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.153+0000 7fd7a77fe640 1 -- 172.21.15.45:0/48244423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c009560 con 0x7fd7a8100f10 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.153+0000 7fd7a6ffd640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a8105fe0 0x7fd7a8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.153+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a8100f10 0x7fd7a81971c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd79800d950 tx=0x7fd79800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.153+0000 7fd7a4ff9640 1 -- 172.21.15.45:0/48244423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798009c30 con 0x7fd7a8100f10 2024-01-31T05:23:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.153+0000 7fd7a4ff9640 1 -- 172.21.15.45:0/48244423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd798009d90 con 0x7fd7a8100f10 2024-01-31T05:23:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.153+0000 7fd7adfc9640 1 -- 172.21.15.45:0/48244423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a81a3d30 con 0x7fd7a8100f10 2024-01-31T05:23:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.153+0000 7fd7a4ff9640 1 -- 172.21.15.45:0/48244423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79800f740 con 0x7fd7a8100f10 2024-01-31T05:23:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.153+0000 7fd7adfc9640 1 -- 172.21.15.45:0/48244423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a81a41b0 con 0x7fd7a8100f10 2024-01-31T05:23:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.155+0000 7fd7a4ff9640 1 -- 172.21.15.45:0/48244423 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136128+0+0 (secure 0 0 0) 0x7fd798010a20 con 0x7fd7a8100f10 2024-01-31T05:23:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.156+0000 7fd7a4ff9640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd7900a1690 0x7fd7900a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.156+0000 7fd7a6ffd640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd7900a1690 0x7fd7900a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.156+0000 7fd7a4ff9640 1 -- 172.21.15.45:0/48244423 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd7980cce20 con 0x7fd7a8100f10 2024-01-31T05:23:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.156+0000 7fd7b0d83740 1 Processor -- start 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.156+0000 7fd7b0d83740 1 -- start start 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.156+0000 7fd7a6ffd640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd7900a1690 0x7fd7900a3b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009900 tx=0x7fd79c004f30 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7b0d83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 0x55c489a00440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7b0d83740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00980 0x55c489a09210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7b0d83740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a09750 0x55c489a0bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7b0d83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48996f040 con 0x55c489a00060 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7b0d83740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48996c730 con 0x55c489a00980 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7b0d83740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c489968020 con 0x55c489a09750 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 0x55c489a00440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 0x55c489a00440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52698/0 (socket says 172.21.15.45:52698) 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7a77fe640 1 -- 172.21.15.45:0/2329492375 learned_addr learned my addr 172.21.15.45:0/2329492375 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7a6ffd640 1 --2- 172.21.15.45:0/2329492375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00980 0x55c489a09210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7a7fff640 1 --2- 172.21.15.45:0/2329492375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a09750 0x55c489a0bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7a77fe640 1 -- 172.21.15.45:0/2329492375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a09750 msgr2=0x55c489a0bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/2329492375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a09750 0x55c489a0bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7a77fe640 1 -- 172.21.15.45:0/2329492375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00980 msgr2=0x55c489a09210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/2329492375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00980 0x55c489a09210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7a77fe640 1 -- 172.21.15.45:0/2329492375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c48995fe40 con 0x55c489a00060 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/2329492375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 0x55c489a00440 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd798096490 tx=0x7fd79809a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9871cd97cd1c9dda server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd78dffb640 1 -- 172.21.15.45:0/2329492375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7980a1db0 con 0x55c489a00060 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd78dffb640 1 -- 172.21.15.45:0/2329492375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd798096130 con 0x55c489a00060 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.157+0000 7fd78dffb640 1 -- 172.21.15.45:0/2329492375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7980ca720 con 0x55c489a00060 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.158+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2329492375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 msgr2=0x55c489a00440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.158+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/2329492375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 0x55c489a00440 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd798096490 tx=0x7fd79809a120 comp rx=0 tx=0).stop 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.158+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2329492375 shutdown_connections 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.158+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/2329492375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a09750 0x55c489a0bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.158+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/2329492375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00980 0x55c489a09210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.158+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/2329492375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 0x55c489a00440 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.158+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2329492375 >> 172.21.15.45:0/2329492375 conn(0x55c4899fa980 msgr2=0x55c4899fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.158+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2329492375 shutdown_connections 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.158+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2329492375 wait complete. 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7b0d83740 1 -- start start 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7b0d83740 1 Processor -- start 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7b0d83740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00060 0x55c489a06730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7b0d83740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a00980 0x55c489a06c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7b0d83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a09750 0x55c489a071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7b0d83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4899211b0 con 0x55c489a09750 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7b0d83740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c489920e50 con 0x55c489a00060 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7b0d83740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4898e6090 con 0x55c489a00980 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7b0d83740 10 client.0 _ll_register_callbacks cb 0x55c48990ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a09750 0x55c489a071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a09750 0x55c489a071b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52712/0 (socket says 172.21.15.45:52712) 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7a7fff640 1 -- 172.21.15.45:0/587360217 learned_addr learned my addr 172.21.15.45:0/587360217 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7a6ffd640 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a00980 0x55c489a06c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.159+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00060 0x55c489a06730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd7a7fff640 1 -- 172.21.15.45:0/587360217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a00980 msgr2=0x55c489a06c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd7a7fff640 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a00980 0x55c489a06c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd7a7fff640 1 -- 172.21.15.45:0/587360217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00060 msgr2=0x55c489a06730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd7a7fff640 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00060 0x55c489a06730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd7a7fff640 1 -- 172.21.15.45:0/587360217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a81a41b0 con 0x55c489a09750 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00060 0x55c489a06730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd7a7fff640 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a09750 0x55c489a071b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00116e0 tx=0x7fd7a0011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411a86ba649d6924 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd786ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd786ffd640 1 -- 172.21.15.45:0/587360217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a000c940 con 0x55c489a09750 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd786ffd640 1 -- 172.21.15.45:0/587360217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd7a000caa0 con 0x55c489a09750 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd7b0d83740 1 -- 172.21.15.45:0/587360217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c489961010 con 0x55c489a09750 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd786ffd640 1 -- 172.21.15.45:0/587360217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a000cc60 con 0x55c489a09750 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd7b0d83740 10 client.15441 fetch_fsmap learned FSMap version 21 2024-01-31T05:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd7b0d83740 1 -- 172.21.15.45:0/587360217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55c4899666a0 con 0x55c489a09750 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd786ffd640 1 -- 172.21.15.45:0/587360217 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c489961010 con 0x55c489a09750 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.160+0000 7fd786ffd640 1 -- 172.21.15.45:0/587360217 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd7a0012510 con 0x55c489a09750 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.161+0000 7fd7b0d83740 10 client.15441 fetch_fsmap finished waiting for FSMap version 21 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.161+0000 7fd7b0d83740 10 client.15441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.161+0000 7fd7b0d83740 20 client.15441 populate_metadata read hostname 'smithi045' 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.161+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c489a19620 0x55c489a1ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.161+0000 7fd7b0d83740 4 client.15441 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.161+0000 7fd7b0d83740 1 -- 172.21.15.45:0/587360217 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c489960d30 con 0x55c489a19620 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.161+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c489a19620 0x55c489a1ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.161+0000 7fd786ffd640 10 client.15441 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.161+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c489a19620 0x55c489a1ba40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59b0f2aaf9550907 server_cookie=1b2e42200735dac8 in_seq=0 out_seq=0 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd786ffd640 1 -- 172.21.15.45:0/587360217 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c489960d30 con 0x55c489a19620 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd786ffd640 10 client.15441 handle_command_reply: tid=0 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd7b0d83740 1 client.15441 shutdown 2024-01-31T05:23:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd7b0d83740 1 -- 172.21.15.45:0/587360217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a09750 msgr2=0x55c489a071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a09750 0x55c489a071b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00116e0 tx=0x7fd7a0011ae0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd7b0d83740 1 -- 172.21.15.45:0/587360217 shutdown_connections 2024-01-31T05:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c489a19620 0x55c489a1ba40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a09750 0x55c489a071b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a00980 0x55c489a06c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd786ffd640 0 client.15441 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/587360217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00060 0x55c489a06730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd7b0d83740 1 -- 172.21.15.45:0/587360217 >> 172.21.15.45:0/587360217 conn(0x55c4899fa980 msgr2=0x55c4899fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd7b0d83740 1 -- 172.21.15.45:0/587360217 shutdown_connections 2024-01-31T05:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd7b0d83740 1 -- 172.21.15.45:0/587360217 wait complete. 2024-01-31T05:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.162+0000 7fd7b0d83740 20 client.15441 trim_cache size 0 max 16384 2024-01-31T05:23:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.175+0000 7fd7b0d83740 1 Processor -- start 2024-01-31T05:23:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7b0d83740 1 -- start start 2024-01-31T05:23:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7b0d83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 0x55c489a08e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7b0d83740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00980 0x55c489a06fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7b0d83740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a09750 0x55c489a03990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7b0d83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4899fb550 con 0x55c489a00060 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7b0d83740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c489a19420 con 0x55c489a00980 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7b0d83740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c489a20c90 con 0x55c489a09750 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 0x55c489a08e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 0x55c489a08e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52728/0 (socket says 172.21.15.45:52728) 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7a77fe640 1 -- 172.21.15.45:0/3124108558 learned_addr learned my addr 172.21.15.45:0/3124108558 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7a7fff640 1 --2- 172.21.15.45:0/3124108558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a09750 0x55c489a03990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7a6ffd640 1 --2- 172.21.15.45:0/3124108558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00980 0x55c489a06fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7a77fe640 1 -- 172.21.15.45:0/3124108558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a09750 msgr2=0x55c489a03990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/3124108558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a09750 0x55c489a03990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7a77fe640 1 -- 172.21.15.45:0/3124108558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00980 msgr2=0x55c489a06fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/3124108558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00980 0x55c489a06fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7a77fe640 1 -- 172.21.15.45:0/3124108558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c489903030 con 0x55c489a00060 2024-01-31T05:23:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7a7fff640 1 --2- 172.21.15.45:0/3124108558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a09750 0x55c489a03990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/3124108558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 0x55c489a08e60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd79800dfe0 tx=0x7fd798095de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bd3d403f0e2bb2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd78d7fa640 1 -- 172.21.15.45:0/3124108558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7980a1db0 con 0x55c489a00060 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd78d7fa640 1 -- 172.21.15.45:0/3124108558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7980b9070 con 0x55c489a00060 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.176+0000 7fd78d7fa640 1 -- 172.21.15.45:0/3124108558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7980ca720 con 0x55c489a00060 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.177+0000 7fd7b0d83740 1 -- 172.21.15.45:0/3124108558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 msgr2=0x55c489a08e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.177+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/3124108558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 0x55c489a08e60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd79800dfe0 tx=0x7fd798095de0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.177+0000 7fd7b0d83740 1 -- 172.21.15.45:0/3124108558 shutdown_connections 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.177+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/3124108558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a09750 0x55c489a03990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.177+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/3124108558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a00980 0x55c489a06fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.177+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/3124108558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00060 0x55c489a08e60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.177+0000 7fd7b0d83740 1 -- 172.21.15.45:0/3124108558 >> 172.21.15.45:0/3124108558 conn(0x55c4899fa980 msgr2=0x55c489a05000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.177+0000 7fd7b0d83740 1 -- 172.21.15.45:0/3124108558 shutdown_connections 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.177+0000 7fd7b0d83740 1 -- 172.21.15.45:0/3124108558 wait complete. 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.177+0000 7fd7b0d83740 1 -- start start 2024-01-31T05:23:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7b0d83740 1 Processor -- start 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7b0d83740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a00060 0x55c489a100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7b0d83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00980 0x55c489a10620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7b0d83740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a09750 0x55c489a0e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7b0d83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c489a0ec40 con 0x55c489a00980 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7b0d83740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c489a0edb0 con 0x55c489a09750 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7b0d83740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c489a0ef70 con 0x55c489a00060 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7b0d83740 10 client.0 _ll_register_callbacks cb 0x7fd798009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7a6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00980 0x55c489a10620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7a6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00980 0x55c489a10620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52734/0 (socket says 172.21.15.45:52734) 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7a6ffd640 1 -- 172.21.15.45:0/2160838251 learned_addr learned my addr 172.21.15.45:0/2160838251 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7a7fff640 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a09750 0x55c489a0e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a00060 0x55c489a100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7a6ffd640 1 -- 172.21.15.45:0/2160838251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a00060 msgr2=0x55c489a100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7a6ffd640 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a00060 0x55c489a100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7a6ffd640 1 -- 172.21.15.45:0/2160838251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a09750 msgr2=0x55c489a0e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7a6ffd640 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a09750 0x55c489a0e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7a6ffd640 1 -- 172.21.15.45:0/2160838251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c48995fe40 con 0x55c489a00980 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd7a6ffd640 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00980 0x55c489a10620 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd7a806a3a0 tx=0x7fd79c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebcdc55475834de server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.178+0000 7fd786ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd786ffd640 1 -- 172.21.15.45:0/2160838251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0035f0 con 0x55c489a00980 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd786ffd640 1 -- 172.21.15.45:0/2160838251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd79c046040 con 0x55c489a00980 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2160838251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd798096130 con 0x55c489a00980 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd786ffd640 1 -- 172.21.15.45:0/2160838251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd79c003750 con 0x55c489a00980 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd7b0d83740 10 client.15447 fetch_fsmap learned FSMap version 21 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2160838251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55c489928a50 con 0x55c489a00980 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd786ffd640 1 -- 172.21.15.45:0/2160838251 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd798096130 con 0x55c489a00980 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd786ffd640 1 -- 172.21.15.45:0/2160838251 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd79c03c040 con 0x55c489a00980 2024-01-31T05:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd7b0d83740 10 client.15447 fetch_fsmap finished waiting for FSMap version 21 2024-01-31T05:23:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd7b0d83740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:23:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd7b0d83740 20 client.15447 populate_metadata read hostname 'smithi045' 2024-01-31T05:23:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c489a70e90 0x55c489a732b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd7b0d83740 4 client.15447 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:23:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2160838251 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c489667600 con 0x55c489a70e90 2024-01-31T05:23:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.179+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c489a70e90 0x55c489a732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.180+0000 7fd786ffd640 10 client.15447 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.180+0000 7fd7a77fe640 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c489a70e90 0x55c489a732b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af41e41ce0c98858 server_cookie=a86915f674a036b1 in_seq=0 out_seq=0 2024-01-31T05:23:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd786ffd640 1 -- 172.21.15.45:0/2160838251 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c489667600 con 0x55c489a70e90 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd786ffd640 10 client.15447 handle_command_reply: tid=0 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd7b0d83740 1 client.15447 shutdown 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2160838251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00980 msgr2=0x55c489a10620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00980 0x55c489a10620 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd7a806a3a0 tx=0x7fd79c009260 comp rx=0 tx=0).stop 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2160838251 shutdown_connections 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c489a70e90 0x55c489a732b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c489a00980 0x55c489a10620 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd786ffd640 0 client.15447 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c489a09750 0x55c489a0e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd7b0d83740 1 --2- 172.21.15.45:0/2160838251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c489a00060 0x55c489a100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2160838251 >> 172.21.15.45:0/2160838251 conn(0x55c4899fa980 msgr2=0x55c4899f9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2160838251 shutdown_connections 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd7b0d83740 1 -- 172.21.15.45:0/2160838251 wait complete. 2024-01-31T05:23:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.187+0000 7fd7b0d83740 20 client.15447 trim_cache size 0 max 16384 2024-01-31T05:23:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:23:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:23:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:23:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.188+0000 7fd7adfc9640 1 -- 172.21.15.45:0/48244423 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd7900a1690 msgr2=0x7fd7900a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.188+0000 7fd7adfc9640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd7900a1690 0x7fd7900a3b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009900 tx=0x7fd79c004f30 comp rx=0 tx=0).stop 2024-01-31T05:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.188+0000 7fd7adfc9640 1 -- 172.21.15.45:0/48244423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a8100f10 msgr2=0x7fd7a81971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.188+0000 7fd7adfc9640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a8100f10 0x7fd7a81971c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd79800d950 tx=0x7fd79800de10 comp rx=0 tx=0).stop 2024-01-31T05:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.188+0000 7fd7adfc9640 1 -- 172.21.15.45:0/48244423 shutdown_connections 2024-01-31T05:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.188+0000 7fd7adfc9640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd7900a1690 0x7fd7900a3b50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.188+0000 7fd7adfc9640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a819ba80 0x7fd7a819bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.188+0000 7fd7adfc9640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a8105fe0 0x7fd7a8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.188+0000 7fd7adfc9640 1 --2- 172.21.15.45:0/48244423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a8100f10 0x7fd7a81971c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.188+0000 7fd7adfc9640 1 -- 172.21.15.45:0/48244423 >> 172.21.15.45:0/48244423 conn(0x7fd7a80fc7b0 msgr2=0x7fd7a80fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.188+0000 7fd7adfc9640 1 -- 172.21.15.45:0/48244423 shutdown_connections 2024-01-31T05:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:23.188+0000 7fd7adfc9640 1 -- 172.21.15.45:0/48244423 wait complete. 2024-01-31T05:23:24.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:24 smithi186 ceph-mon[35659]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-01-31T05:23:24.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:24 smithi045 ceph-mon[31585]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-01-31T05:23:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:24 smithi156 ceph-mon[35720]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-01-31T05:23:26.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:26 smithi186 ceph-mon[35659]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-01-31T05:23:26.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:26 smithi045 ceph-mon[31585]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-01-31T05:23:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:26 smithi156 ceph-mon[35720]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-01-31T05:23:28.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:28 smithi186 ceph-mon[35659]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-01-31T05:23:28.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:28 smithi045 ceph-mon[31585]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-01-31T05:23:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:28 smithi156 ceph-mon[35720]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-01-31T05:23:30.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:30 smithi186 ceph-mon[35659]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-01-31T05:23:30.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:30 smithi045 ceph-mon[31585]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-01-31T05:23:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:30 smithi156 ceph-mon[35720]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-01-31T05:23:32.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:32 smithi186 ceph-mon[35659]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-31T05:23:32.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:32 smithi045 ceph-mon[31585]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-31T05:23:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:32 smithi156 ceph-mon[35720]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-31T05:23:34.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:34 smithi186 ceph-mon[35659]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-01-31T05:23:34.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:34 smithi045 ceph-mon[31585]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-01-31T05:23:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:34 smithi156 ceph-mon[35720]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-01-31T05:23:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.276+0000 7f27f0271640 1 -- 172.21.15.45:0/2513235418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e80688b0 msgr2=0x7f27e8068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.276+0000 7f27f0271640 1 --2- 172.21.15.45:0/2513235418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e80688b0 0x7f27e8068cf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009900 tx=0x7f27d802bcd0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.276+0000 7f27f0271640 1 -- 172.21.15.45:0/2513235418 shutdown_connections 2024-01-31T05:23:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.276+0000 7f27f0271640 1 --2- 172.21.15.45:0/2513235418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e80688b0 0x7f27e8068cf0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.276+0000 7f27f0271640 1 --2- 172.21.15.45:0/2513235418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e810d4b0 0x7f27e810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.276+0000 7f27f0271640 1 --2- 172.21.15.45:0/2513235418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27e8101c70 0x7f27e8068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.276+0000 7f27f0271640 1 -- 172.21.15.45:0/2513235418 >> 172.21.15.45:0/2513235418 conn(0x7f27e8076a40 msgr2=0x7f27e8076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.276+0000 7f27f0271640 1 -- 172.21.15.45:0/2513235418 shutdown_connections 2024-01-31T05:23:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.276+0000 7f27f0271640 1 -- 172.21.15.45:0/2513235418 wait complete. 2024-01-31T05:23:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.277+0000 7f27f0271640 1 Processor -- start 2024-01-31T05:23:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.277+0000 7f27f0271640 1 -- start start 2024-01-31T05:23:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.277+0000 7f27f0271640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e80688b0 0x7f27e8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.277+0000 7f27f0271640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27e8101c70 0x7f27e81998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.277+0000 7f27f0271640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e810d4b0 0x7f27e819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.277+0000 7f27f0271640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e819e200 con 0x7f27e80688b0 2024-01-31T05:23:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.277+0000 7f27f0271640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e819e370 con 0x7f27e810d4b0 2024-01-31T05:23:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.277+0000 7f27f0271640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e819e530 con 0x7f27e8101c70 2024-01-31T05:23:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27ee00d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e80688b0 0x7f27e8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27ee00d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e80688b0 0x7f27e8199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35526/0 (socket says 172.21.15.45:35526) 2024-01-31T05:23:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27ee00d640 1 -- 172.21.15.45:0/3131461661 learned_addr learned my addr 172.21.15.45:0/3131461661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27ed80c640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27e8101c70 0x7f27e81998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27ee00d640 1 -- 172.21.15.45:0/3131461661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27e8101c70 msgr2=0x7f27e81998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27ee00d640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27e8101c70 0x7f27e81998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27ee00d640 1 -- 172.21.15.45:0/3131461661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e810d4b0 msgr2=0x7f27e819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27ee80e640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e810d4b0 0x7f27e819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27ee00d640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e810d4b0 0x7f27e819dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27ee00d640 1 -- 172.21.15.45:0/3131461661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d8009560 con 0x7f27e80688b0 2024-01-31T05:23:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27ee00d640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e80688b0 0x7f27e8199390 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f27e400d950 tx=0x7f27e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27df7fe640 1 -- 172.21.15.45:0/3131461661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4009a30 con 0x7f27e80688b0 2024-01-31T05:23:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27f0271640 1 -- 172.21.15.45:0/3131461661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e819e810 con 0x7f27e80688b0 2024-01-31T05:23:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27df7fe640 1 -- 172.21.15.45:0/3131461661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27e4009b90 con 0x7f27e80688b0 2024-01-31T05:23:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27df7fe640 1 -- 172.21.15.45:0/3131461661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e40047a0 con 0x7f27e80688b0 2024-01-31T05:23:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.278+0000 7f27f0271640 1 -- 172.21.15.45:0/3131461661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e81a6090 con 0x7f27e80688b0 2024-01-31T05:23:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.281+0000 7f27df7fe640 1 -- 172.21.15.45:0/3131461661 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136128+0+0 (secure 0 0 0) 0x7f27e4009d00 con 0x7f27e80688b0 2024-01-31T05:23:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.281+0000 7f27df7fe640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f27d40a1610 0x7f27d40a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.281+0000 7f27df7fe640 1 -- 172.21.15.45:0/3131461661 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f27e40cc760 con 0x7f27e80688b0 2024-01-31T05:23:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.282+0000 7f27ed80c640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f27d40a1610 0x7f27d40a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.282+0000 7f27f3032740 1 Processor -- start 2024-01-31T05:23:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.282+0000 7f27f3032740 1 -- start start 2024-01-31T05:23:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.282+0000 7f27ed80c640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f27d40a1610 0x7f27d40a3ad0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27e8077260 tx=0x7f27d8004eb0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27f3032740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d060 0x56438307d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27f3032740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d980 0x564383086210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27f3032740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564383086750 0x564383088b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27f3032740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564382fec040 con 0x56438307d060 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27f3032740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564382fe9730 con 0x564383086750 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27f3032740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564382fe5020 con 0x56438307d980 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27ee00d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d060 0x56438307d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27ee00d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d060 0x56438307d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35536/0 (socket says 172.21.15.45:35536) 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27ee00d640 1 -- 172.21.15.45:0/150031660 learned_addr learned my addr 172.21.15.45:0/150031660 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27ee80e640 1 --2- 172.21.15.45:0/150031660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564383086750 0x564383088b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27ed80c640 1 --2- 172.21.15.45:0/150031660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d980 0x564383086210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27ee00d640 1 -- 172.21.15.45:0/150031660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d980 msgr2=0x564383086210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27ee00d640 1 --2- 172.21.15.45:0/150031660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d980 0x564383086210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27ee00d640 1 -- 172.21.15.45:0/150031660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564383086750 msgr2=0x564383088b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27ee00d640 1 --2- 172.21.15.45:0/150031660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564383086750 0x564383088b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27ee00d640 1 -- 172.21.15.45:0/150031660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564382fdce40 con 0x56438307d060 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.283+0000 7f27ee00d640 1 --2- 172.21.15.45:0/150031660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d060 0x56438307d440 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f27e4095dd0 tx=0x7f27e4098d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440f1e5168e382d2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.284+0000 7f27dcbfe640 1 -- 172.21.15.45:0/150031660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e409d4b0 con 0x56438307d060 2024-01-31T05:23:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.284+0000 7f27dcbfe640 1 -- 172.21.15.45:0/150031660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27e40a16f0 con 0x56438307d060 2024-01-31T05:23:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.284+0000 7f27dcbfe640 1 -- 172.21.15.45:0/150031660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e40a1260 con 0x56438307d060 2024-01-31T05:23:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.284+0000 7f27f3032740 1 -- 172.21.15.45:0/150031660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d060 msgr2=0x56438307d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.284+0000 7f27f3032740 1 --2- 172.21.15.45:0/150031660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d060 0x56438307d440 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f27e4095dd0 tx=0x7f27e4098d80 comp rx=0 tx=0).stop 2024-01-31T05:23:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.284+0000 7f27f3032740 1 -- 172.21.15.45:0/150031660 shutdown_connections 2024-01-31T05:23:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.284+0000 7f27f3032740 1 --2- 172.21.15.45:0/150031660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564383086750 0x564383088b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.284+0000 7f27f3032740 1 --2- 172.21.15.45:0/150031660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d980 0x564383086210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.284+0000 7f27f3032740 1 --2- 172.21.15.45:0/150031660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d060 0x56438307d440 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.284+0000 7f27f3032740 1 -- 172.21.15.45:0/150031660 >> 172.21.15.45:0/150031660 conn(0x564383077980 msgr2=0x564383077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27f3032740 1 -- 172.21.15.45:0/150031660 shutdown_connections 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27f3032740 1 -- 172.21.15.45:0/150031660 wait complete. 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27f3032740 1 -- start start 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27f3032740 1 Processor -- start 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27f3032740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d060 0x564383083730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27f3032740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d980 0x564383083c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27f3032740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 0x5643830841b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27f3032740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564382f9e1b0 con 0x564383086750 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27ee80e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 0x5643830841b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27f3032740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564382f9de50 con 0x56438307d980 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27f3032740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564382f63090 con 0x56438307d060 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27ee80e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 0x5643830841b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35542/0 (socket says 172.21.15.45:35542) 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27ee80e640 1 -- 172.21.15.45:0/558611503 learned_addr learned my addr 172.21.15.45:0/558611503 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.285+0000 7f27f3032740 10 client.0 _ll_register_callbacks cb 0x564382f88a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27ee00d640 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d060 0x564383083730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27ed80c640 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d980 0x564383083c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27ee80e640 1 -- 172.21.15.45:0/558611503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d060 msgr2=0x564383083730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27ee80e640 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d060 0x564383083730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27ee80e640 1 -- 172.21.15.45:0/558611503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d980 msgr2=0x564383083c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27ee80e640 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d980 0x564383083c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27ee80e640 1 -- 172.21.15.45:0/558611503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e81a6090 con 0x564383086750 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27ee00d640 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d060 0x564383083730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27ee80e640 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 0x5643830841b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f27e00124b0 tx=0x7f27e0012970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19610afb7eac3ae2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27d1ffb640 1 -- 172.21.15.45:0/558611503 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0003d80 con 0x564383086750 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27d1ffb640 1 -- 172.21.15.45:0/558611503 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f27e0004050 con 0x564383086750 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27f3032740 1 -- 172.21.15.45:0/558611503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564382fde010 con 0x564383086750 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.286+0000 7f27d1ffb640 1 -- 172.21.15.45:0/558611503 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27e0004210 con 0x564383086750 2024-01-31T05:23:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.287+0000 7f27f3032740 10 client.15459 fetch_fsmap learned FSMap version 21 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.287+0000 7f27f3032740 1 -- 172.21.15.45:0/558611503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x564382fe36a0 con 0x564383086750 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.287+0000 7f27d1ffb640 1 -- 172.21.15.45:0/558611503 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27e0004830 con 0x564383086750 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.287+0000 7f27d1ffb640 1 -- 172.21.15.45:0/558611503 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f27e001b440 con 0x564383086750 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.287+0000 7f27f3032740 10 client.15459 fetch_fsmap finished waiting for FSMap version 21 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.287+0000 7f27f3032740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.287+0000 7f27f3032740 20 client.15459 populate_metadata read hostname 'smithi045' 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.287+0000 7f27f3032740 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5643830965b0 0x564383098a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.287+0000 7f27f3032740 4 client.15459 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.287+0000 7f27f3032740 1 -- 172.21.15.45:0/558611503 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564382fde010 con 0x5643830965b0 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.287+0000 7f27ee00d640 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5643830965b0 0x564383098a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27d1ffb640 10 client.15459 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27ee00d640 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5643830965b0 0x564383098a30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f2789c13d013f4 server_cookie=fcbef806c8231c6a in_seq=0 out_seq=0 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27d1ffb640 1 -- 172.21.15.45:0/558611503 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564382fde010 con 0x5643830965b0 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27d1ffb640 10 client.15459 handle_command_reply: tid=0 2024-01-31T05:23:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27f3032740 1 client.15459 shutdown 2024-01-31T05:23:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27f3032740 1 -- 172.21.15.45:0/558611503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 msgr2=0x5643830841b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27f3032740 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 0x5643830841b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f27e00124b0 tx=0x7f27e0012970 comp rx=0 tx=0).stop 2024-01-31T05:23:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27f3032740 1 -- 172.21.15.45:0/558611503 shutdown_connections 2024-01-31T05:23:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27f3032740 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5643830965b0 0x564383098a30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27f3032740 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 0x5643830841b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27f3032740 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d980 0x564383083c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27f3032740 1 --2- 172.21.15.45:0/558611503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d060 0x564383083730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27f3032740 1 -- 172.21.15.45:0/558611503 >> 172.21.15.45:0/558611503 conn(0x564383077980 msgr2=0x5643830796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.288+0000 7f27d1ffb640 0 client.15459 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.289+0000 7f27f3032740 1 -- 172.21.15.45:0/558611503 shutdown_connections 2024-01-31T05:23:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.289+0000 7f27f3032740 1 -- 172.21.15.45:0/558611503 wait complete. 2024-01-31T05:23:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.289+0000 7f27f3032740 20 client.15459 trim_cache size 0 max 16384 2024-01-31T05:23:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27f3032740 1 Processor -- start 2024-01-31T05:23:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27f3032740 1 -- start start 2024-01-31T05:23:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27f3032740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d060 0x564383085e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27f3032740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d980 0x564383083fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27f3032740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564383086750 0x564383082000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27f3032740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564382fde010 con 0x56438307d980 2024-01-31T05:23:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27f3032740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564383071e90 con 0x56438307d060 2024-01-31T05:23:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27f3032740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438307c3f0 con 0x564383086750 2024-01-31T05:23:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27ed80c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d980 0x564383083fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27ed80c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d980 0x564383083fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35554/0 (socket says 172.21.15.45:35554) 2024-01-31T05:23:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27ed80c640 1 -- 172.21.15.45:0/1896867118 learned_addr learned my addr 172.21.15.45:0/1896867118 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27ee00d640 1 --2- 172.21.15.45:0/1896867118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d060 0x564383085e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.302+0000 7f27ee80e640 1 --2- 172.21.15.45:0/1896867118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564383086750 0x564383082000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.303+0000 7f27ed80c640 1 -- 172.21.15.45:0/1896867118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564383086750 msgr2=0x564383082000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.303+0000 7f27ed80c640 1 --2- 172.21.15.45:0/1896867118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564383086750 0x564383082000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.303+0000 7f27ed80c640 1 -- 172.21.15.45:0/1896867118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d060 msgr2=0x564383085e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.303+0000 7f27ed80c640 1 --2- 172.21.15.45:0/1896867118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d060 0x564383085e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.303+0000 7f27ed80c640 1 -- 172.21.15.45:0/1896867118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564382f80030 con 0x56438307d980 2024-01-31T05:23:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.303+0000 7f27ee80e640 1 --2- 172.21.15.45:0/1896867118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564383086750 0x564383082000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:23:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.303+0000 7f27ee00d640 1 --2- 172.21.15.45:0/1896867118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d060 0x564383085e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:23:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.303+0000 7f27ed80c640 1 --2- 172.21.15.45:0/1896867118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d980 0x564383083fd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f27d8035000 tx=0x7f27d803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ddafde511404c5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.303+0000 7f27d3fff640 1 -- 172.21.15.45:0/1896867118 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d80049c0 con 0x56438307d980 2024-01-31T05:23:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.303+0000 7f27d3fff640 1 -- 172.21.15.45:0/1896867118 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27d802cce0 con 0x56438307d980 2024-01-31T05:23:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.303+0000 7f27d3fff640 1 -- 172.21.15.45:0/1896867118 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8003850 con 0x56438307d980 2024-01-31T05:23:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.304+0000 7f27f3032740 1 -- 172.21.15.45:0/1896867118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d980 msgr2=0x564383083fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.304+0000 7f27f3032740 1 --2- 172.21.15.45:0/1896867118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d980 0x564383083fd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f27d8035000 tx=0x7f27d803e000 comp rx=0 tx=0).stop 2024-01-31T05:23:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.304+0000 7f27f3032740 1 -- 172.21.15.45:0/1896867118 shutdown_connections 2024-01-31T05:23:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.304+0000 7f27f3032740 1 --2- 172.21.15.45:0/1896867118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56438307d980 0x564383083fd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.304+0000 7f27f3032740 1 --2- 172.21.15.45:0/1896867118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564383086750 0x564383082000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.304+0000 7f27f3032740 1 --2- 172.21.15.45:0/1896867118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d060 0x564383085e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.304+0000 7f27f3032740 1 -- 172.21.15.45:0/1896867118 >> 172.21.15.45:0/1896867118 conn(0x564383077980 msgr2=0x56438307ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.304+0000 7f27f3032740 1 -- 172.21.15.45:0/1896867118 shutdown_connections 2024-01-31T05:23:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.304+0000 7f27f3032740 1 -- 172.21.15.45:0/1896867118 wait complete. 2024-01-31T05:23:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.304+0000 7f27f3032740 1 -- start start 2024-01-31T05:23:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27f3032740 1 Processor -- start 2024-01-31T05:23:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27f3032740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d060 0x56438308d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27f3032740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d980 0x56438308d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27f3032740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 0x56438308bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27f3032740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438308c220 con 0x564383086750 2024-01-31T05:23:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27f3032740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438308c390 con 0x56438307d980 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27f3032740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438308c550 con 0x56438307d060 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27f3032740 10 client.0 _ll_register_callbacks cb 0x7f27e40051b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27ee80e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 0x56438308bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27ee80e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 0x56438308bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35564/0 (socket says 172.21.15.45:35564) 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27ee80e640 1 -- 172.21.15.45:0/695617596 learned_addr learned my addr 172.21.15.45:0/695617596 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27ed80c640 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d980 0x56438308d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27ee00d640 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d060 0x56438308d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27ee80e640 1 -- 172.21.15.45:0/695617596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d060 msgr2=0x56438308d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27ee80e640 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d060 0x56438308d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.305+0000 7f27ee80e640 1 -- 172.21.15.45:0/695617596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d980 msgr2=0x56438308d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.306+0000 7f27ee80e640 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d980 0x56438308d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.306+0000 7f27ee80e640 1 -- 172.21.15.45:0/695617596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e81105a0 con 0x564383086750 2024-01-31T05:23:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.306+0000 7f27ee00d640 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d060 0x56438308d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:23:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.306+0000 7f27ee80e640 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 0x56438308bb50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f27e00124b0 tx=0x7f27e001b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea1c33001a659b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.306+0000 7f27d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:23:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.306+0000 7f27d1ffb640 1 -- 172.21.15.45:0/695617596 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0012220 con 0x564383086750 2024-01-31T05:23:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.306+0000 7f27d1ffb640 1 -- 172.21.15.45:0/695617596 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f27e0003b80 con 0x564383086750 2024-01-31T05:23:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.306+0000 7f27f3032740 1 -- 172.21.15.45:0/695617596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27e40a16f0 con 0x564383086750 2024-01-31T05:23:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.306+0000 7f27d1ffb640 1 -- 172.21.15.45:0/695617596 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27e0003d40 con 0x564383086750 2024-01-31T05:23:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.306+0000 7f27f3032740 10 client.15465 fetch_fsmap learned FSMap version 21 2024-01-31T05:23:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.306+0000 7f27f3032740 1 -- 172.21.15.45:0/695617596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x564382fa5a50 con 0x564383086750 2024-01-31T05:23:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.306+0000 7f27d1ffb640 1 -- 172.21.15.45:0/695617596 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27e0039080 con 0x564383086750 2024-01-31T05:23:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.307+0000 7f27d1ffb640 1 -- 172.21.15.45:0/695617596 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f27e0010570 con 0x564383086750 2024-01-31T05:23:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.307+0000 7f27f3032740 10 client.15465 fetch_fsmap finished waiting for FSMap version 21 2024-01-31T05:23:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.307+0000 7f27f3032740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:23:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.307+0000 7f27f3032740 20 client.15465 populate_metadata read hostname 'smithi045' 2024-01-31T05:23:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.307+0000 7f27f3032740 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5643830edec0 0x5643830f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.307+0000 7f27f3032740 4 client.15465 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:23:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.307+0000 7f27f3032740 1 -- 172.21.15.45:0/695617596 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564382fddd30 con 0x5643830edec0 2024-01-31T05:23:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.307+0000 7f27ee00d640 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5643830edec0 0x5643830f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.308+0000 7f27d1ffb640 10 client.15465 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.308+0000 7f27ee00d640 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5643830edec0 0x5643830f02e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a47b527ea6df49f9 server_cookie=6f57051236c167a7 in_seq=0 out_seq=0 2024-01-31T05:23:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27d1ffb640 1 -- 172.21.15.45:0/695617596 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564382fddd30 con 0x5643830edec0 2024-01-31T05:23:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27d1ffb640 10 client.15465 handle_command_reply: tid=0 2024-01-31T05:23:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27f3032740 1 client.15465 shutdown 2024-01-31T05:23:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27f3032740 1 -- 172.21.15.45:0/695617596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 msgr2=0x56438308bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27f3032740 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 0x56438308bb50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f27e00124b0 tx=0x7f27e001b6f0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27f3032740 1 -- 172.21.15.45:0/695617596 shutdown_connections 2024-01-31T05:23:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27f3032740 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5643830edec0 0x5643830f02e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27f3032740 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564383086750 0x56438308bb50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27f3032740 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56438307d980 0x56438308d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27f3032740 1 --2- 172.21.15.45:0/695617596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56438307d060 0x56438308d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27f3032740 1 -- 172.21.15.45:0/695617596 >> 172.21.15.45:0/695617596 conn(0x564383077980 msgr2=0x564383081f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27d1ffb640 0 client.15465 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27f3032740 1 -- 172.21.15.45:0/695617596 shutdown_connections 2024-01-31T05:23:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27f3032740 1 -- 172.21.15.45:0/695617596 wait complete. 2024-01-31T05:23:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.312+0000 7f27f3032740 20 client.15465 trim_cache size 0 max 16384 2024-01-31T05:23:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:23:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:23:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:23:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:23:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.313+0000 7f27f0271640 1 -- 172.21.15.45:0/3131461661 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f27d40a1610 msgr2=0x7f27d40a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.313+0000 7f27f0271640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f27d40a1610 0x7f27d40a3ad0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27e8077260 tx=0x7f27d8004eb0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.313+0000 7f27f0271640 1 -- 172.21.15.45:0/3131461661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e80688b0 msgr2=0x7f27e8199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.313+0000 7f27f0271640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e80688b0 0x7f27e8199390 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f27e400d950 tx=0x7f27e400de10 comp rx=0 tx=0).stop 2024-01-31T05:23:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.313+0000 7f27f0271640 1 -- 172.21.15.45:0/3131461661 shutdown_connections 2024-01-31T05:23:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.313+0000 7f27f0271640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f27d40a1610 0x7f27d40a3ad0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.313+0000 7f27f0271640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e810d4b0 0x7f27e819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.313+0000 7f27f0271640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27e8101c70 0x7f27e81998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.313+0000 7f27f0271640 1 --2- 172.21.15.45:0/3131461661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e80688b0 0x7f27e8199390 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.313+0000 7f27f0271640 1 -- 172.21.15.45:0/3131461661 >> 172.21.15.45:0/3131461661 conn(0x7f27e8076a40 msgr2=0x7f27e80fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.313+0000 7f27f0271640 1 -- 172.21.15.45:0/3131461661 shutdown_connections 2024-01-31T05:23:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:36.313+0000 7f27f0271640 1 -- 172.21.15.45:0/3131461661 wait complete. 2024-01-31T05:23:36.843 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:36 smithi186 ceph-mon[35659]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-31T05:23:36.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:36 smithi045 ceph-mon[31585]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-31T05:23:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:36 smithi156 ceph-mon[35720]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-31T05:23:38.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:38 smithi186 ceph-mon[35659]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-01-31T05:23:38.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:38 smithi045 ceph-mon[31585]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-01-31T05:23:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:38 smithi156 ceph-mon[35720]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-01-31T05:23:39.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:39 smithi186 ceph-mon[35659]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:23:39.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:39 smithi045 ceph-mon[31585]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:23:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:39 smithi156 ceph-mon[35720]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:23:42.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:42 smithi186 ceph-mon[35659]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:23:42.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:42 smithi045 ceph-mon[31585]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:23:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:42 smithi156 ceph-mon[35720]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:23:44.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:44 smithi186 ceph-mon[35659]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-31T05:23:44.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:44 smithi045 ceph-mon[31585]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-31T05:23:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:44 smithi156 ceph-mon[35720]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-31T05:23:46.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:46 smithi186 ceph-mon[35659]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:23:46.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:23:46.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:46 smithi045 ceph-mon[31585]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:23:46.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:23:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:46 smithi156 ceph-mon[35720]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:23:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:23:48.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:47 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:23:48.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:47 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:23:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:47 smithi186 ceph-mon[35659]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-31T05:23:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:47 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:23:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:47 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:23:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:47 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:23:48.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:47 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:23:48.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:47 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:23:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:47 smithi045 ceph-mon[31585]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-31T05:23:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:47 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:23:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:47 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:23:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:47 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:23:48.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:47 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:23:48.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:47 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:23:48.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:47 smithi156 ceph-mon[35720]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-31T05:23:48.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:47 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:23:48.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:47 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:23:48.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:47 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:23:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.445+0000 7f1fc0a67640 1 -- 172.21.15.45:0/3997114097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fbc0ff460 msgr2=0x7f1fbc0ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.445+0000 7f1fc0a67640 1 --2- 172.21.15.45:0/3997114097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fbc0ff460 0x7f1fbc0ff840 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009930 tx=0x7f1fac02bc30 comp rx=0 tx=0).stop 2024-01-31T05:23:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.445+0000 7f1fc0a67640 1 -- 172.21.15.45:0/3997114097 shutdown_connections 2024-01-31T05:23:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.445+0000 7f1fc0a67640 1 --2- 172.21.15.45:0/3997114097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fbc106020 0x7f1fbc10dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.445+0000 7f1fc0a67640 1 --2- 172.21.15.45:0/3997114097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fbc0ffe10 0x7f1fbc105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.445+0000 7f1fc0a67640 1 --2- 172.21.15.45:0/3997114097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fbc0ff460 0x7f1fbc0ff840 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.445+0000 7f1fc0a67640 1 -- 172.21.15.45:0/3997114097 >> 172.21.15.45:0/3997114097 conn(0x7f1fbc076140 msgr2=0x7f1fbc0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.445+0000 7f1fc0a67640 1 -- 172.21.15.45:0/3997114097 shutdown_connections 2024-01-31T05:23:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.445+0000 7f1fc0a67640 1 -- 172.21.15.45:0/3997114097 wait complete. 2024-01-31T05:23:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.446+0000 7f1fc0a67640 1 Processor -- start 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.446+0000 7f1fc0a67640 1 -- start start 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fc0a67640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fbc0ff460 0x7f1fbc197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fc0a67640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fbc106020 0x7f1fbc1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fc0a67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fbc19baf0 0x7f1fbc19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fc0a67640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc19c4e0 con 0x7f1fbc19baf0 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fc0a67640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc19c650 con 0x7f1fbc0ff460 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fc0a67640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc19c7c0 con 0x7f1fbc106020 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fbad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fbc19baf0 0x7f1fbc19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fbad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fbc19baf0 0x7f1fbc19bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37430/0 (socket says 172.21.15.45:37430) 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fbad9d640 1 -- 172.21.15.45:0/1674853558 learned_addr learned my addr 172.21.15.45:0/1674853558 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fbc106020 0x7f1fbc1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fba59c640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fbc0ff460 0x7f1fbc197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fbad9d640 1 -- 172.21.15.45:0/1674853558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fbc106020 msgr2=0x7f1fbc1977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fbad9d640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fbc106020 0x7f1fbc1977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fbad9d640 1 -- 172.21.15.45:0/1674853558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fbc0ff460 msgr2=0x7f1fbc197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fbad9d640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fbc0ff460 0x7f1fbc197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.447+0000 7f1fbad9d640 1 -- 172.21.15.45:0/1674853558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac009560 con 0x7f1fbc19baf0 2024-01-31T05:23:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.448+0000 7f1fbad9d640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fbc19baf0 0x7f1fbc19bfa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400eb10 tx=0x7f1fb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.448+0000 7f1fab7fe640 1 -- 172.21.15.45:0/1674853558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4004b20 con 0x7f1fbc19baf0 2024-01-31T05:23:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.448+0000 7f1fab7fe640 1 -- 172.21.15.45:0/1674853558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb4004c80 con 0x7f1fbc19baf0 2024-01-31T05:23:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.448+0000 7f1fc0a67640 1 -- 172.21.15.45:0/1674853558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fbc1a3da0 con 0x7f1fbc19baf0 2024-01-31T05:23:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.448+0000 7f1fab7fe640 1 -- 172.21.15.45:0/1674853558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4010690 con 0x7f1fbc19baf0 2024-01-31T05:23:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.448+0000 7f1fc0a67640 1 -- 172.21.15.45:0/1674853558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fbc1a42a0 con 0x7f1fbc19baf0 2024-01-31T05:23:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.451+0000 7f1fc382b740 1 Processor -- start 2024-01-31T05:23:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.451+0000 7f1fab7fe640 1 -- 172.21.15.45:0/1674853558 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136128+0+0 (secure 0 0 0) 0x7f1fb4005020 con 0x7f1fbc19baf0 2024-01-31T05:23:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fc382b740 1 -- start start 2024-01-31T05:23:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fc382b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 0x557f94a78e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fc382b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fc382b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a8adf0 0x557f94a8d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fc382b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f949e8040 con 0x557f94a79460 2024-01-31T05:23:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fc382b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f949e5730 con 0x557f94a8adf0 2024-01-31T05:23:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fc382b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f949e1020 con 0x557f94a78ab0 2024-01-31T05:23:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fab7fe640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1fa40a1690 0x7f1fa40a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fb9d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 0x557f94a78e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fb9d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 0x557f94a78e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:60972/0 (socket says 172.21.15.45:60972) 2024-01-31T05:23:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fb9d9b640 1 -- 172.21.15.45:0/3299982038 learned_addr learned my addr 172.21.15.45:0/3299982038 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/3299982038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a8adf0 0x557f94a8d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fab7fe640 1 -- 172.21.15.45:0/1674853558 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1fb4014070 con 0x7f1fbc19baf0 2024-01-31T05:23:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.452+0000 7f1fbad9d640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1fa40a1690 0x7f1fa40a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.453+0000 7f1fb9d9b640 1 -- 172.21.15.45:0/3299982038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a8adf0 msgr2=0x557f94a8d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.453+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/3299982038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a8adf0 0x557f94a8d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.453+0000 7f1fb9d9b640 1 -- 172.21.15.45:0/3299982038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 msgr2=0x557f94a798e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.453+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/3299982038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a798e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.453+0000 7f1fb9d9b640 1 -- 172.21.15.45:0/3299982038 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f949d8e40 con 0x557f94a78ab0 2024-01-31T05:23:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.453+0000 7f1fbad9d640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1fa40a1690 0x7f1fa40a3b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40cac10 tx=0x7f1fb4009a70 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.453+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/3299982038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 0x557f94a78e90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00058c0 tx=0x7f1fb0005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42864ed275298989 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.453+0000 7f1fa8ff9640 1 -- 172.21.15.45:0/3299982038 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0004030 con 0x557f94a78ab0 2024-01-31T05:23:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.453+0000 7f1fa8ff9640 1 -- 172.21.15.45:0/3299982038 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb0005020 con 0x557f94a78ab0 2024-01-31T05:23:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.453+0000 7f1fa8ff9640 1 -- 172.21.15.45:0/3299982038 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0004b20 con 0x557f94a78ab0 2024-01-31T05:23:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.454+0000 7f1fc382b740 1 -- 172.21.15.45:0/3299982038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 msgr2=0x557f94a78e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.454+0000 7f1fc382b740 1 --2- 172.21.15.45:0/3299982038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 0x557f94a78e90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00058c0 tx=0x7f1fb0005850 comp rx=0 tx=0).stop 2024-01-31T05:23:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.454+0000 7f1fc382b740 1 -- 172.21.15.45:0/3299982038 shutdown_connections 2024-01-31T05:23:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.454+0000 7f1fc382b740 1 --2- 172.21.15.45:0/3299982038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.454+0000 7f1fc382b740 1 --2- 172.21.15.45:0/3299982038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a8adf0 0x557f94a8d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.454+0000 7f1fc382b740 1 --2- 172.21.15.45:0/3299982038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 0x557f94a78e90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.454+0000 7f1fc382b740 1 -- 172.21.15.45:0/3299982038 >> 172.21.15.45:0/3299982038 conn(0x557f94a73980 msgr2=0x557f94a73d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.455+0000 7f1fc382b740 1 -- 172.21.15.45:0/3299982038 shutdown_connections 2024-01-31T05:23:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.455+0000 7f1fc382b740 1 -- 172.21.15.45:0/3299982038 wait complete. 2024-01-31T05:23:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.455+0000 7f1fc382b740 1 -- start start 2024-01-31T05:23:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.455+0000 7f1fc382b740 1 Processor -- start 2024-01-31T05:23:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.455+0000 7f1fc382b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 0x557f94a84640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fc382b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a84b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fc382b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a8adf0 0x557f94a8a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fc382b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f9499a1b0 con 0x557f94a79460 2024-01-31T05:23:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fc382b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f94999e50 con 0x557f94a8adf0 2024-01-31T05:23:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fc382b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f9495f090 con 0x557f94a78ab0 2024-01-31T05:23:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a84b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fb9d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 0x557f94a84640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fb9d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 0x557f94a84640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:60980/0 (socket says 172.21.15.45:60980) 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fb9d9b640 1 -- 172.21.15.45:0/3183296354 learned_addr learned my addr 172.21.15.45:0/3183296354 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fc382b740 10 client.0 _ll_register_callbacks cb 0x557f94984a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/3183296354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a8adf0 0x557f94a8a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fba59c640 1 -- 172.21.15.45:0/3183296354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 msgr2=0x557f94a84640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fba59c640 1 --2- 172.21.15.45:0/3183296354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 0x557f94a84640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fba59c640 1 -- 172.21.15.45:0/3183296354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a8adf0 msgr2=0x557f94a8a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fba59c640 1 --2- 172.21.15.45:0/3183296354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a8adf0 0x557f94a8a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fba59c640 1 -- 172.21.15.45:0/3183296354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb0019040 con 0x557f94a79460 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fba59c640 1 --2- 172.21.15.45:0/3183296354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a84b80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009230 tx=0x7f1fac02c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4853f8a56083e121 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.456+0000 7f1fa1ffb640 1 -- 172.21.15.45:0/3183296354 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac004dd0 con 0x557f94a79460 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.457+0000 7f1fc382b740 1 -- 172.21.15.45:0/3183296354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f949da010 con 0x557f94a79460 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.457+0000 7f1fa1ffb640 1 -- 172.21.15.45:0/3183296354 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1fac046070 con 0x557f94a79460 2024-01-31T05:23:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.457+0000 7f1fa1ffb640 1 -- 172.21.15.45:0/3183296354 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fac004040 con 0x557f94a79460 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.457+0000 7f1fc382b740 10 client.15477 fetch_fsmap learned FSMap version 21 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.457+0000 7f1fc382b740 1 -- 172.21.15.45:0/3183296354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x557f949df6a0 con 0x557f94a79460 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.457+0000 7f1fa1ffb640 1 -- 172.21.15.45:0/3183296354 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f949da010 con 0x557f94a79460 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.457+0000 7f1fa1ffb640 1 -- 172.21.15.45:0/3183296354 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1fac052050 con 0x557f94a79460 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.457+0000 7f1fc382b740 10 client.15477 fetch_fsmap finished waiting for FSMap version 21 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.457+0000 7f1fc382b740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.457+0000 7f1fc382b740 20 client.15477 populate_metadata read hostname 'smithi045' 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.457+0000 7f1fc382b740 1 --2- 172.21.15.45:0/3183296354 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f94aa3480 0x557f94aa58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.458+0000 7f1fc382b740 4 client.15477 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.458+0000 7f1fc382b740 1 -- 172.21.15.45:0/3183296354 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f949d9d30 con 0x557f94aa3480 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.458+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/3183296354 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f94aa3480 0x557f94aa58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.458+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/3183296354 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f94aa3480 0x557f94aa58a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2d4f0c408c498df server_cookie=bdf59ced984ff6d7 in_seq=0 out_seq=0 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.458+0000 7f1fa1ffb640 10 client.15477 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.459+0000 7f1fa1ffb640 1 -- 172.21.15.45:0/3183296354 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f949d9d30 con 0x557f94aa3480 2024-01-31T05:23:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.459+0000 7f1fa1ffb640 10 client.15477 handle_command_reply: tid=0 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.459+0000 7f1fc382b740 1 client.15477 shutdown 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.459+0000 7f1fc382b740 1 -- 172.21.15.45:0/3183296354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 msgr2=0x557f94a84b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.459+0000 7f1fc382b740 1 --2- 172.21.15.45:0/3183296354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a84b80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009230 tx=0x7f1fac02c9b0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.459+0000 7f1fc382b740 1 -- 172.21.15.45:0/3183296354 shutdown_connections 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.459+0000 7f1fc382b740 1 --2- 172.21.15.45:0/3183296354 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f94aa3480 0x557f94aa58a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.459+0000 7f1fc382b740 1 --2- 172.21.15.45:0/3183296354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a84b80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.459+0000 7f1fa1ffb640 0 client.15477 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.460+0000 7f1fc382b740 1 --2- 172.21.15.45:0/3183296354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a8adf0 0x557f94a8a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.460+0000 7f1fc382b740 1 --2- 172.21.15.45:0/3183296354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a78ab0 0x557f94a84640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.460+0000 7f1fc382b740 1 -- 172.21.15.45:0/3183296354 >> 172.21.15.45:0/3183296354 conn(0x557f94a73980 msgr2=0x557f94a756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.460+0000 7f1fc382b740 1 -- 172.21.15.45:0/3183296354 shutdown_connections 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.460+0000 7f1fc382b740 1 -- 172.21.15.45:0/3183296354 wait complete. 2024-01-31T05:23:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.460+0000 7f1fc382b740 20 client.15477 trim_cache size 0 max 16384 2024-01-31T05:23:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.486+0000 7f1fc382b740 1 Processor -- start 2024-01-31T05:23:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.486+0000 7f1fc382b740 1 -- start start 2024-01-31T05:23:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.486+0000 7f1fc382b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a78ab0 0x557f94a8a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.486+0000 7f1fc382b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a728a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.486+0000 7f1fc382b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a8adf0 0x557f94a7b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.486+0000 7f1fc382b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f949d9d30 con 0x557f94a79460 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.486+0000 7f1fc382b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f949da010 con 0x557f94a78ab0 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.486+0000 7f1fc382b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f94a6f2b0 con 0x557f94a8adf0 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37442/0 (socket says 172.21.15.45:37442) 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fba59c640 1 -- 172.21.15.45:0/4020872620 learned_addr learned my addr 172.21.15.45:0/4020872620 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/4020872620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a78ab0 0x557f94a8a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/4020872620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a8adf0 0x557f94a7b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fba59c640 1 -- 172.21.15.45:0/4020872620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a8adf0 msgr2=0x557f94a7b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fba59c640 1 --2- 172.21.15.45:0/4020872620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a8adf0 0x557f94a7b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fba59c640 1 -- 172.21.15.45:0/4020872620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a78ab0 msgr2=0x557f94a8a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fba59c640 1 --2- 172.21.15.45:0/4020872620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a78ab0 0x557f94a8a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fba59c640 1 -- 172.21.15.45:0/4020872620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f9497c030 con 0x557f94a79460 2024-01-31T05:23:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fba59c640 1 --2- 172.21.15.45:0/4020872620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a728a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1fac003690 tx=0x7f1fac02cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafdf09b3f8a9092 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fa3fff640 1 -- 172.21.15.45:0/4020872620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac004dd0 con 0x557f94a79460 2024-01-31T05:23:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fa3fff640 1 -- 172.21.15.45:0/4020872620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fac044070 con 0x557f94a79460 2024-01-31T05:23:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.487+0000 7f1fa3fff640 1 -- 172.21.15.45:0/4020872620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac003e90 con 0x557f94a79460 2024-01-31T05:23:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.488+0000 7f1fc382b740 1 -- 172.21.15.45:0/4020872620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 msgr2=0x557f94a728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.488+0000 7f1fc382b740 1 --2- 172.21.15.45:0/4020872620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a728a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1fac003690 tx=0x7f1fac02cd30 comp rx=0 tx=0).stop 2024-01-31T05:23:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.488+0000 7f1fc382b740 1 -- 172.21.15.45:0/4020872620 shutdown_connections 2024-01-31T05:23:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.488+0000 7f1fc382b740 1 --2- 172.21.15.45:0/4020872620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a79460 0x557f94a728a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.488+0000 7f1fc382b740 1 --2- 172.21.15.45:0/4020872620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a8adf0 0x557f94a7b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.488+0000 7f1fc382b740 1 --2- 172.21.15.45:0/4020872620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a78ab0 0x557f94a8a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.488+0000 7f1fc382b740 1 -- 172.21.15.45:0/4020872620 >> 172.21.15.45:0/4020872620 conn(0x557f94a73980 msgr2=0x557f94a72190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.488+0000 7f1fc382b740 1 -- 172.21.15.45:0/4020872620 shutdown_connections 2024-01-31T05:23:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.488+0000 7f1fc382b740 1 -- 172.21.15.45:0/4020872620 wait complete. 2024-01-31T05:23:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fc382b740 1 -- start start 2024-01-31T05:23:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fc382b740 1 Processor -- start 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fc382b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a78ab0 0x557f94af3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fc382b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a79460 0x557f94af3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fc382b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a8adf0 0x557f94aed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fc382b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f94aedfe0 con 0x557f94a8adf0 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fc382b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f94aee150 con 0x557f94a78ab0 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fc382b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f94aee310 con 0x557f94a79460 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fc382b740 10 client.0 _ll_register_callbacks cb 0x7f1fb0009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fb9d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a8adf0 0x557f94aed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fb9d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a8adf0 0x557f94aed940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37456/0 (socket says 172.21.15.45:37456) 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fb9d9b640 1 -- 172.21.15.45:0/490976741 learned_addr learned my addr 172.21.15.45:0/490976741 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fba59c640 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a79460 0x557f94af3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.489+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a78ab0 0x557f94af3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fba59c640 1 -- 172.21.15.45:0/490976741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a78ab0 msgr2=0x557f94af3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fba59c640 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a78ab0 0x557f94af3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fba59c640 1 -- 172.21.15.45:0/490976741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a8adf0 msgr2=0x557f94aed940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fba59c640 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a8adf0 0x557f94aed940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fba59c640 1 -- 172.21.15.45:0/490976741 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f949df6a0 con 0x557f94a79460 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a8adf0 0x557f94aed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fba59c640 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a79460 0x557f94af3a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1fac02c170 tx=0x7f1fac02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de4fe923276a034a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fa1ffb640 1 -- 172.21.15.45:0/490976741 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0051b0 con 0x557f94a79460 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fa1ffb640 1 -- 172.21.15.45:0/490976741 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1fac046040 con 0x557f94a79460 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fc382b740 1 -- 172.21.15.45:0/490976741 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fb0005020 con 0x557f94a79460 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fa1ffb640 1 -- 172.21.15.45:0/490976741 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fac005310 con 0x557f94a79460 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fc382b740 10 client.25033 fetch_fsmap learned FSMap version 21 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.490+0000 7f1fc382b740 1 -- 172.21.15.45:0/490976741 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x557f949a1a50 con 0x557f94a79460 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.491+0000 7f1fa1ffb640 1 -- 172.21.15.45:0/490976741 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fac0040f0 con 0x557f94a79460 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.491+0000 7f1fa1ffb640 1 -- 172.21.15.45:0/490976741 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1fac052040 con 0x557f94a79460 2024-01-31T05:23:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.491+0000 7f1fc382b740 10 client.25033 fetch_fsmap finished waiting for FSMap version 21 2024-01-31T05:23:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.491+0000 7f1fc382b740 10 client.25033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:23:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.491+0000 7f1fc382b740 20 client.25033 populate_metadata read hostname 'smithi045' 2024-01-31T05:23:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.491+0000 7f1fc382b740 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f94aa3480 0x557f94afcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:23:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.491+0000 7f1fc382b740 4 client.25033 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:23:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.491+0000 7f1fc382b740 1 -- 172.21.15.45:0/490976741 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f1fb0005020 con 0x557f94aa3480 2024-01-31T05:23:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.491+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f94aa3480 0x557f94afcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:23:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.491+0000 7f1fa1ffb640 10 client.25033 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.491+0000 7f1fb9d9b640 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f94aa3480 0x557f94afcf30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d546fc6b79b364d5 server_cookie=8b2b6c7e7314725d in_seq=0 out_seq=0 2024-01-31T05:23:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fa1ffb640 1 -- 172.21.15.45:0/490976741 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f1fb0005020 con 0x557f94aa3480 2024-01-31T05:23:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fa1ffb640 10 client.25033 handle_command_reply: tid=0 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fc382b740 1 client.25033 shutdown 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fc382b740 1 -- 172.21.15.45:0/490976741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a79460 msgr2=0x557f94af3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fc382b740 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a79460 0x557f94af3a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1fac02c170 tx=0x7f1fac02bc30 comp rx=0 tx=0).stop 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fc382b740 1 -- 172.21.15.45:0/490976741 shutdown_connections 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fc382b740 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f94aa3480 0x557f94afcf30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fc382b740 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f94a8adf0 0x557f94aed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fa1ffb640 0 client.25033 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fc382b740 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f94a79460 0x557f94af3a60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fc382b740 1 --2- 172.21.15.45:0/490976741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f94a78ab0 0x557f94af3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fc382b740 1 -- 172.21.15.45:0/490976741 >> 172.21.15.45:0/490976741 conn(0x557f94a73980 msgr2=0x557f94a72a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.495+0000 7f1fc382b740 1 -- 172.21.15.45:0/490976741 shutdown_connections 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.496+0000 7f1fc382b740 1 -- 172.21.15.45:0/490976741 wait complete. 2024-01-31T05:23:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.496+0000 7f1fc382b740 20 client.25033 trim_cache size 0 max 16384 2024-01-31T05:23:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:23:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:23:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:23:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:23:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.496+0000 7f1fc0a67640 1 -- 172.21.15.45:0/1674853558 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1fa40a1690 msgr2=0x7f1fa40a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.496+0000 7f1fc0a67640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1fa40a1690 0x7f1fa40a3b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40cac10 tx=0x7f1fb4009a70 comp rx=0 tx=0).stop 2024-01-31T05:23:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.496+0000 7f1fc0a67640 1 -- 172.21.15.45:0/1674853558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fbc19baf0 msgr2=0x7f1fbc19bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:23:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.496+0000 7f1fc0a67640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fbc19baf0 0x7f1fbc19bfa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400eb10 tx=0x7f1fb400c2d0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.497+0000 7f1fc0a67640 1 -- 172.21.15.45:0/1674853558 shutdown_connections 2024-01-31T05:23:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.497+0000 7f1fc0a67640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1fa40a1690 0x7f1fa40a3b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.497+0000 7f1fc0a67640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fbc19baf0 0x7f1fbc19bfa0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.497+0000 7f1fc0a67640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fbc106020 0x7f1fbc1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.497+0000 7f1fc0a67640 1 --2- 172.21.15.45:0/1674853558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fbc0ff460 0x7f1fbc197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:23:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.497+0000 7f1fc0a67640 1 -- 172.21.15.45:0/1674853558 >> 172.21.15.45:0/1674853558 conn(0x7f1fbc076140 msgr2=0x7f1fbc1041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:23:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.497+0000 7f1fc0a67640 1 -- 172.21.15.45:0/1674853558 shutdown_connections 2024-01-31T05:23:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:23:49.497+0000 7f1fc0a67640 1 -- 172.21.15.45:0/1674853558 wait complete. 2024-01-31T05:23:50.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:50 smithi186 ceph-mon[35659]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:23:50.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:50 smithi045 ceph-mon[31585]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:23:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:50 smithi156 ceph-mon[35720]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:23:52.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:52 smithi186 ceph-mon[35659]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:23:52.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:52 smithi045 ceph-mon[31585]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:23:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:52 smithi156 ceph-mon[35720]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-31T05:23:54.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:54 smithi186 ceph-mon[35659]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-31T05:23:54.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:54 smithi045 ceph-mon[31585]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-31T05:23:54.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:54 smithi156 ceph-mon[35720]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-31T05:23:56.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:56 smithi186 ceph-mon[35659]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:23:56.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:56 smithi045 ceph-mon[31585]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:23:56.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:56 smithi156 ceph-mon[35720]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:23:58.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:23:58 smithi186 ceph-mon[35659]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-31T05:23:58.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:23:58 smithi045 ceph-mon[31585]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-31T05:23:58.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:23:58 smithi156 ceph-mon[35720]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-31T05:23:59.295 INFO:tasks.workunit.client.0.smithi045.stderr:Updating files: 96% (11784/12183) Updating files: 97% (11818/12183) Updating files: 98% (11940/12183) Updating files: 99% (12062/12183) Updating files: 100% (12183/12183) Updating files: 100% (12183/12183), done. 2024-01-31T05:23:59.601 INFO:tasks.workunit.client.0.smithi045.stderr:Note: switching to '661946e7044723130c03627323f6a879a9848cb5'. 2024-01-31T05:23:59.601 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-01-31T05:23:59.601 INFO:tasks.workunit.client.0.smithi045.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-01-31T05:23:59.601 INFO:tasks.workunit.client.0.smithi045.stderr:changes and commit them, and you can discard any commits you make in this 2024-01-31T05:23:59.601 INFO:tasks.workunit.client.0.smithi045.stderr:state without impacting any branches by switching back to a branch. 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr:If you want to create a new branch to retain commits you create, you may 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr:do so (now or later) by using -c with the switch command. Example: 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr: git switch -c 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr:Or undo this operation with: 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr: git switch - 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-01-31T05:23:59.602 INFO:tasks.workunit.client.0.smithi045.stderr:HEAD is now at 661946e7044 Merge PR #53901 into wip-vshankar-testing-20240125.044502 2024-01-31T05:23:59.603 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-01-31T05:23:59.663 INFO:tasks.workunit.client.0.smithi045.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-01-31T05:23:59.665 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-01-31T05:23:59.665 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-01-31T05:24:00.378 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-01-31T05:24:00.449 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-01-31T05:24:00.488 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-01-31T05:24:00.489 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-01-31T05:24:00.489 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-01-31T05:24:00.528 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-01-31T05:24:00.531 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:24:00.531 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-01-31T05:24:00.586 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-01-31T05:24:00.587 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-01-31T05:24:00.587 DEBUG:teuthology.orchestra.run.smithi045:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=661946e7044723130c03627323f6a879a9848cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-01-31T05:24:00.646 INFO:tasks.workunit.client.0.smithi045.stderr:+ dbench 1 2024-01-31T05:24:00.661 INFO:tasks.workunit.client.0.smithi045.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-01-31T05:24:00.661 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-01-31T05:24:00.661 INFO:tasks.workunit.client.0.smithi045.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-01-31T05:24:00.670 INFO:tasks.workunit.client.0.smithi045.stdout:0 of 1 processes prepared for launch 0 sec 2024-01-31T05:24:00.770 INFO:tasks.workunit.client.0.smithi045.stdout:1 of 1 processes prepared for launch 0 sec 2024-01-31T05:24:00.770 INFO:tasks.workunit.client.0.smithi045.stdout:releasing clients 2024-01-31T05:24:00.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:00 smithi186 ceph-mon[35659]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:24:00.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:00 smithi045 ceph-mon[31585]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:24:00.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:00 smithi156 ceph-mon[35720]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:24:01.770 INFO:tasks.workunit.client.0.smithi045.stdout: 1 3197 34.28 MB/sec warmup 1 sec latency 6.615 ms 2024-01-31T05:24:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.586+0000 7f5dd312b640 1 -- 172.21.15.45:0/3880867268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc105fe0 msgr2=0x7f5dcc10dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.586+0000 7f5dd312b640 1 --2- 172.21.15.45:0/3880867268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc105fe0 0x7f5dcc10dbb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009cd0 tx=0x7f5dc402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.586+0000 7f5dd312b640 1 -- 172.21.15.45:0/3880867268 shutdown_connections 2024-01-31T05:24:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.586+0000 7f5dd312b640 1 --2- 172.21.15.45:0/3880867268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc105fe0 0x7f5dcc10dbb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.586+0000 7f5dd312b640 1 --2- 172.21.15.45:0/3880867268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dcc1018c0 0x7f5dcc105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.586+0000 7f5dd312b640 1 --2- 172.21.15.45:0/3880867268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dcc100f10 0x7f5dcc1012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.586+0000 7f5dd312b640 1 -- 172.21.15.45:0/3880867268 >> 172.21.15.45:0/3880867268 conn(0x7f5dcc0fc7b0 msgr2=0x7f5dcc0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.586+0000 7f5dd312b640 1 -- 172.21.15.45:0/3880867268 shutdown_connections 2024-01-31T05:24:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.586+0000 7f5dd312b640 1 -- 172.21.15.45:0/3880867268 wait complete. 2024-01-31T05:24:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd312b640 1 Processor -- start 2024-01-31T05:24:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd312b640 1 -- start start 2024-01-31T05:24:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd312b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dcc100f10 0x7f5dcc1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd312b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dcc105fe0 0x7f5dcc1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd312b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc19ba60 0x7f5dcc19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd312b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc19c450 con 0x7f5dcc19ba60 2024-01-31T05:24:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd312b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc19c5c0 con 0x7f5dcc100f10 2024-01-31T05:24:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd312b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc19c730 con 0x7f5dcc105fe0 2024-01-31T05:24:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd16c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc19ba60 0x7f5dcc19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd16c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc19ba60 0x7f5dcc19bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52592/0 (socket says 172.21.15.45:52592) 2024-01-31T05:24:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd16c8640 1 -- 172.21.15.45:0/2408703172 learned_addr learned my addr 172.21.15.45:0/2408703172 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd0ec7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dcc100f10 0x7f5dcc1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dcc105fe0 0x7f5dcc1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd16c8640 1 -- 172.21.15.45:0/2408703172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dcc105fe0 msgr2=0x7f5dcc1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd16c8640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dcc105fe0 0x7f5dcc1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd16c8640 1 -- 172.21.15.45:0/2408703172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dcc100f10 msgr2=0x7f5dcc1971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd16c8640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dcc100f10 0x7f5dcc1971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd16c8640 1 -- 172.21.15.45:0/2408703172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc4009900 con 0x7f5dcc19ba60 2024-01-31T05:24:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd0ec7640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dcc100f10 0x7f5dcc1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:24:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.587+0000 7f5dd16c8640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc19ba60 0x7f5dcc19bf10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009ca0 tx=0x7f5dc4032550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.588+0000 7f5dc9ffb640 1 -- 172.21.15.45:0/2408703172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4032690 con 0x7f5dcc19ba60 2024-01-31T05:24:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.588+0000 7f5dd312b640 1 -- 172.21.15.45:0/2408703172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc1a3d10 con 0x7f5dcc19ba60 2024-01-31T05:24:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.588+0000 7f5dc9ffb640 1 -- 172.21.15.45:0/2408703172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dc4003970 con 0x7f5dcc19ba60 2024-01-31T05:24:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.588+0000 7f5dc9ffb640 1 -- 172.21.15.45:0/2408703172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40045e0 con 0x7f5dcc19ba60 2024-01-31T05:24:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.588+0000 7f5dd312b640 1 -- 172.21.15.45:0/2408703172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc1a4190 con 0x7f5dcc19ba60 2024-01-31T05:24:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.590+0000 7f5dc9ffb640 1 -- 172.21.15.45:0/2408703172 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 136128+0+0 (secure 0 0 0) 0x7f5dc403b040 con 0x7f5dcc19ba60 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.590+0000 7f5dd5fc9740 1 Processor -- start 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.590+0000 7f5dd5fc9740 1 -- start start 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dc9ffb640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5dbc0a1610 0x7f5dbc0a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd5fc9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da0060 0x564322da0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd5fc9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0980 0x564322da9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd5fc9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da9750 0x564322dabb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd5fc9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564322d0f040 con 0x564322da9750 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd5fc9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564322d0c730 con 0x564322da0060 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd5fc9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564322d08020 con 0x564322da0980 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5dbc0a1610 0x7f5dbc0a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dc9ffb640 1 -- 172.21.15.45:0/2408703172 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5dc40efa70 con 0x7f5dcc19ba60 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd0ec7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da0060 0x564322da0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd0ec7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da0060 0x564322da0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:39270/0 (socket says 172.21.15.45:39270) 2024-01-31T05:24:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd0ec7640 1 -- 172.21.15.45:0/3164644357 learned_addr learned my addr 172.21.15.45:0/3164644357 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd16c8640 1 --2- 172.21.15.45:0/3164644357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da9750 0x564322dabb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/3164644357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0980 0x564322da9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd0ec7640 1 -- 172.21.15.45:0/3164644357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0980 msgr2=0x564322da9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd0ec7640 1 --2- 172.21.15.45:0/3164644357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0980 0x564322da9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd0ec7640 1 -- 172.21.15.45:0/3164644357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da9750 msgr2=0x564322dabb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd0ec7640 1 --2- 172.21.15.45:0/3164644357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da9750 0x564322dabb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd0ec7640 1 -- 172.21.15.45:0/3164644357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564322cffe40 con 0x564322da0060 2024-01-31T05:24:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd16c8640 1 --2- 172.21.15.45:0/3164644357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da9750 0x564322dabb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:24:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/3164644357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0980 0x564322da9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:24:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5dbc0a1610 0x7f5dbc0a3ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x564322d9fb10 tx=0x7f5db80097c0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.591+0000 7f5dd0ec7640 1 --2- 172.21.15.45:0/3164644357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da0060 0x564322da0440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00115f0 tx=0x7f5dc0011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b933cf750ffac5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.592+0000 7f5db6ffd640 1 -- 172.21.15.45:0/3164644357 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc000f640 con 0x564322da0060 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.592+0000 7f5db6ffd640 1 -- 172.21.15.45:0/3164644357 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dc000fc60 con 0x564322da0060 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.592+0000 7f5db6ffd640 1 -- 172.21.15.45:0/3164644357 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0019c50 con 0x564322da0060 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.592+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3164644357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da0060 msgr2=0x564322da0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.592+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3164644357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da0060 0x564322da0440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00115f0 tx=0x7f5dc0011ab0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.592+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3164644357 shutdown_connections 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.592+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3164644357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da9750 0x564322dabb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.592+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3164644357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0980 0x564322da9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.592+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3164644357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da0060 0x564322da0440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.592+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3164644357 >> 172.21.15.45:0/3164644357 conn(0x564322d9a980 msgr2=0x564322d9ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.592+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3164644357 shutdown_connections 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.592+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3164644357 wait complete. 2024-01-31T05:24:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd5fc9740 1 -- start start 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd5fc9740 1 Processor -- start 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd5fc9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0060 0x564322da66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd5fc9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0980 0x564322da6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd5fc9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da9750 0x564322da7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd5fc9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564322cc11b0 con 0x564322da0060 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd5fc9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564322cc0e50 con 0x564322da9750 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd5fc9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564322c86090 con 0x564322da0980 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd5fc9740 10 client.0 _ll_register_callbacks cb 0x564322caba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd0ec7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0060 0x564322da66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd0ec7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0060 0x564322da66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52602/0 (socket says 172.21.15.45:52602) 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd0ec7640 1 -- 172.21.15.45:0/3729884441 learned_addr learned my addr 172.21.15.45:0/3729884441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dd16c8640 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da9750 0x564322da7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.593+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0980 0x564322da6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd0ec7640 1 -- 172.21.15.45:0/3729884441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0980 msgr2=0x564322da6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd0ec7640 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0980 0x564322da6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd0ec7640 1 -- 172.21.15.45:0/3729884441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da9750 msgr2=0x564322da7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd0ec7640 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da9750 0x564322da7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd0ec7640 1 -- 172.21.15.45:0/3729884441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc0009d40 con 0x564322da0060 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd16c8640 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da9750 0x564322da7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd0ec7640 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0060 0x564322da66c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00057c0 tx=0x7f5dc0005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226216f0221817f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dabfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dabfff640 1 -- 172.21.15.45:0/3729884441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc001ae70 con 0x564322da0060 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dabfff640 1 -- 172.21.15.45:0/3729884441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5dc000f640 con 0x564322da0060 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3729884441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564322d01010 con 0x564322da0060 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dabfff640 1 -- 172.21.15.45:0/3729884441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dc000f800 con 0x564322da0060 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd5fc9740 10 client.15495 fetch_fsmap learned FSMap version 21 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3729884441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x564322d066a0 con 0x564322da0060 2024-01-31T05:24:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dabfff640 1 -- 172.21.15.45:0/3729884441 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564322d01010 con 0x564322da0060 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dabfff640 1 -- 172.21.15.45:0/3729884441 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5dc00046c0 con 0x564322da0060 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd5fc9740 10 client.15495 fetch_fsmap finished waiting for FSMap version 21 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd5fc9740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd5fc9740 20 client.15495 populate_metadata read hostname 'smithi045' 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x564322db9440 0x564322dbb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd5fc9740 4 client.15495 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.594+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3729884441 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564322d00d30 con 0x564322db9440 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.595+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x564322db9440 0x564322dbb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.595+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x564322db9440 0x564322dbb860 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d50d571369aadc server_cookie=1ecc979698b677e5 in_seq=0 out_seq=0 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.595+0000 7f5dabfff640 10 client.15495 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dabfff640 1 -- 172.21.15.45:0/3729884441 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564322d00d30 con 0x564322db9440 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dabfff640 10 client.15495 handle_command_reply: tid=0 2024-01-31T05:24:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dd5fc9740 1 client.15495 shutdown 2024-01-31T05:24:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3729884441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0060 msgr2=0x564322da66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0060 0x564322da66c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00057c0 tx=0x7f5dc0005d90 comp rx=0 tx=0).stop 2024-01-31T05:24:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3729884441 shutdown_connections 2024-01-31T05:24:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x564322db9440 0x564322dbb860 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da9750 0x564322da7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0980 0x564322da6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dabfff640 0 client.15495 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3729884441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0060 0x564322da66c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3729884441 >> 172.21.15.45:0/3729884441 conn(0x564322d9a980 msgr2=0x564322d9c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3729884441 shutdown_connections 2024-01-31T05:24:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3729884441 wait complete. 2024-01-31T05:24:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.596+0000 7f5dd5fc9740 20 client.15495 trim_cache size 0 max 16384 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dd5fc9740 1 Processor -- start 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dd5fc9740 1 -- start start 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dd5fc9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0060 0x564322da8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dd5fc9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 0x564322da6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dd5fc9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da9750 0x564322da3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dd5fc9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564322da15b0 con 0x564322da0980 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dd5fc9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564322d00d30 con 0x564322da9750 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dd5fc9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564322d01010 con 0x564322da0060 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dcbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 0x564322da6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dcbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 0x564322da6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52618/0 (socket says 172.21.15.45:52618) 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dcbfff640 1 -- 172.21.15.45:0/3800241003 learned_addr learned my addr 172.21.15.45:0/3800241003 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dd0ec7640 1 --2- 172.21.15.45:0/3800241003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0060 0x564322da8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dd16c8640 1 --2- 172.21.15.45:0/3800241003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da9750 0x564322da3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dcbfff640 1 -- 172.21.15.45:0/3800241003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0060 msgr2=0x564322da8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/3800241003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0060 0x564322da8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dcbfff640 1 -- 172.21.15.45:0/3800241003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da9750 msgr2=0x564322da3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/3800241003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da9750 0x564322da3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dcbfff640 1 -- 172.21.15.45:0/3800241003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564322ca3030 con 0x564322da0980 2024-01-31T05:24:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.610+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/3800241003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 0x564322da6f60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5db8015010 tx=0x7f5db8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d65d7635ce260d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.611+0000 7f5db67fc640 1 -- 172.21.15.45:0/3800241003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8026040 con 0x564322da0980 2024-01-31T05:24:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.611+0000 7f5db67fc640 1 -- 172.21.15.45:0/3800241003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db8004400 con 0x564322da0980 2024-01-31T05:24:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.611+0000 7f5db67fc640 1 -- 172.21.15.45:0/3800241003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db80285c0 con 0x564322da0980 2024-01-31T05:24:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.611+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3800241003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 msgr2=0x564322da6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.611+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3800241003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 0x564322da6f60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5db8015010 tx=0x7f5db8015040 comp rx=0 tx=0).stop 2024-01-31T05:24:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.611+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3800241003 shutdown_connections 2024-01-31T05:24:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.611+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3800241003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 0x564322da6f60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.611+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3800241003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da9750 0x564322da3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.611+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/3800241003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da0060 0x564322da8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.611+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3800241003 >> 172.21.15.45:0/3800241003 conn(0x564322d9a980 msgr2=0x564322da19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.611+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3800241003 shutdown_connections 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.611+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/3800241003 wait complete. 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dd5fc9740 1 -- start start 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dd5fc9740 1 Processor -- start 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dd5fc9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da0060 0x564322db0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dd5fc9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 0x564322daf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dd5fc9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da9750 0x564322daf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dd5fc9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564322dafde0 con 0x564322da0980 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dd5fc9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564322daff50 con 0x564322da0060 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dd5fc9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564322e18570 con 0x564322da9750 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dd5fc9740 10 client.0 _ll_register_callbacks cb 0x7f5dc0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dcbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 0x564322daf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dcbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 0x564322daf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52626/0 (socket says 172.21.15.45:52626) 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dcbfff640 1 -- 172.21.15.45:0/909088611 learned_addr learned my addr 172.21.15.45:0/909088611 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dd0ec7640 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da0060 0x564322db0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.612+0000 7f5dd16c8640 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da9750 0x564322daf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dcbfff640 1 -- 172.21.15.45:0/909088611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da9750 msgr2=0x564322daf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da9750 0x564322daf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dcbfff640 1 -- 172.21.15.45:0/909088611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da0060 msgr2=0x564322db0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da0060 0x564322db0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dcbfff640 1 -- 172.21.15.45:0/909088611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db8013340 con 0x564322da0980 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dcbfff640 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 0x564322daf100 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5db8013210 tx=0x7f5db8005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bc823a4d23f46b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dabfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dabfff640 1 -- 172.21.15.45:0/909088611 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8026040 con 0x564322da0980 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dabfff640 1 -- 172.21.15.45:0/909088611 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5db8003e10 con 0x564322da0980 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/909088611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5dc000fc60 con 0x564322da0980 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dabfff640 1 -- 172.21.15.45:0/909088611 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5db8028540 con 0x564322da0980 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dd5fc9740 10 client.15501 fetch_fsmap learned FSMap version 21 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/909088611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x564322cc8a50 con 0x564322da0980 2024-01-31T05:24:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dabfff640 1 -- 172.21.15.45:0/909088611 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dc000fc60 con 0x564322da0980 2024-01-31T05:24:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.613+0000 7f5dabfff640 1 -- 172.21.15.45:0/909088611 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5db800fb10 con 0x564322da0980 2024-01-31T05:24:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.614+0000 7f5dd5fc9740 10 client.15501 fetch_fsmap finished waiting for FSMap version 21 2024-01-31T05:24:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.614+0000 7f5dd5fc9740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:24:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.614+0000 7f5dd5fc9740 20 client.15501 populate_metadata read hostname 'smithi045' 2024-01-31T05:24:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.614+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x564322e19ce0 0x564322e1c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.614+0000 7f5dd5fc9740 4 client.15501 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:24:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.614+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/909088611 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564322a07600 con 0x564322e19ce0 2024-01-31T05:24:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.614+0000 7f5dd0ec7640 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x564322e19ce0 0x564322e1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.614+0000 7f5dabfff640 10 client.15501 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.614+0000 7f5dd0ec7640 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x564322e19ce0 0x564322e1c100 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96007a621ac5ebbc server_cookie=55d86d329071c6c0 in_seq=0 out_seq=0 2024-01-31T05:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dabfff640 1 -- 172.21.15.45:0/909088611 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564322a07600 con 0x564322e19ce0 2024-01-31T05:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dabfff640 10 client.15501 handle_command_reply: tid=0 2024-01-31T05:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dd5fc9740 1 client.15501 shutdown 2024-01-31T05:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/909088611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 msgr2=0x564322daf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 0x564322daf100 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5db8013210 tx=0x7f5db8005760 comp rx=0 tx=0).stop 2024-01-31T05:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/909088611 shutdown_connections 2024-01-31T05:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x564322e19ce0 0x564322e1c100 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564322da0980 0x564322daf100 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dabfff640 0 client.15501 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564322da9750 0x564322daf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dd5fc9740 1 --2- 172.21.15.45:0/909088611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564322da0060 0x564322db0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/909088611 >> 172.21.15.45:0/909088611 conn(0x564322d9a980 msgr2=0x564322d99190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/909088611 shutdown_connections 2024-01-31T05:24:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.671+0000 7f5dd5fc9740 1 -- 172.21.15.45:0/909088611 wait complete. 2024-01-31T05:24:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd5fc9740 20 client.15501 trim_cache size 0 max 16384 2024-01-31T05:24:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:24:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:24:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:24:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:24:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd312b640 1 -- 172.21.15.45:0/2408703172 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5dbc0a1610 msgr2=0x7f5dbc0a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd312b640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5dbc0a1610 0x7f5dbc0a3ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x564322d9fb10 tx=0x7f5db80097c0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd312b640 1 -- 172.21.15.45:0/2408703172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc19ba60 msgr2=0x7f5dcc19bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd312b640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc19ba60 0x7f5dcc19bf10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009ca0 tx=0x7f5dc4032550 comp rx=0 tx=0).stop 2024-01-31T05:24:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd312b640 1 -- 172.21.15.45:0/2408703172 shutdown_connections 2024-01-31T05:24:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd312b640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5dbc0a1610 0x7f5dbc0a3ad0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd312b640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dcc19ba60 0x7f5dcc19bf10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd312b640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dcc105fe0 0x7f5dcc1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd312b640 1 --2- 172.21.15.45:0/2408703172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dcc100f10 0x7f5dcc1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd312b640 1 -- 172.21.15.45:0/2408703172 >> 172.21.15.45:0/2408703172 conn(0x7f5dcc0fc7b0 msgr2=0x7f5dcc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd312b640 1 -- 172.21.15.45:0/2408703172 shutdown_connections 2024-01-31T05:24:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:02.672+0000 7f5dd312b640 1 -- 172.21.15.45:0/2408703172 wait complete. 2024-01-31T05:24:02.770 INFO:tasks.workunit.client.0.smithi045.stdout: 1 5905 23.51 MB/sec warmup 2 sec latency 34.354 ms 2024-01-31T05:24:02.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:02 smithi186 ceph-mon[35659]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:24:02.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:02 smithi045 ceph-mon[31585]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:24:02.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:02 smithi156 ceph-mon[35720]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-31T05:24:03.771 INFO:tasks.workunit.client.0.smithi045.stdout: 1 8352 20.44 MB/sec warmup 3 sec latency 6.993 ms 2024-01-31T05:24:04.770 INFO:tasks.workunit.client.0.smithi045.stdout: 1 11313 19.61 MB/sec warmup 4 sec latency 10.019 ms 2024-01-31T05:24:04.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:04 smithi186 ceph-mon[35659]: pgmap v44: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 279 KiB/s wr, 52 op/s 2024-01-31T05:24:04.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:04 smithi045 ceph-mon[31585]: pgmap v44: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 279 KiB/s wr, 52 op/s 2024-01-31T05:24:04.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:04 smithi156 ceph-mon[35720]: pgmap v44: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 279 KiB/s wr, 52 op/s 2024-01-31T05:24:05.771 INFO:tasks.workunit.client.0.smithi045.stdout: 1 14412 19.42 MB/sec warmup 5 sec latency 31.212 ms 2024-01-31T05:24:06.771 INFO:tasks.workunit.client.0.smithi045.stdout: 1 16530 17.39 MB/sec warmup 6 sec latency 116.618 ms 2024-01-31T05:24:06.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:06 smithi186 ceph-mon[35659]: pgmap v45: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 279 KiB/s wr, 49 op/s 2024-01-31T05:24:06.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:06 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:24:06.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:06 smithi045 ceph-mon[31585]: pgmap v45: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 279 KiB/s wr, 49 op/s 2024-01-31T05:24:06.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:06 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:24:06.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:06 smithi156 ceph-mon[35720]: pgmap v45: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 279 KiB/s wr, 49 op/s 2024-01-31T05:24:06.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:06 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:24:07.771 INFO:tasks.workunit.client.0.smithi045.stdout: 1 16530 14.90 MB/sec warmup 7 sec latency 1116.648 ms 2024-01-31T05:24:08.771 INFO:tasks.workunit.client.0.smithi045.stdout: 1 16530 13.04 MB/sec warmup 8 sec latency 2116.699 ms 2024-01-31T05:24:08.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:08 smithi186 ceph-mon[35659]: pgmap v46: 97 pgs: 97 active+clean; 9.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 877 KiB/s wr, 171 op/s 2024-01-31T05:24:08.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:08 smithi045 ceph-mon[31585]: pgmap v46: 97 pgs: 97 active+clean; 9.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 877 KiB/s wr, 171 op/s 2024-01-31T05:24:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:08 smithi156 ceph-mon[35720]: pgmap v46: 97 pgs: 97 active+clean; 9.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 877 KiB/s wr, 171 op/s 2024-01-31T05:24:09.771 INFO:tasks.workunit.client.0.smithi045.stdout: 1 16530 11.59 MB/sec warmup 9 sec latency 3116.733 ms 2024-01-31T05:24:10.771 INFO:tasks.workunit.client.0.smithi045.stdout: 1 16530 10.43 MB/sec warmup 10 sec latency 4116.804 ms 2024-01-31T05:24:10.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:10 smithi186 ceph-mon[35659]: pgmap v47: 97 pgs: 97 active+clean; 9.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 877 KiB/s wr, 169 op/s 2024-01-31T05:24:10.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:10 smithi045 ceph-mon[31585]: pgmap v47: 97 pgs: 97 active+clean; 9.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 877 KiB/s wr, 169 op/s 2024-01-31T05:24:10.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:10 smithi156 ceph-mon[35720]: pgmap v47: 97 pgs: 97 active+clean; 9.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 877 KiB/s wr, 169 op/s 2024-01-31T05:24:11.771 INFO:tasks.workunit.client.0.smithi045.stdout: 1 17813 10.46 MB/sec warmup 11 sec latency 4573.383 ms 2024-01-31T05:24:11.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:11 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:24:11.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:11 smithi186 ceph-mon[35659]: pgmap v48: 97 pgs: 97 active+clean; 9.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 877 KiB/s wr, 169 op/s 2024-01-31T05:24:11.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:11 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:24:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:11 smithi045 ceph-mon[31585]: pgmap v48: 97 pgs: 97 active+clean; 9.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 877 KiB/s wr, 169 op/s 2024-01-31T05:24:11.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:11 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:24:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:11 smithi156 ceph-mon[35720]: pgmap v48: 97 pgs: 97 active+clean; 9.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 877 KiB/s wr, 169 op/s 2024-01-31T05:24:12.771 INFO:tasks.workunit.client.0.smithi045.stdout: 1 20173 10.30 MB/sec warmup 12 sec latency 81.167 ms 2024-01-31T05:24:13.771 INFO:tasks.workunit.client.0.smithi045.stdout: 1 20173 9.51 MB/sec warmup 13 sec latency 1081.283 ms 2024-01-31T05:24:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:13 smithi186 ceph-mon[35659]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-01-31T05:24:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:13 smithi186 ceph-mon[35659]: pgmap v49: 97 pgs: 97 active+clean; 11 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 193 op/s 2024-01-31T05:24:13.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:13 smithi045 ceph-mon[31585]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-01-31T05:24:13.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:13 smithi045 ceph-mon[31585]: pgmap v49: 97 pgs: 97 active+clean; 11 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 193 op/s 2024-01-31T05:24:13.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:13 smithi156 ceph-mon[35720]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-01-31T05:24:13.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:13 smithi156 ceph-mon[35720]: pgmap v49: 97 pgs: 97 active+clean; 11 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 193 op/s 2024-01-31T05:24:14.771 INFO:tasks.workunit.client.0.smithi045.stdout: 1 20173 8.83 MB/sec warmup 14 sec latency 2081.402 ms 2024-01-31T05:24:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.770+0000 7f468d7ae640 1 -- 172.21.15.45:0/1933286440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4688100f30 msgr2=0x7f4688101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.770+0000 7f468d7ae640 1 --2- 172.21.15.45:0/1933286440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4688100f30 0x7f4688101310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4678009930 tx=0x7f467802bc30 comp rx=0 tx=0).stop 2024-01-31T05:24:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.770+0000 7f468d7ae640 1 -- 172.21.15.45:0/1933286440 shutdown_connections 2024-01-31T05:24:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.770+0000 7f468d7ae640 1 --2- 172.21.15.45:0/1933286440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4688106000 0x7f468810dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.770+0000 7f468d7ae640 1 --2- 172.21.15.45:0/1933286440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46881018e0 0x7f4688105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.770+0000 7f468d7ae640 1 --2- 172.21.15.45:0/1933286440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4688100f30 0x7f4688101310 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.770+0000 7f468d7ae640 1 -- 172.21.15.45:0/1933286440 >> 172.21.15.45:0/1933286440 conn(0x7f46880fc7d0 msgr2=0x7f46880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.770+0000 7f468d7ae640 1 -- 172.21.15.45:0/1933286440 shutdown_connections 2024-01-31T05:24:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.770+0000 7f468d7ae640 1 -- 172.21.15.45:0/1933286440 wait complete. 2024-01-31T05:24:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f468d7ae640 1 Processor -- start 2024-01-31T05:24:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f468d7ae640 1 -- start start 2024-01-31T05:24:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f468d7ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4688100f30 0x7f4688197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f468d7ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4688106000 0x7f4688197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f468d7ae640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f468819bac0 0x7f468819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f468d7ae640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468819c4b0 con 0x7f4688100f30 2024-01-31T05:24:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f468d7ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468819c620 con 0x7f4688106000 2024-01-31T05:24:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f468d7ae640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468819c790 con 0x7f468819bac0 2024-01-31T05:24:15.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 20173 8.24 MB/sec warmup 15 sec latency 3081.447 ms 2024-01-31T05:24:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f4686ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4688100f30 0x7f4688197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f4686ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4688100f30 0x7f4688197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60584/0 (socket says 172.21.15.45:60584) 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f4686ffd640 1 -- 172.21.15.45:0/123284775 learned_addr learned my addr 172.21.15.45:0/123284775 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f46867fc640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4688106000 0x7f4688197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.771+0000 7f46877fe640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f468819bac0 0x7f468819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.772+0000 7f4686ffd640 1 -- 172.21.15.45:0/123284775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f468819bac0 msgr2=0x7f468819bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.772+0000 7f4686ffd640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f468819bac0 0x7f468819bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.772+0000 7f4686ffd640 1 -- 172.21.15.45:0/123284775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4688106000 msgr2=0x7f4688197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.772+0000 7f4686ffd640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4688106000 0x7f4688197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.772+0000 7f4686ffd640 1 -- 172.21.15.45:0/123284775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4678009560 con 0x7f4688100f30 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.772+0000 7f4686ffd640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4688100f30 0x7f4688197200 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4678005940 tx=0x7f4678004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.772+0000 7f466ffff640 1 -- 172.21.15.45:0/123284775 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46780042d0 con 0x7f4688100f30 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.772+0000 7f468d7ae640 1 -- 172.21.15.45:0/123284775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46881a3d70 con 0x7f4688100f30 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.772+0000 7f466ffff640 1 -- 172.21.15.45:0/123284775 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4678004430 con 0x7f4688100f30 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.772+0000 7f466ffff640 1 -- 172.21.15.45:0/123284775 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678003cc0 con 0x7f4688100f30 2024-01-31T05:24:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.772+0000 7f468d7ae640 1 -- 172.21.15.45:0/123284775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46881a41d0 con 0x7f4688100f30 2024-01-31T05:24:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.774+0000 7f466ffff640 1 -- 172.21.15.45:0/123284775 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f4678005000 con 0x7f4688100f30 2024-01-31T05:24:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.775+0000 7f4690575740 1 Processor -- start 2024-01-31T05:24:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.775+0000 7f4690575740 1 -- start start 2024-01-31T05:24:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.775+0000 7f466ffff640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f46700a14f0 0x7f46700a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.775+0000 7f46877fe640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f46700a14f0 0x7f46700a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.775+0000 7f4690575740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 0x562c0921ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.775+0000 7f4690575740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921f460 0x562c0922c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.775+0000 7f466ffff640 1 -- 172.21.15.45:0/123284775 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f46780ef610 con 0x7f4688100f30 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4690575740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c0922cb60 0x562c0922ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4690575740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c0918e040 con 0x562c0921eb40 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4690575740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c0918b730 con 0x562c0922cb60 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4690575740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09187020 con 0x562c0921f460 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4686ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 0x562c0921ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4686ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 0x562c0921ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60586/0 (socket says 172.21.15.45:60586) 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4686ffd640 1 -- 172.21.15.45:0/3127438358 learned_addr learned my addr 172.21.15.45:0/3127438358 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f46867fc640 1 --2- 172.21.15.45:0/3127438358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921f460 0x562c0922c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f46877fe640 1 --2- 172.21.15.45:0/3127438358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c0922cb60 0x562c0922ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4686ffd640 1 -- 172.21.15.45:0/3127438358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921f460 msgr2=0x562c0922c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4686ffd640 1 --2- 172.21.15.45:0/3127438358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921f460 0x562c0922c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4686ffd640 1 -- 172.21.15.45:0/3127438358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c0922cb60 msgr2=0x562c0922ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4686ffd640 1 --2- 172.21.15.45:0/3127438358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c0922cb60 0x562c0922ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4686ffd640 1 -- 172.21.15.45:0/3127438358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c0917ee40 con 0x562c0921eb40 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f46877fe640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f46700a14f0 0x7f46700a39b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4680005fd0 tx=0x7f4680010740 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f4686ffd640 1 --2- 172.21.15.45:0/3127438358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 0x562c0921ef20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f46780b8f80 tx=0x7f46780bc110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9fffa0bc105a10 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f466d7fa640 1 -- 172.21.15.45:0/3127438358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46780bc510 con 0x562c0921eb40 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f466d7fa640 1 -- 172.21.15.45:0/3127438358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46780c0660 con 0x562c0921eb40 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.776+0000 7f466d7fa640 1 -- 172.21.15.45:0/3127438358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46780053d0 con 0x562c0921eb40 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.777+0000 7f4690575740 1 -- 172.21.15.45:0/3127438358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 msgr2=0x562c0921ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.777+0000 7f4690575740 1 --2- 172.21.15.45:0/3127438358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 0x562c0921ef20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f46780b8f80 tx=0x7f46780bc110 comp rx=0 tx=0).stop 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.777+0000 7f4690575740 1 -- 172.21.15.45:0/3127438358 shutdown_connections 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.777+0000 7f4690575740 1 --2- 172.21.15.45:0/3127438358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c0922cb60 0x562c0922ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.777+0000 7f4690575740 1 --2- 172.21.15.45:0/3127438358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921f460 0x562c0922c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.777+0000 7f4690575740 1 --2- 172.21.15.45:0/3127438358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 0x562c0921ef20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.777+0000 7f4690575740 1 -- 172.21.15.45:0/3127438358 >> 172.21.15.45:0/3127438358 conn(0x562c09219980 msgr2=0x562c09219d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.777+0000 7f4690575740 1 -- 172.21.15.45:0/3127438358 shutdown_connections 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.777+0000 7f4690575740 1 -- 172.21.15.45:0/3127438358 wait complete. 2024-01-31T05:24:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.777+0000 7f4690575740 1 -- start start 2024-01-31T05:24:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.777+0000 7f4690575740 1 Processor -- start 2024-01-31T05:24:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f4690575740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921eb40 0x562c09229a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f4690575740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 0x562c09229f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f4690575740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c0922cb60 0x562c0922a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f4690575740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c091401b0 con 0x562c0921f460 2024-01-31T05:24:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f4690575740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c0913fe50 con 0x562c0922cb60 2024-01-31T05:24:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f4690575740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09105090 con 0x562c0921eb40 2024-01-31T05:24:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f4690575740 10 client.0 _ll_register_callbacks cb 0x562c0912aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:24:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f46867fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 0x562c09229f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f46867fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 0x562c09229f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60600/0 (socket says 172.21.15.45:60600) 2024-01-31T05:24:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f46867fc640 1 -- 172.21.15.45:0/4026273511 learned_addr learned my addr 172.21.15.45:0/4026273511 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f4686ffd640 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921eb40 0x562c09229a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f46877fe640 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c0922cb60 0x562c0922a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f46867fc640 1 -- 172.21.15.45:0/4026273511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921eb40 msgr2=0x562c09229a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f46867fc640 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921eb40 0x562c09229a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f46867fc640 1 -- 172.21.15.45:0/4026273511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c0922cb60 msgr2=0x562c0922a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f46867fc640 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c0922cb60 0x562c0922a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f46867fc640 1 -- 172.21.15.45:0/4026273511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46881a41d0 con 0x562c0921f460 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f46877fe640 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c0922cb60 0x562c0922a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f46867fc640 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 0x562c09229f80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f467c010720 tx=0x7f467c010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1296c489862780f7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.778+0000 7f466aff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f466aff5640 1 -- 172.21.15.45:0/4026273511 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c015070 con 0x562c0921f460 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f466aff5640 1 -- 172.21.15.45:0/4026273511 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f467c00fb30 con 0x562c0921f460 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f4690575740 1 -- 172.21.15.45:0/4026273511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c09180010 con 0x562c0921f460 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f466aff5640 1 -- 172.21.15.45:0/4026273511 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f467c00fcf0 con 0x562c0921f460 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f4690575740 10 client.15513 fetch_fsmap learned FSMap version 23 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f4690575740 1 -- 172.21.15.45:0/4026273511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562c091856a0 con 0x562c0921f460 2024-01-31T05:24:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f466aff5640 1 -- 172.21.15.45:0/4026273511 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c09180010 con 0x562c0921f460 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f466aff5640 1 -- 172.21.15.45:0/4026273511 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f467c00e4d0 con 0x562c0921f460 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f4690575740 10 client.15513 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f4690575740 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f4690575740 20 client.15513 populate_metadata read hostname 'smithi045' 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f4690575740 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562c0923c7b0 0x562c0923ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f4690575740 4 client.15513 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f4690575740 1 -- 172.21.15.45:0/4026273511 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c0917fd30 con 0x562c0923c7b0 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.779+0000 7f4686ffd640 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562c0923c7b0 0x562c0923ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.780+0000 7f4686ffd640 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562c0923c7b0 0x562c0923ebd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=290e24a11ad5a6f3 server_cookie=79d73a2b5afcd84c in_seq=0 out_seq=0 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.780+0000 7f466aff5640 10 client.15513 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f466aff5640 1 -- 172.21.15.45:0/4026273511 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c0917fd30 con 0x562c0923c7b0 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f466aff5640 10 client.15513 handle_command_reply: tid=0 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f4690575740 1 client.15513 shutdown 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f4690575740 1 -- 172.21.15.45:0/4026273511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 msgr2=0x562c09229f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f4690575740 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 0x562c09229f80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f467c010720 tx=0x7f467c010be0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f4690575740 1 -- 172.21.15.45:0/4026273511 shutdown_connections 2024-01-31T05:24:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f4690575740 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562c0923c7b0 0x562c0923ebd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f4690575740 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 0x562c09229f80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f4690575740 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c0922cb60 0x562c0922a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f466aff5640 0 client.15513 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f4690575740 1 --2- 172.21.15.45:0/4026273511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921eb40 0x562c09229a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f4690575740 1 -- 172.21.15.45:0/4026273511 >> 172.21.15.45:0/4026273511 conn(0x562c09219980 msgr2=0x562c0921b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f4690575740 1 -- 172.21.15.45:0/4026273511 shutdown_connections 2024-01-31T05:24:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f4690575740 1 -- 172.21.15.45:0/4026273511 wait complete. 2024-01-31T05:24:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.781+0000 7f4690575740 20 client.15513 trim_cache size 0 max 16384 2024-01-31T05:24:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.794+0000 7f4690575740 1 Processor -- start 2024-01-31T05:24:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.794+0000 7f4690575740 1 -- start start 2024-01-31T05:24:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f4690575740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c09219980 0x562c0922c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f4690575740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921eb40 0x562c09228a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f4690575740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 0x562c0922a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f4690575740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c092150b0 con 0x562c0921f460 2024-01-31T05:24:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f4690575740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c0923aee0 con 0x562c09219980 2024-01-31T05:24:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f4690575740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09243e50 con 0x562c0921eb40 2024-01-31T05:24:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f46877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 0x562c0922a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f46877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 0x562c0922a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60614/0 (socket says 172.21.15.45:60614) 2024-01-31T05:24:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f46867fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921eb40 0x562c09228a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f46877fe640 1 -- 172.21.15.45:0/1078168583 learned_addr learned my addr 172.21.15.45:0/1078168583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f4686ffd640 1 --2- 172.21.15.45:0/1078168583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c09219980 0x562c0922c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f46867fc640 1 -- 172.21.15.45:0/1078168583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c09219980 msgr2=0x562c0922c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f46867fc640 1 --2- 172.21.15.45:0/1078168583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c09219980 0x562c0922c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f46867fc640 1 -- 172.21.15.45:0/1078168583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 msgr2=0x562c0922a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f46867fc640 1 --2- 172.21.15.45:0/1078168583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 0x562c0922a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f46867fc640 1 -- 172.21.15.45:0/1078168583 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c09122030 con 0x562c0921eb40 2024-01-31T05:24:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.795+0000 7f46867fc640 1 --2- 172.21.15.45:0/1078168583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921eb40 0x562c09228a20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f467c00e830 tx=0x7f467c005670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207509a445c459ff server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.796+0000 7f466cff9640 1 -- 172.21.15.45:0/1078168583 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c015070 con 0x562c0921eb40 2024-01-31T05:24:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.796+0000 7f466cff9640 1 -- 172.21.15.45:0/1078168583 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f467c016040 con 0x562c0921eb40 2024-01-31T05:24:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.796+0000 7f466cff9640 1 -- 172.21.15.45:0/1078168583 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c022410 con 0x562c0921eb40 2024-01-31T05:24:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.796+0000 7f4690575740 1 -- 172.21.15.45:0/1078168583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921eb40 msgr2=0x562c09228a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.796+0000 7f4690575740 1 --2- 172.21.15.45:0/1078168583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921eb40 0x562c09228a20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f467c00e830 tx=0x7f467c005670 comp rx=0 tx=0).stop 2024-01-31T05:24:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.796+0000 7f4690575740 1 -- 172.21.15.45:0/1078168583 shutdown_connections 2024-01-31T05:24:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.796+0000 7f4690575740 1 --2- 172.21.15.45:0/1078168583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921f460 0x562c0922a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.796+0000 7f4690575740 1 --2- 172.21.15.45:0/1078168583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921eb40 0x562c09228a20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.796+0000 7f4690575740 1 --2- 172.21.15.45:0/1078168583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c09219980 0x562c0922c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.796+0000 7f4690575740 1 -- 172.21.15.45:0/1078168583 >> 172.21.15.45:0/1078168583 conn(0x562c0921e6b0 msgr2=0x562c09228310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.796+0000 7f4690575740 1 -- 172.21.15.45:0/1078168583 shutdown_connections 2024-01-31T05:24:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.796+0000 7f4690575740 1 -- 172.21.15.45:0/1078168583 wait complete. 2024-01-31T05:24:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f4690575740 1 -- start start 2024-01-31T05:24:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f4690575740 1 Processor -- start 2024-01-31T05:24:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f4690575740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c09219980 0x562c09232040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f4690575740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 0x562c09232580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f4690575740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921f460 0x562c09232ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f4690575740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09292ce0 con 0x562c0921eb40 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f4690575740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09292e20 con 0x562c09219980 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f4690575740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09292f60 con 0x562c0921f460 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f4690575740 10 client.0 _ll_register_callbacks cb 0x7f4678005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f46867fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 0x562c09232580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f46867fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 0x562c09232580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60626/0 (socket says 172.21.15.45:60626) 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f46867fc640 1 -- 172.21.15.45:0/2205609124 learned_addr learned my addr 172.21.15.45:0/2205609124 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f46877fe640 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921f460 0x562c09232ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f4686ffd640 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c09219980 0x562c09232040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f46867fc640 1 -- 172.21.15.45:0/2205609124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921f460 msgr2=0x562c09232ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f46867fc640 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921f460 0x562c09232ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f46867fc640 1 -- 172.21.15.45:0/2205609124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c09219980 msgr2=0x562c09232040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f46867fc640 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c09219980 0x562c09232040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.797+0000 7f46867fc640 1 -- 172.21.15.45:0/2205609124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c091856a0 con 0x562c0921eb40 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f46867fc640 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 0x562c09232580 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f467c00e980 tx=0x7f467c00eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b618151775c3e744 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f466aff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f466aff5640 1 -- 172.21.15.45:0/2205609124 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c015070 con 0x562c0921eb40 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f466aff5640 1 -- 172.21.15.45:0/2205609124 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f467c00e5f0 con 0x562c0921eb40 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f4690575740 1 -- 172.21.15.45:0/2205609124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46780c0660 con 0x562c0921eb40 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f466aff5640 1 -- 172.21.15.45:0/2205609124 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f467c03d460 con 0x562c0921eb40 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f4690575740 10 client.15519 fetch_fsmap learned FSMap version 23 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f4690575740 1 -- 172.21.15.45:0/2205609124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562c09147a50 con 0x562c0921eb40 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f466aff5640 1 -- 172.21.15.45:0/2205609124 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f467c041020 con 0x562c0921eb40 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f466aff5640 1 -- 172.21.15.45:0/2205609124 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f467c033620 con 0x562c0921eb40 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f4690575740 10 client.15519 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f4690575740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f4690575740 20 client.15519 populate_metadata read hostname 'smithi045' 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f4690575740 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562c0922cb60 0x562c092963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f4690575740 4 client.15519 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:24:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.798+0000 7f4690575740 1 -- 172.21.15.45:0/2205609124 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562c08e86600 con 0x562c0922cb60 2024-01-31T05:24:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.799+0000 7f4686ffd640 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562c0922cb60 0x562c092963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.799+0000 7f466aff5640 10 client.15519 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.799+0000 7f4686ffd640 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562c0922cb60 0x562c092963c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b07dd659692f7ee server_cookie=15797add5efd3435 in_seq=0 out_seq=0 2024-01-31T05:24:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f466aff5640 1 -- 172.21.15.45:0/2205609124 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562c08e86600 con 0x562c0922cb60 2024-01-31T05:24:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f466aff5640 10 client.15519 handle_command_reply: tid=0 2024-01-31T05:24:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f4690575740 1 client.15519 shutdown 2024-01-31T05:24:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f4690575740 1 -- 172.21.15.45:0/2205609124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 msgr2=0x562c09232580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f4690575740 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 0x562c09232580 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f467c00e980 tx=0x7f467c00eb40 comp rx=0 tx=0).stop 2024-01-31T05:24:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f4690575740 1 -- 172.21.15.45:0/2205609124 shutdown_connections 2024-01-31T05:24:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f4690575740 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562c0922cb60 0x562c092963c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f4690575740 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c0921eb40 0x562c09232580 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f4690575740 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562c0921f460 0x562c09232ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f4690575740 1 --2- 172.21.15.45:0/2205609124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c09219980 0x562c09232040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f466aff5640 0 client.15519 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f4690575740 1 -- 172.21.15.45:0/2205609124 >> 172.21.15.45:0/2205609124 conn(0x562c0921e6b0 msgr2=0x562c0921add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f4690575740 1 -- 172.21.15.45:0/2205609124 shutdown_connections 2024-01-31T05:24:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f4690575740 1 -- 172.21.15.45:0/2205609124 wait complete. 2024-01-31T05:24:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f4690575740 20 client.15519 trim_cache size 0 max 16384 2024-01-31T05:24:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:24:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:24:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:24:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:24:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f468d7ae640 1 -- 172.21.15.45:0/123284775 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f46700a14f0 msgr2=0x7f46700a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f468d7ae640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f46700a14f0 0x7f46700a39b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4680005fd0 tx=0x7f4680010740 comp rx=0 tx=0).stop 2024-01-31T05:24:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f468d7ae640 1 -- 172.21.15.45:0/123284775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4688100f30 msgr2=0x7f4688197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f468d7ae640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4688100f30 0x7f4688197200 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4678005940 tx=0x7f4678004dd0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f468d7ae640 1 -- 172.21.15.45:0/123284775 shutdown_connections 2024-01-31T05:24:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f468d7ae640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f46700a14f0 0x7f46700a39b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f468d7ae640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f468819bac0 0x7f468819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f468d7ae640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4688106000 0x7f4688197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f468d7ae640 1 --2- 172.21.15.45:0/123284775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4688100f30 0x7f4688197200 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f468d7ae640 1 -- 172.21.15.45:0/123284775 >> 172.21.15.45:0/123284775 conn(0x7f46880fc7d0 msgr2=0x7f46880feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f468d7ae640 1 -- 172.21.15.45:0/123284775 shutdown_connections 2024-01-31T05:24:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:15.838+0000 7f468d7ae640 1 -- 172.21.15.45:0/123284775 wait complete. 2024-01-31T05:24:16.771 INFO:tasks.workunit.client.0.smithi045.stdout: 1 21491 8.40 MB/sec warmup 16 sec latency 3539.266 ms 2024-01-31T05:24:16.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:16 smithi186 ceph-mon[35659]: pgmap v50: 97 pgs: 97 active+clean; 11 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 762 KiB/s wr, 146 op/s 2024-01-31T05:24:16.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:16 smithi045 ceph-mon[31585]: pgmap v50: 97 pgs: 97 active+clean; 11 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 762 KiB/s wr, 146 op/s 2024-01-31T05:24:16.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:16 smithi156 ceph-mon[35720]: pgmap v50: 97 pgs: 97 active+clean; 11 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 762 KiB/s wr, 146 op/s 2024-01-31T05:24:17.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 23815 8.44 MB/sec warmup 17 sec latency 39.303 ms 2024-01-31T05:24:18.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 23815 7.97 MB/sec warmup 18 sec latency 1039.397 ms 2024-01-31T05:24:18.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:18 smithi186 ceph-mon[35659]: pgmap v51: 97 pgs: 97 active+clean; 8.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 954 KiB/s wr, 194 op/s 2024-01-31T05:24:18.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:18 smithi045 ceph-mon[31585]: pgmap v51: 97 pgs: 97 active+clean; 8.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 954 KiB/s wr, 194 op/s 2024-01-31T05:24:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:18 smithi156 ceph-mon[35720]: pgmap v51: 97 pgs: 97 active+clean; 8.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 954 KiB/s wr, 194 op/s 2024-01-31T05:24:19.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 23815 7.55 MB/sec warmup 19 sec latency 2039.500 ms 2024-01-31T05:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 23815 7.17 MB/sec warmup 20 sec latency 3039.607 ms 2024-01-31T05:24:20.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:20 smithi186 ceph-mon[35659]: pgmap v52: 97 pgs: 97 active+clean; 8.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 355 KiB/s wr, 72 op/s 2024-01-31T05:24:20.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:20 smithi045 ceph-mon[31585]: pgmap v52: 97 pgs: 97 active+clean; 8.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 355 KiB/s wr, 72 op/s 2024-01-31T05:24:20.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:20 smithi156 ceph-mon[35720]: pgmap v52: 97 pgs: 97 active+clean; 8.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 355 KiB/s wr, 72 op/s 2024-01-31T05:24:21.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 25216 7.37 MB/sec warmup 21 sec latency 3500.274 ms 2024-01-31T05:24:22.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 27769 7.58 MB/sec warmup 22 sec latency 44.192 ms 2024-01-31T05:24:22.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:22 smithi186 ceph-mon[35659]: pgmap v53: 97 pgs: 97 active+clean; 8.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 355 KiB/s wr, 72 op/s 2024-01-31T05:24:22.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:22 smithi045 ceph-mon[31585]: pgmap v53: 97 pgs: 97 active+clean; 8.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 355 KiB/s wr, 72 op/s 2024-01-31T05:24:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:22 smithi156 ceph-mon[35720]: pgmap v53: 97 pgs: 97 active+clean; 8.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 355 KiB/s wr, 72 op/s 2024-01-31T05:24:23.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 30008 7.79 MB/sec warmup 23 sec latency 7.168 ms 2024-01-31T05:24:24.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32719 8.06 MB/sec warmup 24 sec latency 6.561 ms 2024-01-31T05:24:24.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:24 smithi186 ceph-mon[35659]: pgmap v54: 97 pgs: 97 active+clean; 11 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 540 KiB/s wr, 98 op/s 2024-01-31T05:24:24.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:24 smithi045 ceph-mon[31585]: pgmap v54: 97 pgs: 97 active+clean; 11 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 540 KiB/s wr, 98 op/s 2024-01-31T05:24:24.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:24 smithi156 ceph-mon[35720]: pgmap v54: 97 pgs: 97 active+clean; 11 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 540 KiB/s wr, 98 op/s 2024-01-31T05:24:25.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 35794 8.48 MB/sec warmup 25 sec latency 15.899 ms 2024-01-31T05:24:26.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 38680 8.67 MB/sec warmup 26 sec latency 7.803 ms 2024-01-31T05:24:26.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:26 smithi186 ceph-mon[35659]: pgmap v55: 97 pgs: 97 active+clean; 11 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 376 KiB/s wr, 74 op/s 2024-01-31T05:24:26.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:26 smithi045 ceph-mon[31585]: pgmap v55: 97 pgs: 97 active+clean; 11 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 376 KiB/s wr, 74 op/s 2024-01-31T05:24:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:26 smithi156 ceph-mon[35720]: pgmap v55: 97 pgs: 97 active+clean; 11 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 376 KiB/s wr, 74 op/s 2024-01-31T05:24:27.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 41460 8.83 MB/sec warmup 27 sec latency 6.157 ms 2024-01-31T05:24:28.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 43666 9.01 MB/sec warmup 28 sec latency 11.024 ms 2024-01-31T05:24:28.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:28 smithi186 ceph-mon[35659]: pgmap v56: 97 pgs: 97 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1019 KiB/s wr, 157 op/s 2024-01-31T05:24:28.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:28 smithi045 ceph-mon[31585]: pgmap v56: 97 pgs: 97 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1019 KiB/s wr, 157 op/s 2024-01-31T05:24:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.928+0000 7fe5feec5640 1 -- 172.21.15.45:0/2281686066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f81018c0 msgr2=0x7fe5f8105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.928+0000 7fe5feec5640 1 --2- 172.21.15.45:0/2281686066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f81018c0 0x7fe5f8105900 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009930 tx=0x7fe5ec02bc30 comp rx=0 tx=0).stop 2024-01-31T05:24:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.928+0000 7fe5feec5640 1 -- 172.21.15.45:0/2281686066 shutdown_connections 2024-01-31T05:24:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.928+0000 7fe5feec5640 1 --2- 172.21.15.45:0/2281686066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f81018c0 0x7fe5f8105900 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.928+0000 7fe5feec5640 1 --2- 172.21.15.45:0/2281686066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f8105fe0 0x7fe5f810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.928+0000 7fe5feec5640 1 --2- 172.21.15.45:0/2281686066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f8100f10 0x7fe5f81012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.928+0000 7fe5feec5640 1 -- 172.21.15.45:0/2281686066 >> 172.21.15.45:0/2281686066 conn(0x7fe5f80fc7b0 msgr2=0x7fe5f80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.928+0000 7fe5feec5640 1 -- 172.21.15.45:0/2281686066 shutdown_connections 2024-01-31T05:24:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.928+0000 7fe5feec5640 1 -- 172.21.15.45:0/2281686066 wait complete. 2024-01-31T05:24:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5feec5640 1 Processor -- start 2024-01-31T05:24:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5feec5640 1 -- start start 2024-01-31T05:24:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5feec5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f8100f10 0x7fe5f8197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5feec5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f8105fe0 0x7fe5f8197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5feec5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f819bb00 0x7fe5f819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5feec5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f819c4f0 con 0x7fe5f8105fe0 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5feec5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f819c660 con 0x7fe5f8100f10 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5feec5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f819c7d0 con 0x7fe5f819bb00 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5f7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f8105fe0 0x7fe5f8197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5f7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f8105fe0 0x7fe5f8197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47464/0 (socket says 172.21.15.45:47464) 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5f7fff640 1 -- 172.21.15.45:0/1104270280 learned_addr learned my addr 172.21.15.45:0/1104270280 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5fd462640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f819bb00 0x7fe5f819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5f7fff640 1 -- 172.21.15.45:0/1104270280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f819bb00 msgr2=0x7fe5f819bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5f7fff640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f819bb00 0x7fe5f819bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5f7fff640 1 -- 172.21.15.45:0/1104270280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f8100f10 msgr2=0x7fe5f8197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5f7fff640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f8100f10 0x7fe5f8197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.929+0000 7fe5f7fff640 1 -- 172.21.15.45:0/1104270280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec009560 con 0x7fe5f8105fe0 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.930+0000 7fe5f7fff640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f8105fe0 0x7fe5f8197780 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009230 tx=0x7fe5ec02cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.930+0000 7fe5f5ffb640 1 -- 172.21.15.45:0/1104270280 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec004250 con 0x7fe5f8105fe0 2024-01-31T05:24:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.930+0000 7fe5f5ffb640 1 -- 172.21.15.45:0/1104270280 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5ec0043b0 con 0x7fe5f8105fe0 2024-01-31T05:24:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.930+0000 7fe5f5ffb640 1 -- 172.21.15.45:0/1104270280 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec003cc0 con 0x7fe5f8105fe0 2024-01-31T05:24:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.930+0000 7fe5feec5640 1 -- 172.21.15.45:0/1104270280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f81a3db0 con 0x7fe5f8105fe0 2024-01-31T05:24:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.930+0000 7fe5feec5640 1 -- 172.21.15.45:0/1104270280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f81a4180 con 0x7fe5f8105fe0 2024-01-31T05:24:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.932+0000 7fe5f5ffb640 1 -- 172.21.15.45:0/1104270280 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fe5ec005000 con 0x7fe5f8105fe0 2024-01-31T05:24:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.932+0000 7fe601c82740 1 Processor -- start 2024-01-31T05:24:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.932+0000 7fe601c82740 1 -- start start 2024-01-31T05:24:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe601c82740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f4447bb80 0x558f4447bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe601c82740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447c4a0 0x558f44489620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe601c82740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 0x558f4448bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe601c82740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f443eb040 con 0x558f44489b60 2024-01-31T05:24:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe601c82740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f443e8730 con 0x558f4447bb80 2024-01-31T05:24:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe601c82740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f443e4020 con 0x558f4447c4a0 2024-01-31T05:24:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5f7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 0x558f4448bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5fcc61640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447c4a0 0x558f44489620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5f7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 0x558f4448bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47480/0 (socket says 172.21.15.45:47480) 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5f7fff640 1 -- 172.21.15.45:0/3596615266 learned_addr learned my addr 172.21.15.45:0/3596615266 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5fd462640 1 --2- 172.21.15.45:0/3596615266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f4447bb80 0x558f4447bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5f5ffb640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe5e00a1570 0x7fe5e00a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5fd462640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe5e00a1570 0x7fe5e00a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5fcc61640 1 -- 172.21.15.45:0/3596615266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f4447bb80 msgr2=0x558f4447bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5fcc61640 1 --2- 172.21.15.45:0/3596615266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f4447bb80 0x558f4447bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5f5ffb640 1 -- 172.21.15.45:0/1104270280 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe5ec0ef680 con 0x7fe5f8105fe0 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5fcc61640 1 -- 172.21.15.45:0/3596615266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 msgr2=0x558f4448bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5fcc61640 1 --2- 172.21.15.45:0/3596615266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 0x558f4448bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5fcc61640 1 -- 172.21.15.45:0/3596615266 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f443dbe40 con 0x558f4447c4a0 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.933+0000 7fe5fcc61640 1 --2- 172.21.15.45:0/3596615266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447c4a0 0x558f44489620 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009b40 tx=0x7fe5e800e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36f2be8999ac2be8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe5fd462640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe5e00a1570 0x7fe5e00a3a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00094d0 tx=0x7fe5f0004ab0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe5deffd640 1 -- 172.21.15.45:0/3596615266 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800ed80 con 0x558f4447c4a0 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe5deffd640 1 -- 172.21.15.45:0/3596615266 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e80044f0 con 0x558f4447c4a0 2024-01-31T05:24:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe5deffd640 1 -- 172.21.15.45:0/3596615266 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8004a00 con 0x558f4447c4a0 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe601c82740 1 -- 172.21.15.45:0/3596615266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447c4a0 msgr2=0x558f44489620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe601c82740 1 --2- 172.21.15.45:0/3596615266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447c4a0 0x558f44489620 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009b40 tx=0x7fe5e800e350 comp rx=0 tx=0).stop 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe601c82740 1 -- 172.21.15.45:0/3596615266 shutdown_connections 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe601c82740 1 --2- 172.21.15.45:0/3596615266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 0x558f4448bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe601c82740 1 --2- 172.21.15.45:0/3596615266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447c4a0 0x558f44489620 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe601c82740 1 --2- 172.21.15.45:0/3596615266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f4447bb80 0x558f4447bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe601c82740 1 -- 172.21.15.45:0/3596615266 >> 172.21.15.45:0/3596615266 conn(0x558f44476980 msgr2=0x558f44476d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe601c82740 1 -- 172.21.15.45:0/3596615266 shutdown_connections 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.934+0000 7fe601c82740 1 -- 172.21.15.45:0/3596615266 wait complete. 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe601c82740 1 -- start start 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe601c82740 1 Processor -- start 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe601c82740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447bb80 0x558f44486a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe601c82740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447c4a0 0x558f44486fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe601c82740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f44489b60 0x558f444874f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe601c82740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f4439d1b0 con 0x558f4447c4a0 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe601c82740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f4439ce50 con 0x558f44489b60 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe601c82740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f44362090 con 0x558f4447bb80 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe601c82740 10 client.0 _ll_register_callbacks cb 0x558f44387a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe5f7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447c4a0 0x558f44486fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe5fcc61640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447bb80 0x558f44486a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe5f7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447c4a0 0x558f44486fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47496/0 (socket says 172.21.15.45:47496) 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe5f7fff640 1 -- 172.21.15.45:0/1215146304 learned_addr learned my addr 172.21.15.45:0/1215146304 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.935+0000 7fe5fd462640 1 --2- 172.21.15.45:0/1215146304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f44489b60 0x558f444874f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe5f7fff640 1 -- 172.21.15.45:0/1215146304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447bb80 msgr2=0x558f44486a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe5f7fff640 1 --2- 172.21.15.45:0/1215146304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447bb80 0x558f44486a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe5f7fff640 1 -- 172.21.15.45:0/1215146304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f44489b60 msgr2=0x558f444874f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe5f7fff640 1 --2- 172.21.15.45:0/1215146304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f44489b60 0x558f444874f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe5f7fff640 1 -- 172.21.15.45:0/1215146304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e80097f0 con 0x558f4447c4a0 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe5f7fff640 1 --2- 172.21.15.45:0/1215146304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447c4a0 0x558f44486fb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec000c00 tx=0x7fe5ec0ecef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9529372d3528a033 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe5d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe5d7fff640 1 -- 172.21.15.45:0/1215146304 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec0bbee0 con 0x558f4447c4a0 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe5d7fff640 1 -- 172.21.15.45:0/1215146304 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe5ec03ce30 con 0x558f4447c4a0 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe601c82740 1 -- 172.21.15.45:0/1215146304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f443dd010 con 0x558f4447c4a0 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe5d7fff640 1 -- 172.21.15.45:0/1215146304 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5ec0d4ea0 con 0x558f4447c4a0 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe601c82740 10 client.15531 fetch_fsmap learned FSMap version 23 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe601c82740 1 -- 172.21.15.45:0/1215146304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558f443e26a0 con 0x558f4447c4a0 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe5d7fff640 1 -- 172.21.15.45:0/1215146304 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f443dd010 con 0x558f4447c4a0 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe5d7fff640 1 -- 172.21.15.45:0/1215146304 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe5ec03c070 con 0x558f4447c4a0 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe601c82740 10 client.15531 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe601c82740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe601c82740 20 client.15531 populate_metadata read hostname 'smithi045' 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe601c82740 1 --2- 172.21.15.45:0/1215146304 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558f444997b0 0x558f4449bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe601c82740 4 client.15531 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.936+0000 7fe601c82740 1 -- 172.21.15.45:0/1215146304 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f443dcd30 con 0x558f444997b0 2024-01-31T05:24:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.937+0000 7fe5fcc61640 1 --2- 172.21.15.45:0/1215146304 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558f444997b0 0x558f4449bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.937+0000 7fe5d7fff640 10 client.15531 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.937+0000 7fe5fcc61640 1 --2- 172.21.15.45:0/1215146304 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558f444997b0 0x558f4449bbd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2dd82c44e0567b server_cookie=77723afe7b370bd5 in_seq=0 out_seq=0 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe5d7fff640 1 -- 172.21.15.45:0/1215146304 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f443dcd30 con 0x558f444997b0 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe5d7fff640 10 client.15531 handle_command_reply: tid=0 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe601c82740 1 client.15531 shutdown 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe601c82740 1 -- 172.21.15.45:0/1215146304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447c4a0 msgr2=0x558f44486fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe601c82740 1 --2- 172.21.15.45:0/1215146304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447c4a0 0x558f44486fb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec000c00 tx=0x7fe5ec0ecef0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe601c82740 1 -- 172.21.15.45:0/1215146304 shutdown_connections 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe601c82740 1 --2- 172.21.15.45:0/1215146304 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558f444997b0 0x558f4449bbd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe601c82740 1 --2- 172.21.15.45:0/1215146304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447c4a0 0x558f44486fb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe5d7fff640 0 client.15531 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe601c82740 1 --2- 172.21.15.45:0/1215146304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f44489b60 0x558f444874f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe601c82740 1 --2- 172.21.15.45:0/1215146304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447bb80 0x558f44486a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe601c82740 1 -- 172.21.15.45:0/1215146304 >> 172.21.15.45:0/1215146304 conn(0x558f44476980 msgr2=0x558f444786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe601c82740 1 -- 172.21.15.45:0/1215146304 shutdown_connections 2024-01-31T05:24:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe601c82740 1 -- 172.21.15.45:0/1215146304 wait complete. 2024-01-31T05:24:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.938+0000 7fe601c82740 20 client.15531 trim_cache size 0 max 16384 2024-01-31T05:24:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe601c82740 1 Processor -- start 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe601c82740 1 -- start start 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe601c82740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447bb80 0x558f44489130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe601c82740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f4447c4a0 0x558f44485a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe601c82740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 0x558f44483cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe601c82740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f443dcd30 con 0x558f44489b60 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe601c82740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f444722b0 con 0x558f4447c4a0 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe601c82740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f4448bf50 con 0x558f4447bb80 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe5fd462640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 0x558f44483cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe5fd462640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 0x558f44483cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47506/0 (socket says 172.21.15.45:47506) 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe5fd462640 1 -- 172.21.15.45:0/1653556367 learned_addr learned my addr 172.21.15.45:0/1653556367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe5fcc61640 1 --2- 172.21.15.45:0/1653556367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447bb80 0x558f44489130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe5f7fff640 1 --2- 172.21.15.45:0/1653556367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f4447c4a0 0x558f44485a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe5fd462640 1 -- 172.21.15.45:0/1653556367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447bb80 msgr2=0x558f44489130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe5fd462640 1 --2- 172.21.15.45:0/1653556367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447bb80 0x558f44489130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe5fd462640 1 -- 172.21.15.45:0/1653556367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f4447c4a0 msgr2=0x558f44485a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe5fd462640 1 --2- 172.21.15.45:0/1653556367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f4447c4a0 0x558f44485a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe5fd462640 1 -- 172.21.15.45:0/1653556367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f4437f030 con 0x558f44489b60 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.952+0000 7fe5fd462640 1 --2- 172.21.15.45:0/1653556367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 0x558f44483cd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0012f90 tx=0x7fe5f0009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d21d5605bf8c0fd server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.953+0000 7fe5de7fc640 1 -- 172.21.15.45:0/1653556367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f000bd30 con 0x558f44489b60 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.953+0000 7fe5de7fc640 1 -- 172.21.15.45:0/1653556367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5f000be90 con 0x558f44489b60 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.953+0000 7fe5de7fc640 1 -- 172.21.15.45:0/1653556367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00236d0 con 0x558f44489b60 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.953+0000 7fe601c82740 1 -- 172.21.15.45:0/1653556367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 msgr2=0x558f44483cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.953+0000 7fe601c82740 1 --2- 172.21.15.45:0/1653556367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 0x558f44483cd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0012f90 tx=0x7fe5f0009a70 comp rx=0 tx=0).stop 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.953+0000 7fe601c82740 1 -- 172.21.15.45:0/1653556367 shutdown_connections 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.953+0000 7fe601c82740 1 --2- 172.21.15.45:0/1653556367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f44489b60 0x558f44483cd0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.953+0000 7fe601c82740 1 --2- 172.21.15.45:0/1653556367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f4447c4a0 0x558f44485a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.953+0000 7fe601c82740 1 --2- 172.21.15.45:0/1653556367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447bb80 0x558f44489130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.953+0000 7fe601c82740 1 -- 172.21.15.45:0/1653556367 >> 172.21.15.45:0/1653556367 conn(0x558f44476980 msgr2=0x558f44485340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.953+0000 7fe601c82740 1 -- 172.21.15.45:0/1653556367 shutdown_connections 2024-01-31T05:24:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.953+0000 7fe601c82740 1 -- 172.21.15.45:0/1653556367 wait complete. 2024-01-31T05:24:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe601c82740 1 -- start start 2024-01-31T05:24:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe601c82740 1 Processor -- start 2024-01-31T05:24:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe601c82740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447bb80 0x558f4448fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe601c82740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447c4a0 0x558f44490100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe601c82740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f44489b60 0x558f444efc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe601c82740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f444f01c0 con 0x558f4447bb80 2024-01-31T05:24:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe601c82740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f444f0300 con 0x558f44489b60 2024-01-31T05:24:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe601c82740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f444f0470 con 0x558f4447c4a0 2024-01-31T05:24:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe601c82740 10 client.0 _ll_register_callbacks cb 0x7fe5e8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:24:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe5fcc61640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447bb80 0x558f4448fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe5fcc61640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447bb80 0x558f4448fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47512/0 (socket says 172.21.15.45:47512) 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe5fcc61640 1 -- 172.21.15.45:0/4077277711 learned_addr learned my addr 172.21.15.45:0/4077277711 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe5f7fff640 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447c4a0 0x558f44490100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe5fd462640 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f44489b60 0x558f444efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe5fcc61640 1 -- 172.21.15.45:0/4077277711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447c4a0 msgr2=0x558f44490100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe5fcc61640 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447c4a0 0x558f44490100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe5fcc61640 1 -- 172.21.15.45:0/4077277711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f44489b60 msgr2=0x558f444efc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.954+0000 7fe5fcc61640 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f44489b60 0x558f444efc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe5fcc61640 1 -- 172.21.15.45:0/4077277711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f001a040 con 0x558f4447bb80 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe5fcc61640 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447bb80 0x558f4448fbc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x558f4447d160 tx=0x7fe5e800e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b20d7f28600d742 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe5d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe5d7fff640 1 -- 172.21.15.45:0/4077277711 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e801f040 con 0x558f4447bb80 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe5d7fff640 1 -- 172.21.15.45:0/4077277711 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe5e8014070 con 0x558f4447bb80 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe601c82740 1 -- 172.21.15.45:0/4077277711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5e80044f0 con 0x558f4447bb80 2024-01-31T05:24:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe5d7fff640 1 -- 172.21.15.45:0/4077277711 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5e8027070 con 0x558f4447bb80 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe601c82740 10 client.15537 fetch_fsmap learned FSMap version 23 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe601c82740 1 -- 172.21.15.45:0/4077277711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558f443a4a50 con 0x558f4447bb80 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe5d7fff640 1 -- 172.21.15.45:0/4077277711 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e80044f0 con 0x558f4447bb80 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe5d7fff640 1 -- 172.21.15.45:0/4077277711 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe5e8018710 con 0x558f4447bb80 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe601c82740 10 client.15537 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe601c82740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe601c82740 20 client.15537 populate_metadata read hostname 'smithi045' 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe601c82740 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558f444f1ce0 0x558f444f4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe601c82740 4 client.15537 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.955+0000 7fe601c82740 1 -- 172.21.15.45:0/4077277711 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558f440e3600 con 0x558f444f1ce0 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.956+0000 7fe5f7fff640 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558f444f1ce0 0x558f444f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.956+0000 7fe5f7fff640 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558f444f1ce0 0x558f444f4100 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9c18533f91d8ce2 server_cookie=2770c2b59a3bd4b8 in_seq=0 out_seq=0 2024-01-31T05:24:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.956+0000 7fe5d7fff640 10 client.15537 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe5d7fff640 1 -- 172.21.15.45:0/4077277711 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558f440e3600 con 0x558f444f1ce0 2024-01-31T05:24:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe5d7fff640 10 client.15537 handle_command_reply: tid=0 2024-01-31T05:24:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe601c82740 1 client.15537 shutdown 2024-01-31T05:24:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe601c82740 1 -- 172.21.15.45:0/4077277711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447bb80 msgr2=0x558f4448fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe601c82740 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447bb80 0x558f4448fbc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x558f4447d160 tx=0x7fe5e800e910 comp rx=0 tx=0).stop 2024-01-31T05:24:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe601c82740 1 -- 172.21.15.45:0/4077277711 shutdown_connections 2024-01-31T05:24:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe601c82740 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558f444f1ce0 0x558f444f4100 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe601c82740 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f44489b60 0x558f444efc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe601c82740 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f4447c4a0 0x558f44490100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe5d7fff640 0 client.15537 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe601c82740 1 --2- 172.21.15.45:0/4077277711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f4447bb80 0x558f4448fbc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe601c82740 1 -- 172.21.15.45:0/4077277711 >> 172.21.15.45:0/4077277711 conn(0x558f44476980 msgr2=0x558f44477d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe601c82740 1 -- 172.21.15.45:0/4077277711 shutdown_connections 2024-01-31T05:24:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.965+0000 7fe601c82740 1 -- 172.21.15.45:0/4077277711 wait complete. 2024-01-31T05:24:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe601c82740 20 client.15537 trim_cache size 0 max 16384 2024-01-31T05:24:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:24:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:24:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:24:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:24:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe5feec5640 1 -- 172.21.15.45:0/1104270280 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe5e00a1570 msgr2=0x7fe5e00a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe5feec5640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe5e00a1570 0x7fe5e00a3a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00094d0 tx=0x7fe5f0004ab0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe5feec5640 1 -- 172.21.15.45:0/1104270280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f8105fe0 msgr2=0x7fe5f8197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe5feec5640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f8105fe0 0x7fe5f8197780 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009230 tx=0x7fe5ec02cd20 comp rx=0 tx=0).stop 2024-01-31T05:24:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe5feec5640 1 -- 172.21.15.45:0/1104270280 shutdown_connections 2024-01-31T05:24:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe5feec5640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe5e00a1570 0x7fe5e00a3a30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe5feec5640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f8105fe0 0x7fe5f8197780 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe5feec5640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f819bb00 0x7fe5f819bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe5feec5640 1 --2- 172.21.15.45:0/1104270280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f8100f10 0x7fe5f8197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe5feec5640 1 -- 172.21.15.45:0/1104270280 >> 172.21.15.45:0/1104270280 conn(0x7fe5f80fc7b0 msgr2=0x7fe5f80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe5feec5640 1 -- 172.21.15.45:0/1104270280 shutdown_connections 2024-01-31T05:24:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:28.966+0000 7fe5feec5640 1 -- 172.21.15.45:0/1104270280 wait complete. 2024-01-31T05:24:28.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:28 smithi156 ceph-mon[35720]: pgmap v56: 97 pgs: 97 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1019 KiB/s wr, 157 op/s 2024-01-31T05:24:29.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 46705 9.30 MB/sec warmup 29 sec latency 6.907 ms 2024-01-31T05:24:30.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 49928 9.61 MB/sec warmup 30 sec latency 6.841 ms 2024-01-31T05:24:30.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:30 smithi186 ceph-mon[35659]: pgmap v57: 97 pgs: 97 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 827 KiB/s wr, 109 op/s 2024-01-31T05:24:30.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:30 smithi045 ceph-mon[31585]: pgmap v57: 97 pgs: 97 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 827 KiB/s wr, 109 op/s 2024-01-31T05:24:30.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:30 smithi156 ceph-mon[35720]: pgmap v57: 97 pgs: 97 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 827 KiB/s wr, 109 op/s 2024-01-31T05:24:31.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 52286 9.57 MB/sec warmup 31 sec latency 33.198 ms 2024-01-31T05:24:32.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 54520 9.69 MB/sec warmup 32 sec latency 11.339 ms 2024-01-31T05:24:32.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:32 smithi186 ceph-mon[35659]: pgmap v58: 97 pgs: 97 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 827 KiB/s wr, 109 op/s 2024-01-31T05:24:32.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:32 smithi045 ceph-mon[31585]: pgmap v58: 97 pgs: 97 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 827 KiB/s wr, 109 op/s 2024-01-31T05:24:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:32 smithi156 ceph-mon[35720]: pgmap v58: 97 pgs: 97 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 827 KiB/s wr, 109 op/s 2024-01-31T05:24:33.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 57283 9.91 MB/sec warmup 33 sec latency 31.089 ms 2024-01-31T05:24:34.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 60439 10.14 MB/sec warmup 34 sec latency 6.952 ms 2024-01-31T05:24:34.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:34 smithi186 ceph-mon[35659]: pgmap v59: 97 pgs: 97 active+clean; 43 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 3.1 MiB/s wr, 201 op/s 2024-01-31T05:24:34.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:34 smithi045 ceph-mon[31585]: pgmap v59: 97 pgs: 97 active+clean; 43 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 3.1 MiB/s wr, 201 op/s 2024-01-31T05:24:34.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:34 smithi156 ceph-mon[35720]: pgmap v59: 97 pgs: 97 active+clean; 43 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 3.1 MiB/s wr, 201 op/s 2024-01-31T05:24:35.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 62820 10.11 MB/sec warmup 35 sec latency 17.341 ms 2024-01-31T05:24:36.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 64850 10.17 MB/sec warmup 36 sec latency 38.192 ms 2024-01-31T05:24:36.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:36 smithi186 ceph-mon[35659]: pgmap v60: 97 pgs: 97 active+clean; 43 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 2.9 MiB/s wr, 175 op/s 2024-01-31T05:24:36.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:36 smithi045 ceph-mon[31585]: pgmap v60: 97 pgs: 97 active+clean; 43 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 2.9 MiB/s wr, 175 op/s 2024-01-31T05:24:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:36 smithi156 ceph-mon[35720]: pgmap v60: 97 pgs: 97 active+clean; 43 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 2.9 MiB/s wr, 175 op/s 2024-01-31T05:24:37.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 67780 10.39 MB/sec warmup 37 sec latency 26.696 ms 2024-01-31T05:24:38.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 70401 10.44 MB/sec warmup 38 sec latency 40.992 ms 2024-01-31T05:24:38.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:38 smithi186 ceph-mon[35659]: pgmap v61: 97 pgs: 97 active+clean; 60 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 289 op/s 2024-01-31T05:24:38.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:38 smithi045 ceph-mon[31585]: pgmap v61: 97 pgs: 97 active+clean; 60 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 289 op/s 2024-01-31T05:24:38.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:38 smithi156 ceph-mon[35720]: pgmap v61: 97 pgs: 97 active+clean; 60 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 289 op/s 2024-01-31T05:24:39.772 INFO:tasks.workunit.client.0.smithi045.stdout: 1 72451 10.47 MB/sec warmup 39 sec latency 70.444 ms 2024-01-31T05:24:40.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 75107 10.57 MB/sec warmup 40 sec latency 12.962 ms 2024-01-31T05:24:40.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:40 smithi186 ceph-mon[35659]: pgmap v62: 97 pgs: 97 active+clean; 60 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-01-31T05:24:40.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:40 smithi045 ceph-mon[31585]: pgmap v62: 97 pgs: 97 active+clean; 60 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-01-31T05:24:40.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:40 smithi156 ceph-mon[35720]: pgmap v62: 97 pgs: 97 active+clean; 60 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-01-31T05:24:41.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 78161 10.74 MB/sec warmup 41 sec latency 38.449 ms 2024-01-31T05:24:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.056+0000 7feff797c640 1 -- 172.21.15.45:0/1195428733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff0068d10 msgr2=0x7feff010db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.056+0000 7feff797c640 1 --2- 172.21.15.45:0/1195428733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff0068d10 0x7feff010db80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009ca0 tx=0x7fefe802c180 comp rx=0 tx=0).stop 2024-01-31T05:24:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.056+0000 7feff797c640 1 -- 172.21.15.45:0/1195428733 shutdown_connections 2024-01-31T05:24:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.056+0000 7feff797c640 1 --2- 172.21.15.45:0/1195428733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff0068d10 0x7feff010db80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.056+0000 7feff797c640 1 --2- 172.21.15.45:0/1195428733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feff0068370 0x7feff00687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.056+0000 7feff797c640 1 --2- 172.21.15.45:0/1195428733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feff0106de0 0x7feff01071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.056+0000 7feff797c640 1 -- 172.21.15.45:0/1195428733 >> 172.21.15.45:0/1195428733 conn(0x7feff0076a40 msgr2=0x7feff0076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.056+0000 7feff797c640 1 -- 172.21.15.45:0/1195428733 shutdown_connections 2024-01-31T05:24:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.056+0000 7feff797c640 1 -- 172.21.15.45:0/1195428733 wait complete. 2024-01-31T05:24:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff797c640 1 Processor -- start 2024-01-31T05:24:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff797c640 1 -- start start 2024-01-31T05:24:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff797c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff0068370 0x7feff0197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff797c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feff0068d10 0x7feff0197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff797c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feff0106de0 0x7feff019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff797c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff019bf70 con 0x7feff0068370 2024-01-31T05:24:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff797c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff019c0e0 con 0x7feff0068d10 2024-01-31T05:24:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff797c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff019c2a0 con 0x7feff0106de0 2024-01-31T05:24:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff5718640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff0068370 0x7feff0197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff5718640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff0068370 0x7feff0197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45756/0 (socket says 172.21.15.45:45756) 2024-01-31T05:24:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff5718640 1 -- 172.21.15.45:0/3490918330 learned_addr learned my addr 172.21.15.45:0/3490918330 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff4f17640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feff0068d10 0x7feff0197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.057+0000 7feff5f19640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feff0106de0 0x7feff019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.058+0000 7feff5718640 1 -- 172.21.15.45:0/3490918330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feff0106de0 msgr2=0x7feff019b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.058+0000 7feff5718640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feff0106de0 0x7feff019b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.058+0000 7feff5718640 1 -- 172.21.15.45:0/3490918330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feff0068d10 msgr2=0x7feff0197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.058+0000 7feff5718640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feff0068d10 0x7feff0197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.058+0000 7feff5718640 1 -- 172.21.15.45:0/3490918330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe8009900 con 0x7feff0068370 2024-01-31T05:24:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.058+0000 7feff5718640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff0068370 0x7feff0197100 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fefec00e9f0 tx=0x7fefec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.058+0000 7fefe67fc640 1 -- 172.21.15.45:0/3490918330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec009700 con 0x7feff0068370 2024-01-31T05:24:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.058+0000 7feff797c640 1 -- 172.21.15.45:0/3490918330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff019c580 con 0x7feff0068370 2024-01-31T05:24:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.058+0000 7fefe67fc640 1 -- 172.21.15.45:0/3490918330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefec00cd80 con 0x7feff0068370 2024-01-31T05:24:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.058+0000 7fefe67fc640 1 -- 172.21.15.45:0/3490918330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec018860 con 0x7feff0068370 2024-01-31T05:24:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.058+0000 7feff797c640 1 -- 172.21.15.45:0/3490918330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff01a3e00 con 0x7feff0068370 2024-01-31T05:24:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7fefe67fc640 1 -- 172.21.15.45:0/3490918330 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fefec0189c0 con 0x7feff0068370 2024-01-31T05:24:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feffa81a740 1 Processor -- start 2024-01-31T05:24:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feffa81a740 1 -- start start 2024-01-31T05:24:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feffa81a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfcd060 0x56226bfcd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feffa81a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd980 0x56226bfd6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feffa81a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfd6750 0x56226bfd8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feffa81a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226bf3c040 con 0x56226bfd6750 2024-01-31T05:24:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feffa81a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226bf39730 con 0x56226bfcd060 2024-01-31T05:24:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feffa81a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226bf35020 con 0x56226bfcd980 2024-01-31T05:24:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff5718640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd980 0x56226bfd6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff4f17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfd6750 0x56226bfd8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff4f17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfd6750 0x56226bfd8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45762/0 (socket says 172.21.15.45:45762) 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff4f17640 1 -- 172.21.15.45:0/159719348 learned_addr learned my addr 172.21.15.45:0/159719348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff5718640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd980 0x56226bfd6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:57790/0 (socket says 172.21.15.45:57790) 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7fefe67fc640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fefdc0a14f0 0x7fefdc0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff5f19640 1 --2- 172.21.15.45:0/159719348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfcd060 0x56226bfcd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff5f19640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fefdc0a14f0 0x7fefdc0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7fefe67fc640 1 -- 172.21.15.45:0/3490918330 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fefec014070 con 0x7feff0068370 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff4f17640 1 -- 172.21.15.45:0/159719348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd980 msgr2=0x56226bfd6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff4f17640 1 --2- 172.21.15.45:0/159719348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd980 0x56226bfd6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff4f17640 1 -- 172.21.15.45:0/159719348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfcd060 msgr2=0x56226bfcd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff4f17640 1 --2- 172.21.15.45:0/159719348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfcd060 0x56226bfcd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff4f17640 1 -- 172.21.15.45:0/159719348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56226bf2ce40 con 0x56226bfd6750 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff5f19640 1 --2- 172.21.15.45:0/159719348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfcd060 0x56226bfcd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.061+0000 7feff5718640 1 --2- 172.21.15.45:0/159719348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd980 0x56226bfd6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.062+0000 7feff4f17640 1 --2- 172.21.15.45:0/159719348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfd6750 0x56226bfd8b40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fefe000bd10 tx=0x7fefe00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e790c7d00625683 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.062+0000 7feff5f19640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fefdc0a14f0 0x7fefdc0a39b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fefe8004af0 tx=0x7fefe8005c50 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.062+0000 7fefdb7fe640 1 -- 172.21.15.45:0/159719348 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0005220 con 0x56226bfd6750 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.062+0000 7fefdb7fe640 1 -- 172.21.15.45:0/159719348 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefe0015070 con 0x56226bfd6750 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.062+0000 7fefdb7fe640 1 -- 172.21.15.45:0/159719348 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe00116a0 con 0x56226bfd6750 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 -- 172.21.15.45:0/159719348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfd6750 msgr2=0x56226bfd8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 --2- 172.21.15.45:0/159719348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfd6750 0x56226bfd8b40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fefe000bd10 tx=0x7fefe00107a0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 -- 172.21.15.45:0/159719348 shutdown_connections 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 --2- 172.21.15.45:0/159719348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfd6750 0x56226bfd8b40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 --2- 172.21.15.45:0/159719348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd980 0x56226bfd6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 --2- 172.21.15.45:0/159719348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfcd060 0x56226bfcd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 -- 172.21.15.45:0/159719348 >> 172.21.15.45:0/159719348 conn(0x56226bfc7980 msgr2=0x56226bfc7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 -- 172.21.15.45:0/159719348 shutdown_connections 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 -- 172.21.15.45:0/159719348 wait complete. 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 -- start start 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 Processor -- start 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd060 0x56226bfd36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 0x56226bfd3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfd6750 0x56226bfd4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226beee1b0 con 0x56226bfcd980 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226beede50 con 0x56226bfd6750 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226beb3090 con 0x56226bfcd060 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.063+0000 7feffa81a740 10 client.0 _ll_register_callbacks cb 0x56226bed8a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feff4f17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 0x56226bfd3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feff4f17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 0x56226bfd3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45764/0 (socket says 172.21.15.45:45764) 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feff5718640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd060 0x56226bfd36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feff4f17640 1 -- 172.21.15.45:0/1825935372 learned_addr learned my addr 172.21.15.45:0/1825935372 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feff5f19640 1 --2- 172.21.15.45:0/1825935372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfd6750 0x56226bfd4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feff4f17640 1 -- 172.21.15.45:0/1825935372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd060 msgr2=0x56226bfd36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feff4f17640 1 --2- 172.21.15.45:0/1825935372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd060 0x56226bfd36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feff4f17640 1 -- 172.21.15.45:0/1825935372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfd6750 msgr2=0x56226bfd4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feff4f17640 1 --2- 172.21.15.45:0/1825935372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfd6750 0x56226bfd4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feff4f17640 1 -- 172.21.15.45:0/1825935372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe000b940 con 0x56226bfcd980 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feff4f17640 1 --2- 172.21.15.45:0/1825935372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 0x56226bfd3bf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fefe0005d00 tx=0x7fefe0005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a9f0d6b75f151b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7fefd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7fefd8ff9640 1 -- 172.21.15.45:0/1825935372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0021070 con 0x56226bfcd980 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7fefd8ff9640 1 -- 172.21.15.45:0/1825935372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fefe0017070 con 0x56226bfcd980 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feffa81a740 1 -- 172.21.15.45:0/1825935372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56226bf2e010 con 0x56226bfcd980 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7fefd8ff9640 1 -- 172.21.15.45:0/1825935372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefe0005710 con 0x56226bfcd980 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feffa81a740 10 client.15549 fetch_fsmap learned FSMap version 23 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.064+0000 7feffa81a740 1 -- 172.21.15.45:0/1825935372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56226bf336a0 con 0x56226bfcd980 2024-01-31T05:24:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.065+0000 7fefd8ff9640 1 -- 172.21.15.45:0/1825935372 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56226bf2e010 con 0x56226bfcd980 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.065+0000 7fefd8ff9640 1 -- 172.21.15.45:0/1825935372 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fefe00035f0 con 0x56226bfcd980 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.065+0000 7feffa81a740 10 client.15549 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.065+0000 7feffa81a740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.065+0000 7feffa81a740 20 client.15549 populate_metadata read hostname 'smithi045' 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.065+0000 7feffa81a740 1 --2- 172.21.15.45:0/1825935372 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56226bfe64d0 0x56226bfe88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.065+0000 7feffa81a740 4 client.15549 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.065+0000 7feffa81a740 1 -- 172.21.15.45:0/1825935372 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56226bf2dd30 con 0x56226bfe64d0 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.065+0000 7feff5718640 1 --2- 172.21.15.45:0/1825935372 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56226bfe64d0 0x56226bfe88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.065+0000 7fefd8ff9640 10 client.15549 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.065+0000 7feff5718640 1 --2- 172.21.15.45:0/1825935372 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56226bfe64d0 0x56226bfe88f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69cd887ec243e4d6 server_cookie=afaa552ef0d66df7 in_seq=0 out_seq=0 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7fefd8ff9640 1 -- 172.21.15.45:0/1825935372 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56226bf2dd30 con 0x56226bfe64d0 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7fefd8ff9640 10 client.15549 handle_command_reply: tid=0 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7feffa81a740 1 client.15549 shutdown 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7feffa81a740 1 -- 172.21.15.45:0/1825935372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 msgr2=0x56226bfd3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7feffa81a740 1 --2- 172.21.15.45:0/1825935372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 0x56226bfd3bf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fefe0005d00 tx=0x7fefe0005230 comp rx=0 tx=0).stop 2024-01-31T05:24:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7feffa81a740 1 -- 172.21.15.45:0/1825935372 shutdown_connections 2024-01-31T05:24:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7feffa81a740 1 --2- 172.21.15.45:0/1825935372 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56226bfe64d0 0x56226bfe88f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7feffa81a740 1 --2- 172.21.15.45:0/1825935372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 0x56226bfd3bf0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7feffa81a740 1 --2- 172.21.15.45:0/1825935372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfd6750 0x56226bfd4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7feffa81a740 1 --2- 172.21.15.45:0/1825935372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd060 0x56226bfd36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7fefd8ff9640 0 client.15549 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7feffa81a740 1 -- 172.21.15.45:0/1825935372 >> 172.21.15.45:0/1825935372 conn(0x56226bfc7980 msgr2=0x56226bfc96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7feffa81a740 1 -- 172.21.15.45:0/1825935372 shutdown_connections 2024-01-31T05:24:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7feffa81a740 1 -- 172.21.15.45:0/1825935372 wait complete. 2024-01-31T05:24:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.066+0000 7feffa81a740 20 client.15549 trim_cache size 0 max 16384 2024-01-31T05:24:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feffa81a740 1 Processor -- start 2024-01-31T05:24:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feffa81a740 1 -- start start 2024-01-31T05:24:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feffa81a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd060 0x56226bfd5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feffa81a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 0x56226bfd3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feffa81a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfd6750 0x56226bfd0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feffa81a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226bfc8980 con 0x56226bfcd980 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feffa81a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226bfd8f00 con 0x56226bfd6750 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feffa81a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226bfc3240 con 0x56226bfcd060 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feff4f17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 0x56226bfd3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feff4f17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 0x56226bfd3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45766/0 (socket says 172.21.15.45:45766) 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feff4f17640 1 -- 172.21.15.45:0/621355589 learned_addr learned my addr 172.21.15.45:0/621355589 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feff5718640 1 --2- 172.21.15.45:0/621355589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd060 0x56226bfd5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.081+0000 7feff5f19640 1 --2- 172.21.15.45:0/621355589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfd6750 0x56226bfd0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.082+0000 7feff4f17640 1 -- 172.21.15.45:0/621355589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd060 msgr2=0x56226bfd5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.082+0000 7feff4f17640 1 --2- 172.21.15.45:0/621355589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd060 0x56226bfd5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.082+0000 7feff4f17640 1 -- 172.21.15.45:0/621355589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfd6750 msgr2=0x56226bfd0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.082+0000 7feff4f17640 1 --2- 172.21.15.45:0/621355589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfd6750 0x56226bfd0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.082+0000 7feff4f17640 1 -- 172.21.15.45:0/621355589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56226bed0030 con 0x56226bfcd980 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.082+0000 7feff4f17640 1 --2- 172.21.15.45:0/621355589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 0x56226bfd3f50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fefe000b7e0 tx=0x7fefe000bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13f0e4f808f947b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.082+0000 7fefdaffd640 1 -- 172.21.15.45:0/621355589 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0021070 con 0x56226bfcd980 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.082+0000 7fefdaffd640 1 -- 172.21.15.45:0/621355589 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefe0022040 con 0x56226bfcd980 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.082+0000 7fefdaffd640 1 -- 172.21.15.45:0/621355589 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe00048d0 con 0x56226bfcd980 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.083+0000 7feffa81a740 1 -- 172.21.15.45:0/621355589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 msgr2=0x56226bfd3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.083+0000 7feffa81a740 1 --2- 172.21.15.45:0/621355589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 0x56226bfd3f50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fefe000b7e0 tx=0x7fefe000bbe0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.083+0000 7feffa81a740 1 -- 172.21.15.45:0/621355589 shutdown_connections 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.083+0000 7feffa81a740 1 --2- 172.21.15.45:0/621355589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd980 0x56226bfd3f50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.083+0000 7feffa81a740 1 --2- 172.21.15.45:0/621355589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfd6750 0x56226bfd0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.083+0000 7feffa81a740 1 --2- 172.21.15.45:0/621355589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfcd060 0x56226bfd5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.083+0000 7feffa81a740 1 -- 172.21.15.45:0/621355589 >> 172.21.15.45:0/621355589 conn(0x56226bfc7980 msgr2=0x56226bfd1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.083+0000 7feffa81a740 1 -- 172.21.15.45:0/621355589 shutdown_connections 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.083+0000 7feffa81a740 1 -- 172.21.15.45:0/621355589 wait complete. 2024-01-31T05:24:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.083+0000 7feffa81a740 1 -- start start 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.083+0000 7feffa81a740 1 Processor -- start 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.083+0000 7feffa81a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd060 0x56226bfdd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feffa81a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfcd980 0x56226bfdb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feffa81a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfd6750 0x56226bfdbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feffa81a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226bfddd90 con 0x56226bfcd060 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feffa81a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226bfdc360 con 0x56226bfcd980 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feffa81a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226bfdc520 con 0x56226bfd6750 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feffa81a740 10 client.0 _ll_register_callbacks cb 0x7fefe000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feff5718640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd060 0x56226bfdd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feff5718640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd060 0x56226bfdd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45776/0 (socket says 172.21.15.45:45776) 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feff5718640 1 -- 172.21.15.45:0/4162478412 learned_addr learned my addr 172.21.15.45:0/4162478412 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feff5f19640 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfd6750 0x56226bfdbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feff4f17640 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfcd980 0x56226bfdb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feff5718640 1 -- 172.21.15.45:0/4162478412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfd6750 msgr2=0x56226bfdbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feff5718640 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfd6750 0x56226bfdbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feff5718640 1 -- 172.21.15.45:0/4162478412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfcd980 msgr2=0x56226bfdb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feff5718640 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfcd980 0x56226bfdb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feff5718640 1 -- 172.21.15.45:0/4162478412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56226bf336a0 con 0x56226bfcd060 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feff4f17640 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfcd980 0x56226bfdb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feff5718640 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd060 0x56226bfdd4c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fefec096590 tx=0x7fefec0cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2c83b536476653 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7fefd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7fefd8ff9640 1 -- 172.21.15.45:0/4162478412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec0b1070 con 0x56226bfcd060 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7fefd8ff9640 1 -- 172.21.15.45:0/4162478412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fefec09a090 con 0x56226bfcd060 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7feffa81a740 1 -- 172.21.15.45:0/4162478412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefe0015070 con 0x56226bfcd060 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.084+0000 7fefd8ff9640 1 -- 172.21.15.45:0/4162478412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefec0963d0 con 0x56226bfcd060 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7feffa81a740 10 client.15555 fetch_fsmap learned FSMap version 23 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7feffa81a740 1 -- 172.21.15.45:0/4162478412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56226bef5a50 con 0x56226bfcd060 2024-01-31T05:24:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7fefd8ff9640 1 -- 172.21.15.45:0/4162478412 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefec0a6080 con 0x56226bfcd060 2024-01-31T05:24:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7fefd8ff9640 1 -- 172.21.15.45:0/4162478412 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fefec0967a0 con 0x56226bfcd060 2024-01-31T05:24:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7feffa81a740 10 client.15555 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:24:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7feffa81a740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:24:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7feffa81a740 20 client.15555 populate_metadata read hostname 'smithi045' 2024-01-31T05:24:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7feffa81a740 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56226c03ded0 0x56226c0402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7feffa81a740 4 client.15555 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:24:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7feffa81a740 1 -- 172.21.15.45:0/4162478412 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56226bc34600 con 0x56226c03ded0 2024-01-31T05:24:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7feff4f17640 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56226c03ded0 0x56226c0402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7feff4f17640 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56226c03ded0 0x56226c0402f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3a9542ef52c7b3 server_cookie=efc0712ec78f8ca6 in_seq=0 out_seq=0 2024-01-31T05:24:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.085+0000 7fefd8ff9640 10 client.15555 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7fefd8ff9640 1 -- 172.21.15.45:0/4162478412 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56226bc34600 con 0x56226c03ded0 2024-01-31T05:24:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7fefd8ff9640 10 client.15555 handle_command_reply: tid=0 2024-01-31T05:24:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7feffa81a740 1 client.15555 shutdown 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7feffa81a740 1 -- 172.21.15.45:0/4162478412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd060 msgr2=0x56226bfdd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7feffa81a740 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd060 0x56226bfdd4c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fefec096590 tx=0x7fefec0cf040 comp rx=0 tx=0).stop 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7feffa81a740 1 -- 172.21.15.45:0/4162478412 shutdown_connections 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7feffa81a740 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56226c03ded0 0x56226c0402f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7feffa81a740 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56226bfd6750 0x56226bfdbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7feffa81a740 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56226bfcd980 0x56226bfdb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7feffa81a740 1 --2- 172.21.15.45:0/4162478412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56226bfcd060 0x56226bfdd4c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7fefd8ff9640 0 client.15555 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7feffa81a740 1 -- 172.21.15.45:0/4162478412 >> 172.21.15.45:0/4162478412 conn(0x56226bfc7980 msgr2=0x56226bfc6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7feffa81a740 1 -- 172.21.15.45:0/4162478412 shutdown_connections 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7feffa81a740 1 -- 172.21.15.45:0/4162478412 wait complete. 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.095+0000 7feffa81a740 20 client.15555 trim_cache size 0 max 16384 2024-01-31T05:24:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:24:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:24:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:24:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:24:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.096+0000 7feff797c640 1 -- 172.21.15.45:0/3490918330 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fefdc0a14f0 msgr2=0x7fefdc0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.096+0000 7feff797c640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fefdc0a14f0 0x7fefdc0a39b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fefe8004af0 tx=0x7fefe8005c50 comp rx=0 tx=0).stop 2024-01-31T05:24:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.096+0000 7feff797c640 1 -- 172.21.15.45:0/3490918330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff0068370 msgr2=0x7feff0197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.096+0000 7feff797c640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff0068370 0x7feff0197100 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fefec00e9f0 tx=0x7fefec00eeb0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.096+0000 7feff797c640 1 -- 172.21.15.45:0/3490918330 shutdown_connections 2024-01-31T05:24:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.096+0000 7feff797c640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fefdc0a14f0 0x7fefdc0a39b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.096+0000 7feff797c640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feff0106de0 0x7feff019b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.096+0000 7feff797c640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feff0068d10 0x7feff0197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.096+0000 7feff797c640 1 --2- 172.21.15.45:0/3490918330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff0068370 0x7feff0197100 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.096+0000 7feff797c640 1 -- 172.21.15.45:0/3490918330 >> 172.21.15.45:0/3490918330 conn(0x7feff0076a40 msgr2=0x7feff00fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.096+0000 7feff797c640 1 -- 172.21.15.45:0/3490918330 shutdown_connections 2024-01-31T05:24:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:42.096+0000 7feff797c640 1 -- 172.21.15.45:0/3490918330 wait complete. 2024-01-31T05:24:42.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 80446 10.68 MB/sec warmup 42 sec latency 26.689 ms 2024-01-31T05:24:42.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:42 smithi186 ceph-mon[35659]: pgmap v63: 97 pgs: 97 active+clean; 60 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-01-31T05:24:42.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:42 smithi045 ceph-mon[31585]: pgmap v63: 97 pgs: 97 active+clean; 60 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-01-31T05:24:42.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:42 smithi156 ceph-mon[35720]: pgmap v63: 97 pgs: 97 active+clean; 60 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-01-31T05:24:43.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 82604 10.74 MB/sec warmup 43 sec latency 11.833 ms 2024-01-31T05:24:44.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 85499 10.90 MB/sec warmup 44 sec latency 12.839 ms 2024-01-31T05:24:44.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:44 smithi186 ceph-mon[35659]: pgmap v64: 97 pgs: 97 active+clean; 56 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 316 op/s 2024-01-31T05:24:44.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:44 smithi045 ceph-mon[31585]: pgmap v64: 97 pgs: 97 active+clean; 56 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 316 op/s 2024-01-31T05:24:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:44 smithi156 ceph-mon[35720]: pgmap v64: 97 pgs: 97 active+clean; 56 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 316 op/s 2024-01-31T05:24:45.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 88198 10.95 MB/sec warmup 45 sec latency 10.646 ms 2024-01-31T05:24:46.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 90385 10.96 MB/sec warmup 46 sec latency 8.613 ms 2024-01-31T05:24:46.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:46 smithi186 ceph-mon[35659]: pgmap v65: 97 pgs: 97 active+clean; 56 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-01-31T05:24:46.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:46 smithi045 ceph-mon[31585]: pgmap v65: 97 pgs: 97 active+clean; 56 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-01-31T05:24:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:46 smithi156 ceph-mon[35720]: pgmap v65: 97 pgs: 97 active+clean; 56 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-01-31T05:24:47.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 92806 11.03 MB/sec warmup 47 sec latency 28.513 ms 2024-01-31T05:24:48.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 95843 11.16 MB/sec warmup 48 sec latency 6.754 ms 2024-01-31T05:24:48.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:48 smithi186 ceph-mon[35659]: pgmap v66: 97 pgs: 97 active+clean; 62 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-01-31T05:24:48.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:48 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:24:48.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:48 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:48.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:48 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:48.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:48 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:48.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:48 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:48.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:48 smithi045 ceph-mon[31585]: pgmap v66: 97 pgs: 97 active+clean; 62 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-01-31T05:24:48.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:48 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:24:48.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:48 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:48.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:48 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:48.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:48 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:48.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:48 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:48 smithi156 ceph-mon[35720]: pgmap v66: 97 pgs: 97 active+clean; 62 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-01-31T05:24:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:48 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:24:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:48 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:48 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:48 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:48 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:49.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 98194 11.11 MB/sec warmup 49 sec latency 11.428 ms 2024-01-31T05:24:49.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:49 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:24:49.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:49 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:24:49.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:49 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:49.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:49 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:24:49.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:49 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:24:49.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:49 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:49.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:49 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:24:49.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:49 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:24:49.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:49 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:24:50.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 100171 11.14 MB/sec warmup 50 sec latency 11.724 ms 2024-01-31T05:24:50.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:50 smithi186 ceph-mon[35659]: pgmap v67: 97 pgs: 97 active+clean; 62 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-01-31T05:24:50.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:50 smithi045 ceph-mon[31585]: pgmap v67: 97 pgs: 97 active+clean; 62 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-01-31T05:24:50.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:50 smithi156 ceph-mon[35720]: pgmap v67: 97 pgs: 97 active+clean; 62 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-01-31T05:24:51.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 103183 11.28 MB/sec warmup 51 sec latency 12.540 ms 2024-01-31T05:24:52.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 105669 11.28 MB/sec warmup 52 sec latency 19.550 ms 2024-01-31T05:24:52.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:52 smithi186 ceph-mon[35659]: pgmap v68: 97 pgs: 97 active+clean; 62 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-01-31T05:24:52.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:52 smithi045 ceph-mon[31585]: pgmap v68: 97 pgs: 97 active+clean; 62 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-01-31T05:24:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:52 smithi156 ceph-mon[35720]: pgmap v68: 97 pgs: 97 active+clean; 62 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-01-31T05:24:53.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 107853 11.32 MB/sec warmup 53 sec latency 7.944 ms 2024-01-31T05:24:54.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 110481 11.37 MB/sec warmup 54 sec latency 10.618 ms 2024-01-31T05:24:54.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:54 smithi186 ceph-mon[35659]: pgmap v69: 97 pgs: 97 active+clean; 68 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.6 MiB/s wr, 299 op/s 2024-01-31T05:24:54.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:54 smithi045 ceph-mon[31585]: pgmap v69: 97 pgs: 97 active+clean; 68 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.6 MiB/s wr, 299 op/s 2024-01-31T05:24:54.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:54 smithi156 ceph-mon[35720]: pgmap v69: 97 pgs: 97 active+clean; 68 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.6 MiB/s wr, 299 op/s 2024-01-31T05:24:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.186+0000 7f425171f640 1 -- 172.21.15.45:0/4001053227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424c101ed0 msgr2=0x7f424c1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.186+0000 7f425171f640 1 --2- 172.21.15.45:0/4001053227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424c101ed0 0x7f424c1022b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f423c009900 tx=0x7f423c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:24:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.186+0000 7f425171f640 1 -- 172.21.15.45:0/4001053227 shutdown_connections 2024-01-31T05:24:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.186+0000 7f425171f640 1 --2- 172.21.15.45:0/4001053227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f424c10b340 0x7f424c10d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.186+0000 7f425171f640 1 --2- 172.21.15.45:0/4001053227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c1027f0 0x7f424c10acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.186+0000 7f425171f640 1 --2- 172.21.15.45:0/4001053227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424c101ed0 0x7f424c1022b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.186+0000 7f425171f640 1 -- 172.21.15.45:0/4001053227 >> 172.21.15.45:0/4001053227 conn(0x7f424c076d20 msgr2=0x7f424c077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.186+0000 7f425171f640 1 -- 172.21.15.45:0/4001053227 shutdown_connections 2024-01-31T05:24:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.186+0000 7f425171f640 1 -- 172.21.15.45:0/4001053227 wait complete. 2024-01-31T05:24:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f425171f640 1 Processor -- start 2024-01-31T05:24:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f425171f640 1 -- start start 2024-01-31T05:24:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f425171f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424c101ed0 0x7f424c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f425171f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c1027f0 0x7f424c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f425171f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f424c10b340 0x7f424c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f425171f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c19c020 con 0x7f424c101ed0 2024-01-31T05:24:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f425171f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c19c190 con 0x7f424c1027f0 2024-01-31T05:24:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f425171f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c19c350 con 0x7f424c10b340 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f424affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424c101ed0 0x7f424c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f424b7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f424c10b340 0x7f424c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f424affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424c101ed0 0x7f424c1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46800/0 (socket says 172.21.15.45:46800) 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f424affd640 1 -- 172.21.15.45:0/3520903704 learned_addr learned my addr 172.21.15.45:0/3520903704 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f424a7fc640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c1027f0 0x7f424c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f424affd640 1 -- 172.21.15.45:0/3520903704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f424c10b340 msgr2=0x7f424c19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f424affd640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f424c10b340 0x7f424c19ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f424affd640 1 -- 172.21.15.45:0/3520903704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c1027f0 msgr2=0x7f424c1976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f424affd640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c1027f0 0x7f424c1976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f424affd640 1 -- 172.21.15.45:0/3520903704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c009560 con 0x7f424c101ed0 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f424b7fe640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f424c10b340 0x7f424c19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.187+0000 7f424a7fc640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c1027f0 0x7f424c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:24:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.188+0000 7f424affd640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424c101ed0 0x7f424c1971b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f423c006010 tx=0x7f423c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.188+0000 7f4233fff640 1 -- 172.21.15.45:0/3520903704 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c004250 con 0x7f424c101ed0 2024-01-31T05:24:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.188+0000 7f4233fff640 1 -- 172.21.15.45:0/3520903704 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f423c0043b0 con 0x7f424c101ed0 2024-01-31T05:24:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.188+0000 7f425171f640 1 -- 172.21.15.45:0/3520903704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424c19c5d0 con 0x7f424c101ed0 2024-01-31T05:24:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.188+0000 7f4233fff640 1 -- 172.21.15.45:0/3520903704 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c003ba0 con 0x7f424c101ed0 2024-01-31T05:24:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.188+0000 7f425171f640 1 -- 172.21.15.45:0/3520903704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424c1a3eb0 con 0x7f424c101ed0 2024-01-31T05:24:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.190+0000 7f4233fff640 1 -- 172.21.15.45:0/3520903704 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f423c003d00 con 0x7f424c101ed0 2024-01-31T05:24:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.190+0000 7f42545b1740 1 Processor -- start 2024-01-31T05:24:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.190+0000 7f4233fff640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f42340a14f0 0x7f42340a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.190+0000 7f42545b1740 1 -- start start 2024-01-31T05:24:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.190+0000 7f4233fff640 1 -- 172.21.15.45:0/3520903704 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f423c0ef560 con 0x7f424c101ed0 2024-01-31T05:24:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.190+0000 7f424b7fe640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f42340a14f0 0x7f42340a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f42545b1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e060 0x557bf4c4e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f42545b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e980 0x557bf4c57210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f42545b1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c57750 0x557bf4c59b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f42545b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf4bbd040 con 0x557bf4c4e980 2024-01-31T05:24:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f42545b1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf4bba730 con 0x557bf4c4e060 2024-01-31T05:24:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f42545b1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf4bb6020 con 0x557bf4c57750 2024-01-31T05:24:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e980 0x557bf4c57210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e060 0x557bf4c4e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e060 0x557bf4c4e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:32880/0 (socket says 172.21.15.45:32880) 2024-01-31T05:24:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424affd640 1 -- 172.21.15.45:0/1178164219 learned_addr learned my addr 172.21.15.45:0/1178164219 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424b7fe640 1 --2- 172.21.15.45:0/1178164219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c57750 0x557bf4c59b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424affd640 1 -- 172.21.15.45:0/1178164219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c57750 msgr2=0x557bf4c59b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424affd640 1 --2- 172.21.15.45:0/1178164219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c57750 0x557bf4c59b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424affd640 1 -- 172.21.15.45:0/1178164219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e980 msgr2=0x557bf4c57210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424affd640 1 --2- 172.21.15.45:0/1178164219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e980 0x557bf4c57210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424b7fe640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f42340a14f0 0x7f42340a39b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4244009fd0 tx=0x7f424400fb40 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424affd640 1 -- 172.21.15.45:0/1178164219 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bf4bade40 con 0x557bf4c4e060 2024-01-31T05:24:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424a7fc640 1 --2- 172.21.15.45:0/1178164219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e980 0x557bf4c57210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:24:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f424affd640 1 --2- 172.21.15.45:0/1178164219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e060 0x557bf4c4e440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f423c0c42c0 tx=0x7f423c0bbdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2900d5c747a34de1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.191+0000 7f42317fa640 1 -- 172.21.15.45:0/1178164219 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0c05b0 con 0x557bf4c4e060 2024-01-31T05:24:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.192+0000 7f42317fa640 1 -- 172.21.15.45:0/1178164219 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f423c0c47f0 con 0x557bf4c4e060 2024-01-31T05:24:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.192+0000 7f42317fa640 1 -- 172.21.15.45:0/1178164219 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0c03d0 con 0x557bf4c4e060 2024-01-31T05:24:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.192+0000 7f42545b1740 1 -- 172.21.15.45:0/1178164219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e060 msgr2=0x557bf4c4e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.192+0000 7f42545b1740 1 --2- 172.21.15.45:0/1178164219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e060 0x557bf4c4e440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f423c0c42c0 tx=0x7f423c0bbdc0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.192+0000 7f42545b1740 1 -- 172.21.15.45:0/1178164219 shutdown_connections 2024-01-31T05:24:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.192+0000 7f42545b1740 1 --2- 172.21.15.45:0/1178164219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e980 0x557bf4c57210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.192+0000 7f42545b1740 1 --2- 172.21.15.45:0/1178164219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c57750 0x557bf4c59b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.192+0000 7f42545b1740 1 --2- 172.21.15.45:0/1178164219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e060 0x557bf4c4e440 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.192+0000 7f42545b1740 1 -- 172.21.15.45:0/1178164219 >> 172.21.15.45:0/1178164219 conn(0x557bf4c48980 msgr2=0x557bf4c48d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.192+0000 7f42545b1740 1 -- 172.21.15.45:0/1178164219 shutdown_connections 2024-01-31T05:24:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.192+0000 7f42545b1740 1 -- 172.21.15.45:0/1178164219 wait complete. 2024-01-31T05:24:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f42545b1740 1 -- start start 2024-01-31T05:24:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f42545b1740 1 Processor -- start 2024-01-31T05:24:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f42545b1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e060 0x557bf4c546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f42545b1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c4e980 0x557bf4c54bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f42545b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c57750 0x557bf4c55130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f42545b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf4b6f1b0 con 0x557bf4c57750 2024-01-31T05:24:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f42545b1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf4b6ee50 con 0x557bf4c4e060 2024-01-31T05:24:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f42545b1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf4b34090 con 0x557bf4c4e980 2024-01-31T05:24:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f42545b1740 10 client.0 _ll_register_callbacks cb 0x557bf4b59a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:24:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f424b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c57750 0x557bf4c55130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f424a7fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c4e980 0x557bf4c54bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f424b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c57750 0x557bf4c55130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46828/0 (socket says 172.21.15.45:46828) 2024-01-31T05:24:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f424b7fe640 1 -- 172.21.15.45:0/2210488078 learned_addr learned my addr 172.21.15.45:0/2210488078 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f424affd640 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e060 0x557bf4c546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f424b7fe640 1 -- 172.21.15.45:0/2210488078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c4e980 msgr2=0x557bf4c54bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f424b7fe640 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c4e980 0x557bf4c54bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f424b7fe640 1 -- 172.21.15.45:0/2210488078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e060 msgr2=0x557bf4c546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f424b7fe640 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e060 0x557bf4c546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f424b7fe640 1 -- 172.21.15.45:0/2210488078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424c1a3eb0 con 0x557bf4c57750 2024-01-31T05:24:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f424a7fc640 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c4e980 0x557bf4c54bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:24:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f424b7fe640 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c57750 0x557bf4c55130 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4244013f80 tx=0x7f4244004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795a92856d2a6fd0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.193+0000 7f422eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:24:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f422eff5640 1 -- 172.21.15.45:0/2210488078 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244011e80 con 0x557bf4c57750 2024-01-31T05:24:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f422eff5640 1 -- 172.21.15.45:0/2210488078 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4244004100 con 0x557bf4c57750 2024-01-31T05:24:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f42545b1740 1 -- 172.21.15.45:0/2210488078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557bf4baf010 con 0x557bf4c57750 2024-01-31T05:24:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f422eff5640 1 -- 172.21.15.45:0/2210488078 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42440042c0 con 0x557bf4c57750 2024-01-31T05:24:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f42545b1740 10 client.15567 fetch_fsmap learned FSMap version 23 2024-01-31T05:24:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f42545b1740 1 -- 172.21.15.45:0/2210488078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557bf4bb46a0 con 0x557bf4c57750 2024-01-31T05:24:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f422eff5640 1 -- 172.21.15.45:0/2210488078 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557bf4baf010 con 0x557bf4c57750 2024-01-31T05:24:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f422eff5640 1 -- 172.21.15.45:0/2210488078 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f424401d440 con 0x557bf4c57750 2024-01-31T05:24:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f42545b1740 10 client.15567 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:24:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f42545b1740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:24:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f42545b1740 20 client.15567 populate_metadata read hostname 'smithi045' 2024-01-31T05:24:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f42545b1740 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557bf4c672f0 0x557bf4c69710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f42545b1740 4 client.15567 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:24:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f42545b1740 1 -- 172.21.15.45:0/2210488078 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bf4baed30 con 0x557bf4c672f0 2024-01-31T05:24:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.194+0000 7f424affd640 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557bf4c672f0 0x557bf4c69710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.195+0000 7f424affd640 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557bf4c672f0 0x557bf4c69710 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24b0d6e11916fcd server_cookie=57df1949906f4c7a in_seq=0 out_seq=0 2024-01-31T05:24:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.195+0000 7f422eff5640 10 client.15567 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.195+0000 7f422eff5640 1 -- 172.21.15.45:0/2210488078 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557bf4baed30 con 0x557bf4c672f0 2024-01-31T05:24:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.195+0000 7f422eff5640 10 client.15567 handle_command_reply: tid=0 2024-01-31T05:24:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.195+0000 7f42545b1740 1 client.15567 shutdown 2024-01-31T05:24:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.196+0000 7f42545b1740 1 -- 172.21.15.45:0/2210488078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c57750 msgr2=0x557bf4c55130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.196+0000 7f42545b1740 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c57750 0x557bf4c55130 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4244013f80 tx=0x7f4244004ab0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.196+0000 7f42545b1740 1 -- 172.21.15.45:0/2210488078 shutdown_connections 2024-01-31T05:24:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.196+0000 7f42545b1740 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557bf4c672f0 0x557bf4c69710 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.196+0000 7f42545b1740 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c57750 0x557bf4c55130 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.196+0000 7f42545b1740 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c4e980 0x557bf4c54bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.196+0000 7f42545b1740 1 --2- 172.21.15.45:0/2210488078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e060 0x557bf4c546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.196+0000 7f422eff5640 0 client.15567 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.196+0000 7f42545b1740 1 -- 172.21.15.45:0/2210488078 >> 172.21.15.45:0/2210488078 conn(0x557bf4c48980 msgr2=0x557bf4c4a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.196+0000 7f42545b1740 1 -- 172.21.15.45:0/2210488078 shutdown_connections 2024-01-31T05:24:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.196+0000 7f42545b1740 1 -- 172.21.15.45:0/2210488078 wait complete. 2024-01-31T05:24:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.196+0000 7f42545b1740 20 client.15567 trim_cache size 0 max 16384 2024-01-31T05:24:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f42545b1740 1 Processor -- start 2024-01-31T05:24:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f42545b1740 1 -- start start 2024-01-31T05:24:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f42545b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 0x557bf4c56de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f42545b1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e980 0x557bf4c54f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f42545b1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c57750 0x557bf4c51910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f42545b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf4baed30 con 0x557bf4c4e060 2024-01-31T05:24:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f42545b1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf4baf010 con 0x557bf4c4e980 2024-01-31T05:24:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f42545b1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf4c442b0 con 0x557bf4c57750 2024-01-31T05:24:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f424affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 0x557bf4c56de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f424affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 0x557bf4c56de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46840/0 (socket says 172.21.15.45:46840) 2024-01-31T05:24:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f424affd640 1 -- 172.21.15.45:0/2354141348 learned_addr learned my addr 172.21.15.45:0/2354141348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f424b7fe640 1 --2- 172.21.15.45:0/2354141348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c57750 0x557bf4c51910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f424a7fc640 1 --2- 172.21.15.45:0/2354141348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e980 0x557bf4c54f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f424affd640 1 -- 172.21.15.45:0/2354141348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c57750 msgr2=0x557bf4c51910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f424affd640 1 --2- 172.21.15.45:0/2354141348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c57750 0x557bf4c51910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f424affd640 1 -- 172.21.15.45:0/2354141348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e980 msgr2=0x557bf4c54f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f424affd640 1 --2- 172.21.15.45:0/2354141348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e980 0x557bf4c54f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f424affd640 1 -- 172.21.15.45:0/2354141348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bf4b51030 con 0x557bf4c4e060 2024-01-31T05:24:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f424affd640 1 --2- 172.21.15.45:0/2354141348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 0x557bf4c56de0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f423c02c720 tx=0x7f423c0bbef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce573d9649bf3526 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.210+0000 7f4230ff9640 1 -- 172.21.15.45:0/2354141348 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0ecfa0 con 0x557bf4c4e060 2024-01-31T05:24:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.211+0000 7f4230ff9640 1 -- 172.21.15.45:0/2354141348 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f423c0c05b0 con 0x557bf4c4e060 2024-01-31T05:24:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.211+0000 7f4230ff9640 1 -- 172.21.15.45:0/2354141348 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0ef370 con 0x557bf4c4e060 2024-01-31T05:24:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.211+0000 7f42545b1740 1 -- 172.21.15.45:0/2354141348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 msgr2=0x557bf4c56de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.211+0000 7f42545b1740 1 --2- 172.21.15.45:0/2354141348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 0x557bf4c56de0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f423c02c720 tx=0x7f423c0bbef0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.211+0000 7f42545b1740 1 -- 172.21.15.45:0/2354141348 shutdown_connections 2024-01-31T05:24:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.211+0000 7f42545b1740 1 --2- 172.21.15.45:0/2354141348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c57750 0x557bf4c51910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.211+0000 7f42545b1740 1 --2- 172.21.15.45:0/2354141348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c4e980 0x557bf4c54f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.211+0000 7f42545b1740 1 --2- 172.21.15.45:0/2354141348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 0x557bf4c56de0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.211+0000 7f42545b1740 1 -- 172.21.15.45:0/2354141348 >> 172.21.15.45:0/2354141348 conn(0x557bf4c48980 msgr2=0x557bf4c52f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.211+0000 7f42545b1740 1 -- 172.21.15.45:0/2354141348 shutdown_connections 2024-01-31T05:24:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.211+0000 7f42545b1740 1 -- 172.21.15.45:0/2354141348 wait complete. 2024-01-31T05:24:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f42545b1740 1 -- start start 2024-01-31T05:24:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f42545b1740 1 Processor -- start 2024-01-31T05:24:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f42545b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 0x557bf4c5da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f42545b1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c4e980 0x557bf4c5df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f42545b1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c57750 0x557bf4c5c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f42545b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf4c5c660 con 0x557bf4c4e060 2024-01-31T05:24:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f42545b1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf4c5c7d0 con 0x557bf4c57750 2024-01-31T05:24:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f42545b1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf4c5c990 con 0x557bf4c4e980 2024-01-31T05:24:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f424affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 0x557bf4c5da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f42545b1740 10 client.0 _ll_register_callbacks cb 0x7f423c0b80e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:24:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f424affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 0x557bf4c5da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46854/0 (socket says 172.21.15.45:46854) 2024-01-31T05:24:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f424affd640 1 -- 172.21.15.45:0/3036778123 learned_addr learned my addr 172.21.15.45:0/3036778123 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:24:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f424a7fc640 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c4e980 0x557bf4c5df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f424b7fe640 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c57750 0x557bf4c5c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f424affd640 1 -- 172.21.15.45:0/3036778123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c4e980 msgr2=0x557bf4c5df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f424affd640 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c4e980 0x557bf4c5df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f424affd640 1 -- 172.21.15.45:0/3036778123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c57750 msgr2=0x557bf4c5c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:24:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f424affd640 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c57750 0x557bf4c5c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f424affd640 1 -- 172.21.15.45:0/3036778123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bf4bade40 con 0x557bf4c4e060 2024-01-31T05:24:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f424affd640 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 0x557bf4c5da50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f423c0bc160 tx=0x7f423c0eceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df71aeb54346d6a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:24:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.212+0000 7f422eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:24:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f422eff5640 1 -- 172.21.15.45:0/3036778123 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0ecfa0 con 0x557bf4c4e060 2024-01-31T05:24:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f422eff5640 1 -- 172.21.15.45:0/3036778123 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f423c0b8b70 con 0x557bf4c4e060 2024-01-31T05:24:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f42545b1740 1 -- 172.21.15.45:0/3036778123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f423c0c47f0 con 0x557bf4c4e060 2024-01-31T05:24:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f422eff5640 1 -- 172.21.15.45:0/3036778123 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f423c0ef110 con 0x557bf4c4e060 2024-01-31T05:24:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f42545b1740 10 client.15573 fetch_fsmap learned FSMap version 23 2024-01-31T05:24:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f42545b1740 1 -- 172.21.15.45:0/3036778123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557bf4b76a50 con 0x557bf4c4e060 2024-01-31T05:24:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f422eff5640 1 -- 172.21.15.45:0/3036778123 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f423c0c47f0 con 0x557bf4c4e060 2024-01-31T05:24:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f422eff5640 1 -- 172.21.15.45:0/3036778123 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f423c03c070 con 0x557bf4c4e060 2024-01-31T05:24:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f42545b1740 10 client.15573 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:24:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f42545b1740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:24:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f42545b1740 20 client.15573 populate_metadata read hostname 'smithi045' 2024-01-31T05:24:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f42545b1740 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557bf4cbe4e0 0x557bf4cc0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:24:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f42545b1740 4 client.15573 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:24:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f42545b1740 1 -- 172.21.15.45:0/3036778123 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557bf48b5600 con 0x557bf4cbe4e0 2024-01-31T05:24:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.213+0000 7f424a7fc640 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557bf4cbe4e0 0x557bf4cc0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:24:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.214+0000 7f424a7fc640 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557bf4cbe4e0 0x557bf4cc0900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=376a684e78230e55 server_cookie=c8791590e90ccd0a in_seq=0 out_seq=0 2024-01-31T05:24:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.214+0000 7f422eff5640 10 client.15573 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.221+0000 7f422eff5640 1 -- 172.21.15.45:0/3036778123 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557bf48b5600 con 0x557bf4cbe4e0 2024-01-31T05:24:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.221+0000 7f422eff5640 10 client.15573 handle_command_reply: tid=0 2024-01-31T05:24:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f42545b1740 1 client.15573 shutdown 2024-01-31T05:24:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f42545b1740 1 -- 172.21.15.45:0/3036778123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 msgr2=0x557bf4c5da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f42545b1740 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 0x557bf4c5da50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f423c0bc160 tx=0x7f423c0eceb0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f42545b1740 1 -- 172.21.15.45:0/3036778123 shutdown_connections 2024-01-31T05:24:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f42545b1740 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557bf4cbe4e0 0x557bf4cc0900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f42545b1740 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557bf4c57750 0x557bf4c5c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f42545b1740 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bf4c4e980 0x557bf4c5df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f422eff5640 0 client.15573 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:24:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f42545b1740 1 --2- 172.21.15.45:0/3036778123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bf4c4e060 0x557bf4c5da50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f42545b1740 1 -- 172.21.15.45:0/3036778123 >> 172.21.15.45:0/3036778123 conn(0x557bf4c48980 msgr2=0x557bf4c47a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f42545b1740 1 -- 172.21.15.45:0/3036778123 shutdown_connections 2024-01-31T05:24:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f42545b1740 1 -- 172.21.15.45:0/3036778123 wait complete. 2024-01-31T05:24:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f42545b1740 20 client.15573 trim_cache size 0 max 16384 2024-01-31T05:24:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:24:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:24:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:24:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:24:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f425171f640 1 -- 172.21.15.45:0/3520903704 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f42340a14f0 msgr2=0x7f42340a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f425171f640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f42340a14f0 0x7f42340a39b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4244009fd0 tx=0x7f424400fb40 comp rx=0 tx=0).stop 2024-01-31T05:24:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f425171f640 1 -- 172.21.15.45:0/3520903704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424c101ed0 msgr2=0x7f424c1971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:24:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f425171f640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424c101ed0 0x7f424c1971b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f423c006010 tx=0x7f423c005d90 comp rx=0 tx=0).stop 2024-01-31T05:24:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f425171f640 1 -- 172.21.15.45:0/3520903704 shutdown_connections 2024-01-31T05:24:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f425171f640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f42340a14f0 0x7f42340a39b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f425171f640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f424c10b340 0x7f424c19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f425171f640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c1027f0 0x7f424c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.222+0000 7f425171f640 1 --2- 172.21.15.45:0/3520903704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424c101ed0 0x7f424c1971b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:24:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.223+0000 7f425171f640 1 -- 172.21.15.45:0/3520903704 >> 172.21.15.45:0/3520903704 conn(0x7f424c076d20 msgr2=0x7f424c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:24:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.223+0000 7f425171f640 1 -- 172.21.15.45:0/3520903704 shutdown_connections 2024-01-31T05:24:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:24:55.223+0000 7f425171f640 1 -- 172.21.15.45:0/3520903704 wait complete. 2024-01-31T05:24:55.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 113167 11.40 MB/sec warmup 55 sec latency 161.595 ms 2024-01-31T05:24:56.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 114533 11.32 MB/sec warmup 56 sec latency 308.079 ms 2024-01-31T05:24:56.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:56 smithi186 ceph-mon[35659]: pgmap v70: 97 pgs: 97 active+clean; 68 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-01-31T05:24:56.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:56 smithi045 ceph-mon[31585]: pgmap v70: 97 pgs: 97 active+clean; 68 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-01-31T05:24:56.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:56 smithi156 ceph-mon[35720]: pgmap v70: 97 pgs: 97 active+clean; 68 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-01-31T05:24:57.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 117298 11.43 MB/sec warmup 57 sec latency 99.115 ms 2024-01-31T05:24:58.773 INFO:tasks.workunit.client.0.smithi045.stdout: 1 119710 11.38 MB/sec warmup 58 sec latency 7.588 ms 2024-01-31T05:24:58.777 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:24:58 smithi156 ceph-mon[35720]: pgmap v71: 97 pgs: 97 active+clean; 81 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 293 op/s 2024-01-31T05:24:58.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:24:58 smithi186 ceph-mon[35659]: pgmap v71: 97 pgs: 97 active+clean; 81 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 293 op/s 2024-01-31T05:24:58.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:24:58 smithi045 ceph-mon[31585]: pgmap v71: 97 pgs: 97 active+clean; 81 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 293 op/s 2024-01-31T05:24:59.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 122105 11.46 MB/sec warmup 59 sec latency 8.046 ms 2024-01-31T05:25:00.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 124316 11.48 MB/sec warmup 60 sec latency 47.219 ms 2024-01-31T05:25:00.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:00 smithi186 ceph-mon[35659]: pgmap v72: 97 pgs: 97 active+clean; 81 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-01-31T05:25:00.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:00 smithi045 ceph-mon[31585]: pgmap v72: 97 pgs: 97 active+clean; 81 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-01-31T05:25:00.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:00 smithi156 ceph-mon[35720]: pgmap v72: 97 pgs: 97 active+clean; 81 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-01-31T05:25:01.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 126640 11.43 MB/sec warmup 61 sec latency 65.711 ms 2024-01-31T05:25:02.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 128448 11.45 MB/sec warmup 62 sec latency 121.410 ms 2024-01-31T05:25:02.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:02 smithi186 ceph-mon[35659]: pgmap v73: 97 pgs: 97 active+clean; 81 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-01-31T05:25:02.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:02 smithi045 ceph-mon[31585]: pgmap v73: 97 pgs: 97 active+clean; 81 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-01-31T05:25:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:02 smithi156 ceph-mon[35720]: pgmap v73: 97 pgs: 97 active+clean; 81 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-01-31T05:25:03.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 131426 11.54 MB/sec warmup 63 sec latency 35.439 ms 2024-01-31T05:25:04.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 133836 11.51 MB/sec warmup 64 sec latency 10.113 ms 2024-01-31T05:25:04.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:04 smithi186 ceph-mon[35659]: pgmap v74: 97 pgs: 97 active+clean; 85 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 349 op/s 2024-01-31T05:25:04.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:04 smithi045 ceph-mon[31585]: pgmap v74: 97 pgs: 97 active+clean; 85 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 349 op/s 2024-01-31T05:25:04.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:04 smithi156 ceph-mon[35720]: pgmap v74: 97 pgs: 97 active+clean; 85 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 349 op/s 2024-01-31T05:25:05.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 135858 11.53 MB/sec warmup 65 sec latency 19.805 ms 2024-01-31T05:25:06.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 138689 11.61 MB/sec warmup 66 sec latency 13.803 ms 2024-01-31T05:25:06.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:06 smithi186 ceph-mon[35659]: pgmap v75: 97 pgs: 97 active+clean; 85 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 256 op/s 2024-01-31T05:25:06.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:06 smithi045 ceph-mon[31585]: pgmap v75: 97 pgs: 97 active+clean; 85 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 256 op/s 2024-01-31T05:25:06.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:06 smithi156 ceph-mon[35720]: pgmap v75: 97 pgs: 97 active+clean; 85 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 256 op/s 2024-01-31T05:25:07.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 140178 11.52 MB/sec warmup 67 sec latency 97.567 ms 2024-01-31T05:25:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.316+0000 7fc591c5e640 1 -- 172.21.15.45:0/2977505386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc58c106010 msgr2=0x7fc58c10dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.316+0000 7fc591c5e640 1 --2- 172.21.15.45:0/2977505386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc58c106010 0x7fc58c10dbe0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc584009cd0 tx=0x7fc58402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.316+0000 7fc591c5e640 1 -- 172.21.15.45:0/2977505386 shutdown_connections 2024-01-31T05:25:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.316+0000 7fc591c5e640 1 --2- 172.21.15.45:0/2977505386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc58c106010 0x7fc58c10dbe0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.316+0000 7fc591c5e640 1 --2- 172.21.15.45:0/2977505386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc58c1018f0 0x7fc58c105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.316+0000 7fc591c5e640 1 --2- 172.21.15.45:0/2977505386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c100f40 0x7fc58c101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.316+0000 7fc591c5e640 1 -- 172.21.15.45:0/2977505386 >> 172.21.15.45:0/2977505386 conn(0x7fc58c0fc7a0 msgr2=0x7fc58c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.316+0000 7fc591c5e640 1 -- 172.21.15.45:0/2977505386 shutdown_connections 2024-01-31T05:25:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.316+0000 7fc591c5e640 1 -- 172.21.15.45:0/2977505386 wait complete. 2024-01-31T05:25:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc591c5e640 1 Processor -- start 2024-01-31T05:25:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc591c5e640 1 -- start start 2024-01-31T05:25:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc591c5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc58c100f40 0x7fc58c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc591c5e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c106010 0x7fc58c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc591c5e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc58c19ba70 0x7fc58c19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc591c5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c19c460 con 0x7fc58c100f40 2024-01-31T05:25:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc591c5e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c19c5d0 con 0x7fc58c106010 2024-01-31T05:25:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc591c5e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c19c740 con 0x7fc58c19ba70 2024-01-31T05:25:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc58b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc58c100f40 0x7fc58c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc583fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c106010 0x7fc58c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc58b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc58c100f40 0x7fc58c1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39448/0 (socket says 172.21.15.45:39448) 2024-01-31T05:25:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc583fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c106010 0x7fc58c1976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:60542/0 (socket says 172.21.15.45:60542) 2024-01-31T05:25:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc58b7fe640 1 -- 172.21.15.45:0/4230378635 learned_addr learned my addr 172.21.15.45:0/4230378635 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.317+0000 7fc58bfff640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc58c19ba70 0x7fc58c19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.318+0000 7fc58b7fe640 1 -- 172.21.15.45:0/4230378635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc58c19ba70 msgr2=0x7fc58c19bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.318+0000 7fc58b7fe640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc58c19ba70 0x7fc58c19bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.318+0000 7fc58b7fe640 1 -- 172.21.15.45:0/4230378635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c106010 msgr2=0x7fc58c1976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.318+0000 7fc58b7fe640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c106010 0x7fc58c1976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.318+0000 7fc58b7fe640 1 -- 172.21.15.45:0/4230378635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc584009900 con 0x7fc58c100f40 2024-01-31T05:25:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.318+0000 7fc58b7fe640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc58c100f40 0x7fc58c1971b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00e9f0 tx=0x7fc57c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.318+0000 7fc5897fa640 1 -- 172.21.15.45:0/4230378635 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c009700 con 0x7fc58c100f40 2024-01-31T05:25:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.318+0000 7fc591c5e640 1 -- 172.21.15.45:0/4230378635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58c1a3d20 con 0x7fc58c100f40 2024-01-31T05:25:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.318+0000 7fc5897fa640 1 -- 172.21.15.45:0/4230378635 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc57c00cd80 con 0x7fc58c100f40 2024-01-31T05:25:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.318+0000 7fc5897fa640 1 -- 172.21.15.45:0/4230378635 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c018860 con 0x7fc58c100f40 2024-01-31T05:25:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.318+0000 7fc591c5e640 1 -- 172.21.15.45:0/4230378635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c1a41d0 con 0x7fc58c100f40 2024-01-31T05:25:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.321+0000 7fc5897fa640 1 -- 172.21.15.45:0/4230378635 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fc57c0189c0 con 0x7fc58c100f40 2024-01-31T05:25:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.321+0000 7fc594a29740 1 Processor -- start 2024-01-31T05:25:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.321+0000 7fc594a29740 1 -- start start 2024-01-31T05:25:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc5897fa640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fc5780a14f0 0x7fc5780a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc594a29740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bbb060 0x561f56bbb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc594a29740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 0x561f56bc4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc594a29740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bc4750 0x561f56bc6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc58bfff640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fc5780a14f0 0x7fc5780a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc594a29740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f56b2a040 con 0x561f56bbb060 2024-01-31T05:25:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc5897fa640 1 -- 172.21.15.45:0/4230378635 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc57c014070 con 0x7fc58c100f40 2024-01-31T05:25:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc594a29740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f56b27730 con 0x561f56bc4750 2024-01-31T05:25:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc594a29740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f56b23020 con 0x561f56bbb980 2024-01-31T05:25:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc583fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 0x561f56bc4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc58b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bbb060 0x561f56bbb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc583fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 0x561f56bc4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:57050/0 (socket says 172.21.15.45:57050) 2024-01-31T05:25:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc583fff640 1 -- 172.21.15.45:0/2121717990 learned_addr learned my addr 172.21.15.45:0/2121717990 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc58bfff640 1 --2- 172.21.15.45:0/2121717990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bc4750 0x561f56bc6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc583fff640 1 -- 172.21.15.45:0/2121717990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bc4750 msgr2=0x561f56bc6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc583fff640 1 --2- 172.21.15.45:0/2121717990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bc4750 0x561f56bc6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc583fff640 1 -- 172.21.15.45:0/2121717990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bbb060 msgr2=0x561f56bbb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc583fff640 1 --2- 172.21.15.45:0/2121717990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bbb060 0x561f56bbb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc583fff640 1 -- 172.21.15.45:0/2121717990 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f56b1ae40 con 0x561f56bbb980 2024-01-31T05:25:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc58bfff640 1 --2- 172.21.15.45:0/2121717990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bc4750 0x561f56bc6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:25:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.322+0000 7fc58bfff640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fc5780a14f0 0x7fc5780a39b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc584005cc0 tx=0x7fc584032ed0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.323+0000 7fc583fff640 1 --2- 172.21.15.45:0/2121717990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 0x561f56bc4210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc574009c00 tx=0x7fc574011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a18dacb1b25572b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.323+0000 7fc581ffb640 1 -- 172.21.15.45:0/2121717990 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574004230 con 0x561f56bbb980 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.323+0000 7fc581ffb640 1 -- 172.21.15.45:0/2121717990 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc574009e70 con 0x561f56bbb980 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.323+0000 7fc581ffb640 1 -- 172.21.15.45:0/2121717990 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574019640 con 0x561f56bbb980 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.323+0000 7fc594a29740 1 -- 172.21.15.45:0/2121717990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 msgr2=0x561f56bc4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 --2- 172.21.15.45:0/2121717990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 0x561f56bc4210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc574009c00 tx=0x7fc574011810 comp rx=0 tx=0).stop 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 -- 172.21.15.45:0/2121717990 shutdown_connections 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 --2- 172.21.15.45:0/2121717990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bc4750 0x561f56bc6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 --2- 172.21.15.45:0/2121717990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 0x561f56bc4210 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 --2- 172.21.15.45:0/2121717990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bbb060 0x561f56bbb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 -- 172.21.15.45:0/2121717990 >> 172.21.15.45:0/2121717990 conn(0x561f56bb5980 msgr2=0x561f56bb5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 -- 172.21.15.45:0/2121717990 shutdown_connections 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 -- 172.21.15.45:0/2121717990 wait complete. 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 -- start start 2024-01-31T05:25:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 Processor -- start 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb060 0x561f56bc1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bbb980 0x561f56bc1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bc4750 0x561f56bc20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f56adc1b0 con 0x561f56bbb980 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f56adbe50 con 0x561f56bc4750 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f56aa1090 con 0x561f56bbb060 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.324+0000 7fc594a29740 10 client.0 _ll_register_callbacks cb 0x561f56ac6a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc58b7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb060 0x561f56bc1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc583fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bbb980 0x561f56bc1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc58b7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb060 0x561f56bc1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:57058/0 (socket says 172.21.15.45:57058) 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc58bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bc4750 0x561f56bc20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc58b7fe640 1 -- 172.21.15.45:0/277647622 learned_addr learned my addr 172.21.15.45:0/277647622 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc58b7fe640 1 -- 172.21.15.45:0/277647622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bc4750 msgr2=0x561f56bc20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc58b7fe640 1 --2- 172.21.15.45:0/277647622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bc4750 0x561f56bc20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc58b7fe640 1 -- 172.21.15.45:0/277647622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bbb980 msgr2=0x561f56bc1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc58b7fe640 1 --2- 172.21.15.45:0/277647622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bbb980 0x561f56bc1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc58b7fe640 1 -- 172.21.15.45:0/277647622 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc574009830 con 0x561f56bbb060 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc58b7fe640 1 --2- 172.21.15.45:0/277647622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb060 0x561f56bc1630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc57c096b50 tx=0x7fc57c099b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a60291eb4d7d134 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc56affd640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc56affd640 1 -- 172.21.15.45:0/277647622 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c0a1f30 con 0x561f56bbb060 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc56affd640 1 -- 172.21.15.45:0/277647622 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc57c0a2090 con 0x561f56bbb060 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc594a29740 1 -- 172.21.15.45:0/277647622 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f56b1c010 con 0x561f56bbb060 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc56affd640 1 -- 172.21.15.45:0/277647622 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc57c095ed0 con 0x561f56bbb060 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc594a29740 10 client.25135 fetch_fsmap learned FSMap version 23 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.325+0000 7fc594a29740 1 -- 172.21.15.45:0/277647622 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561f56b216a0 con 0x561f56bbb060 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.326+0000 7fc56affd640 1 -- 172.21.15.45:0/277647622 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f56b1c010 con 0x561f56bbb060 2024-01-31T05:25:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.326+0000 7fc56affd640 1 -- 172.21.15.45:0/277647622 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc57c09eec0 con 0x561f56bbb060 2024-01-31T05:25:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.326+0000 7fc594a29740 10 client.25135 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:25:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.326+0000 7fc594a29740 10 client.25135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:25:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.326+0000 7fc594a29740 20 client.25135 populate_metadata read hostname 'smithi045' 2024-01-31T05:25:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.326+0000 7fc594a29740 1 --2- 172.21.15.45:0/277647622 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f56bd4480 0x561f56bd68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.326+0000 7fc594a29740 4 client.25135 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:25:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.326+0000 7fc594a29740 1 -- 172.21.15.45:0/277647622 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f56b1bd30 con 0x561f56bd4480 2024-01-31T05:25:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.326+0000 7fc583fff640 1 --2- 172.21.15.45:0/277647622 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f56bd4480 0x561f56bd68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc583fff640 1 --2- 172.21.15.45:0/277647622 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f56bd4480 0x561f56bd68a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aab81b018f24289 server_cookie=ced890249c5fbcb6 in_seq=0 out_seq=0 2024-01-31T05:25:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc56affd640 10 client.25135 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc56affd640 1 -- 172.21.15.45:0/277647622 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561f56b1bd30 con 0x561f56bd4480 2024-01-31T05:25:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc56affd640 10 client.25135 handle_command_reply: tid=0 2024-01-31T05:25:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc594a29740 1 client.25135 shutdown 2024-01-31T05:25:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc594a29740 1 -- 172.21.15.45:0/277647622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb060 msgr2=0x561f56bc1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc594a29740 1 --2- 172.21.15.45:0/277647622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb060 0x561f56bc1630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc57c096b50 tx=0x7fc57c099b30 comp rx=0 tx=0).stop 2024-01-31T05:25:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc594a29740 1 -- 172.21.15.45:0/277647622 shutdown_connections 2024-01-31T05:25:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc594a29740 1 --2- 172.21.15.45:0/277647622 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f56bd4480 0x561f56bd68a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc594a29740 1 --2- 172.21.15.45:0/277647622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bbb980 0x561f56bc1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc594a29740 1 --2- 172.21.15.45:0/277647622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bc4750 0x561f56bc20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc594a29740 1 --2- 172.21.15.45:0/277647622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb060 0x561f56bc1630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc56affd640 0 client.25135 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.327+0000 7fc594a29740 1 -- 172.21.15.45:0/277647622 >> 172.21.15.45:0/277647622 conn(0x561f56bb5980 msgr2=0x561f56bb76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.328+0000 7fc594a29740 1 -- 172.21.15.45:0/277647622 shutdown_connections 2024-01-31T05:25:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.328+0000 7fc594a29740 1 -- 172.21.15.45:0/277647622 wait complete. 2024-01-31T05:25:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.328+0000 7fc594a29740 20 client.25135 trim_cache size 0 max 16384 2024-01-31T05:25:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc594a29740 1 Processor -- start 2024-01-31T05:25:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc594a29740 1 -- start start 2024-01-31T05:25:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc594a29740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bbb060 0x561f56bbff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc594a29740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 0x561f56bc0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc594a29740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 0x561f56bbe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc594a29740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f56bb6980 con 0x561f56bc4750 2024-01-31T05:25:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc594a29740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f56bb6b40 con 0x561f56bbb060 2024-01-31T05:25:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc594a29740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f56bc0980 con 0x561f56bbb980 2024-01-31T05:25:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc58bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 0x561f56bbe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc583fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 0x561f56bc0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc583fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 0x561f56bc0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:57062/0 (socket says 172.21.15.45:57062) 2024-01-31T05:25:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc583fff640 1 -- 172.21.15.45:0/2790698563 learned_addr learned my addr 172.21.15.45:0/2790698563 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc58bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 0x561f56bbe890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39480/0 (socket says 172.21.15.45:39480) 2024-01-31T05:25:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.342+0000 7fc58b7fe640 1 --2- 172.21.15.45:0/2790698563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bbb060 0x561f56bbff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.343+0000 7fc583fff640 1 -- 172.21.15.45:0/2790698563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bbb060 msgr2=0x561f56bbff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.343+0000 7fc583fff640 1 --2- 172.21.15.45:0/2790698563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bbb060 0x561f56bbff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.343+0000 7fc583fff640 1 -- 172.21.15.45:0/2790698563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 msgr2=0x561f56bbe890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.343+0000 7fc583fff640 1 --2- 172.21.15.45:0/2790698563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 0x561f56bbe890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.343+0000 7fc583fff640 1 -- 172.21.15.45:0/2790698563 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f56abe030 con 0x561f56bbb980 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.343+0000 7fc583fff640 1 --2- 172.21.15.45:0/2790698563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 0x561f56bc0440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc574009c00 tx=0x7fc574009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d08fe3ffedf714 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.343+0000 7fc5817fa640 1 -- 172.21.15.45:0/2790698563 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574027040 con 0x561f56bbb980 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.343+0000 7fc5817fa640 1 -- 172.21.15.45:0/2790698563 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc574005530 con 0x561f56bbb980 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.343+0000 7fc5817fa640 1 -- 172.21.15.45:0/2790698563 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57402d5c0 con 0x561f56bbb980 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.344+0000 7fc594a29740 1 -- 172.21.15.45:0/2790698563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 msgr2=0x561f56bc0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.344+0000 7fc594a29740 1 --2- 172.21.15.45:0/2790698563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 0x561f56bc0440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc574009c00 tx=0x7fc574009620 comp rx=0 tx=0).stop 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.344+0000 7fc594a29740 1 -- 172.21.15.45:0/2790698563 shutdown_connections 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.344+0000 7fc594a29740 1 --2- 172.21.15.45:0/2790698563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 0x561f56bbe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.344+0000 7fc594a29740 1 --2- 172.21.15.45:0/2790698563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb980 0x561f56bc0440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.344+0000 7fc594a29740 1 --2- 172.21.15.45:0/2790698563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bbb060 0x561f56bbff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.344+0000 7fc594a29740 1 -- 172.21.15.45:0/2790698563 >> 172.21.15.45:0/2790698563 conn(0x561f56bb5980 msgr2=0x561f56bcfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.344+0000 7fc594a29740 1 -- 172.21.15.45:0/2790698563 shutdown_connections 2024-01-31T05:25:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.344+0000 7fc594a29740 1 -- 172.21.15.45:0/2790698563 wait complete. 2024-01-31T05:25:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.344+0000 7fc594a29740 1 -- start start 2024-01-31T05:25:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc594a29740 1 Processor -- start 2024-01-31T05:25:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc594a29740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb060 0x561f56bcb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc594a29740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bbb980 0x561f56bc9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc594a29740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 0x561f56bca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc594a29740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f56bcac60 con 0x561f56bc4750 2024-01-31T05:25:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc594a29740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f56c2a9d0 con 0x561f56bbb980 2024-01-31T05:25:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc594a29740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f56c2ab10 con 0x561f56bbb060 2024-01-31T05:25:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc594a29740 10 client.0 _ll_register_callbacks cb 0x7fc574009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:25:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc58bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 0x561f56bca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc58bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 0x561f56bca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39484/0 (socket says 172.21.15.45:39484) 2024-01-31T05:25:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc58b7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb060 0x561f56bcb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc58bfff640 1 -- 172.21.15.45:0/716444772 learned_addr learned my addr 172.21.15.45:0/716444772 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc583fff640 1 --2- 172.21.15.45:0/716444772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bbb980 0x561f56bc9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc58bfff640 1 -- 172.21.15.45:0/716444772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb060 msgr2=0x561f56bcb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc58bfff640 1 --2- 172.21.15.45:0/716444772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb060 0x561f56bcb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc58bfff640 1 -- 172.21.15.45:0/716444772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bbb980 msgr2=0x561f56bc9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc58bfff640 1 --2- 172.21.15.45:0/716444772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bbb980 0x561f56bc9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.345+0000 7fc58bfff640 1 -- 172.21.15.45:0/716444772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f56b1ae40 con 0x561f56bc4750 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc58bfff640 1 --2- 172.21.15.45:0/716444772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 0x561f56bca590 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc584009cd0 tx=0x7fc58402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faac1512048a2077 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc56affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc56affd640 1 -- 172.21.15.45:0/716444772 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584046040 con 0x561f56bc4750 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc56affd640 1 -- 172.21.15.45:0/716444772 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc584034e20 con 0x561f56bc4750 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc594a29740 1 -- 172.21.15.45:0/716444772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc574009e70 con 0x561f56bc4750 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc56affd640 1 -- 172.21.15.45:0/716444772 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc584003d80 con 0x561f56bc4750 2024-01-31T05:25:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc594a29740 10 client.15591 fetch_fsmap learned FSMap version 23 2024-01-31T05:25:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc594a29740 1 -- 172.21.15.45:0/716444772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561f56ae3a50 con 0x561f56bc4750 2024-01-31T05:25:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc56affd640 1 -- 172.21.15.45:0/716444772 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc574009e70 con 0x561f56bc4750 2024-01-31T05:25:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc56affd640 1 -- 172.21.15.45:0/716444772 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc58404c040 con 0x561f56bc4750 2024-01-31T05:25:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc594a29740 10 client.15591 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:25:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc594a29740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:25:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc594a29740 20 client.15591 populate_metadata read hostname 'smithi045' 2024-01-31T05:25:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc594a29740 1 --2- 172.21.15.45:0/716444772 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f56c2c2f0 0x561f56c2e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc594a29740 4 client.15591 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:25:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc594a29740 1 -- 172.21.15.45:0/716444772 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f56822600 con 0x561f56c2c2f0 2024-01-31T05:25:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.346+0000 7fc58b7fe640 1 --2- 172.21.15.45:0/716444772 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f56c2c2f0 0x561f56c2e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.347+0000 7fc58b7fe640 1 --2- 172.21.15.45:0/716444772 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f56c2c2f0 0x561f56c2e710 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d331d2fe8b62a230 server_cookie=1063d23cd1704f19 in_seq=0 out_seq=0 2024-01-31T05:25:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.347+0000 7fc56affd640 10 client.15591 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.354+0000 7fc56affd640 1 -- 172.21.15.45:0/716444772 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f56822600 con 0x561f56c2c2f0 2024-01-31T05:25:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.354+0000 7fc56affd640 10 client.15591 handle_command_reply: tid=0 2024-01-31T05:25:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.354+0000 7fc594a29740 1 client.15591 shutdown 2024-01-31T05:25:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.354+0000 7fc594a29740 1 -- 172.21.15.45:0/716444772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 msgr2=0x561f56bca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.354+0000 7fc594a29740 1 --2- 172.21.15.45:0/716444772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 0x561f56bca590 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc584009cd0 tx=0x7fc58402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.354+0000 7fc594a29740 1 -- 172.21.15.45:0/716444772 shutdown_connections 2024-01-31T05:25:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.354+0000 7fc594a29740 1 --2- 172.21.15.45:0/716444772 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f56c2c2f0 0x561f56c2e710 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.354+0000 7fc594a29740 1 --2- 172.21.15.45:0/716444772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f56bc4750 0x561f56bca590 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.354+0000 7fc594a29740 1 --2- 172.21.15.45:0/716444772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f56bbb980 0x561f56bc9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.354+0000 7fc56affd640 0 client.15591 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.354+0000 7fc594a29740 1 --2- 172.21.15.45:0/716444772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f56bbb060 0x561f56bcb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.354+0000 7fc594a29740 1 -- 172.21.15.45:0/716444772 >> 172.21.15.45:0/716444772 conn(0x561f56bb5980 msgr2=0x561f56bcfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc594a29740 1 -- 172.21.15.45:0/716444772 shutdown_connections 2024-01-31T05:25:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc594a29740 1 -- 172.21.15.45:0/716444772 wait complete. 2024-01-31T05:25:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc594a29740 20 client.15591 trim_cache size 0 max 16384 2024-01-31T05:25:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:25:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:25:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:25:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:25:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc591c5e640 1 -- 172.21.15.45:0/4230378635 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fc5780a14f0 msgr2=0x7fc5780a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc591c5e640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fc5780a14f0 0x7fc5780a39b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc584005cc0 tx=0x7fc584032ed0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc591c5e640 1 -- 172.21.15.45:0/4230378635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc58c100f40 msgr2=0x7fc58c1971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc591c5e640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc58c100f40 0x7fc58c1971b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00e9f0 tx=0x7fc57c00eeb0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc591c5e640 1 -- 172.21.15.45:0/4230378635 shutdown_connections 2024-01-31T05:25:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc591c5e640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fc5780a14f0 0x7fc5780a39b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc591c5e640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc58c19ba70 0x7fc58c19bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc591c5e640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc58c106010 0x7fc58c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc591c5e640 1 --2- 172.21.15.45:0/4230378635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc58c100f40 0x7fc58c1971b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc591c5e640 1 -- 172.21.15.45:0/4230378635 >> 172.21.15.45:0/4230378635 conn(0x7fc58c0fc7a0 msgr2=0x7fc58c0fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc591c5e640 1 -- 172.21.15.45:0/4230378635 shutdown_connections 2024-01-31T05:25:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:08.355+0000 7fc591c5e640 1 -- 172.21.15.45:0/4230378635 wait complete. 2024-01-31T05:25:08.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 142495 11.56 MB/sec warmup 68 sec latency 8.651 ms 2024-01-31T05:25:08.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:08 smithi186 ceph-mon[35659]: pgmap v76: 97 pgs: 97 active+clean; 90 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 375 op/s 2024-01-31T05:25:08.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:08 smithi045 ceph-mon[31585]: pgmap v76: 97 pgs: 97 active+clean; 90 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 375 op/s 2024-01-31T05:25:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:08 smithi156 ceph-mon[35720]: pgmap v76: 97 pgs: 97 active+clean; 90 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 375 op/s 2024-01-31T05:25:09.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 145576 11.65 MB/sec warmup 69 sec latency 7.083 ms 2024-01-31T05:25:10.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 147951 11.61 MB/sec warmup 70 sec latency 15.086 ms 2024-01-31T05:25:10.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:10 smithi186 ceph-mon[35659]: pgmap v77: 97 pgs: 97 active+clean; 90 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2024-01-31T05:25:10.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:10 smithi045 ceph-mon[31585]: pgmap v77: 97 pgs: 97 active+clean; 90 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2024-01-31T05:25:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:10 smithi156 ceph-mon[35720]: pgmap v77: 97 pgs: 97 active+clean; 90 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2024-01-31T05:25:11.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 150155 11.64 MB/sec warmup 71 sec latency 18.364 ms 2024-01-31T05:25:12.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 152680 11.69 MB/sec warmup 72 sec latency 116.777 ms 2024-01-31T05:25:12.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:12 smithi186 ceph-mon[35659]: pgmap v78: 97 pgs: 97 active+clean; 90 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2024-01-31T05:25:12.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:12 smithi045 ceph-mon[31585]: pgmap v78: 97 pgs: 97 active+clean; 90 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2024-01-31T05:25:12.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:12 smithi156 ceph-mon[35720]: pgmap v78: 97 pgs: 97 active+clean; 90 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2024-01-31T05:25:13.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 155103 11.66 MB/sec warmup 73 sec latency 27.929 ms 2024-01-31T05:25:14.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 157314 11.68 MB/sec warmup 74 sec latency 16.589 ms 2024-01-31T05:25:14.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:14 smithi186 ceph-mon[35659]: pgmap v79: 97 pgs: 97 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 379 op/s 2024-01-31T05:25:14.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:14 smithi045 ceph-mon[31585]: pgmap v79: 97 pgs: 97 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 379 op/s 2024-01-31T05:25:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:14 smithi156 ceph-mon[35720]: pgmap v79: 97 pgs: 97 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 379 op/s 2024-01-31T05:25:15.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 160095 11.75 MB/sec warmup 75 sec latency 13.478 ms 2024-01-31T05:25:16.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 163131 11.83 MB/sec warmup 76 sec latency 8.784 ms 2024-01-31T05:25:16.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:16 smithi186 ceph-mon[35659]: pgmap v80: 97 pgs: 97 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 228 op/s 2024-01-31T05:25:16.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:16 smithi045 ceph-mon[31585]: pgmap v80: 97 pgs: 97 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 228 op/s 2024-01-31T05:25:16.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:16 smithi156 ceph-mon[35720]: pgmap v80: 97 pgs: 97 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 228 op/s 2024-01-31T05:25:17.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 164792 11.76 MB/sec warmup 77 sec latency 153.903 ms 2024-01-31T05:25:18.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 167119 11.79 MB/sec warmup 78 sec latency 208.802 ms 2024-01-31T05:25:18.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:18 smithi186 ceph-mon[35659]: pgmap v81: 97 pgs: 97 active+clean; 91 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.9 MiB/s wr, 345 op/s 2024-01-31T05:25:18.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:18 smithi045 ceph-mon[31585]: pgmap v81: 97 pgs: 97 active+clean; 91 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.9 MiB/s wr, 345 op/s 2024-01-31T05:25:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:18 smithi156 ceph-mon[35720]: pgmap v81: 97 pgs: 97 active+clean; 91 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.9 MiB/s wr, 345 op/s 2024-01-31T05:25:19.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 170025 11.82 MB/sec warmup 79 sec latency 21.279 ms 2024-01-31T05:25:20.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 172414 11.81 MB/sec warmup 80 sec latency 9.502 ms 2024-01-31T05:25:20.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:20 smithi186 ceph-mon[35659]: pgmap v82: 97 pgs: 97 active+clean; 91 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-01-31T05:25:20.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:20 smithi045 ceph-mon[31585]: pgmap v82: 97 pgs: 97 active+clean; 91 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-01-31T05:25:20.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:20 smithi156 ceph-mon[35720]: pgmap v82: 97 pgs: 97 active+clean; 91 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-01-31T05:25:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.444+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3635783872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58106000 msgr2=0x7f0f5810dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.444+0000 7f0f5ed24640 1 --2- 172.21.15.45:0/3635783872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58106000 0x7f0f5810dbd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009cd0 tx=0x7f0f5002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.444+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3635783872 shutdown_connections 2024-01-31T05:25:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.444+0000 7f0f5ed24640 1 --2- 172.21.15.45:0/3635783872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58106000 0x7f0f5810dbd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.444+0000 7f0f5ed24640 1 --2- 172.21.15.45:0/3635783872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f581018e0 0x7f0f58105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.444+0000 7f0f5ed24640 1 --2- 172.21.15.45:0/3635783872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f58100f30 0x7f0f58101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.444+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3635783872 >> 172.21.15.45:0/3635783872 conn(0x7f0f580fc7d0 msgr2=0x7f0f580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.444+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3635783872 shutdown_connections 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.444+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3635783872 wait complete. 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f5ed24640 1 Processor -- start 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f5ed24640 1 -- start start 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f5ed24640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f58100f30 0x7f0f581971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f5ed24640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f58106000 0x7f0f58197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f5ed24640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f5819ba80 0x7f0f5819bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f5ed24640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5819c470 con 0x7f0f5819ba80 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f5ed24640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5819c5e0 con 0x7f0f58100f30 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f5ed24640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5819c750 con 0x7f0f58106000 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f5d2c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f5819ba80 0x7f0f5819bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f5cac0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f58100f30 0x7f0f581971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f5d2c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f5819ba80 0x7f0f5819bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40120/0 (socket says 172.21.15.45:40120) 2024-01-31T05:25:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f5d2c1640 1 -- 172.21.15.45:0/3698840663 learned_addr learned my addr 172.21.15.45:0/3698840663 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.445+0000 7f0f57fff640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f58106000 0x7f0f58197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.446+0000 7f0f5d2c1640 1 -- 172.21.15.45:0/3698840663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f58106000 msgr2=0x7f0f58197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.446+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f58106000 0x7f0f58197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.446+0000 7f0f5d2c1640 1 -- 172.21.15.45:0/3698840663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f58100f30 msgr2=0x7f0f581971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.446+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f58100f30 0x7f0f581971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.446+0000 7f0f5d2c1640 1 -- 172.21.15.45:0/3698840663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f50009900 con 0x7f0f5819ba80 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.446+0000 7f0f5cac0640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f58100f30 0x7f0f581971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.446+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f5819ba80 0x7f0f5819bf30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009ca0 tx=0x7f0f50032550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.446+0000 7f0f55ffb640 1 -- 172.21.15.45:0/3698840663 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50032690 con 0x7f0f5819ba80 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.446+0000 7f0f55ffb640 1 -- 172.21.15.45:0/3698840663 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f50003970 con 0x7f0f5819ba80 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.446+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3698840663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f58072f20 con 0x7f0f5819ba80 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.446+0000 7f0f55ffb640 1 -- 172.21.15.45:0/3698840663 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f500045e0 con 0x7f0f5819ba80 2024-01-31T05:25:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.446+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3698840663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f580733d0 con 0x7f0f5819ba80 2024-01-31T05:25:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.448+0000 7f0f55ffb640 1 -- 172.21.15.45:0/3698840663 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f0f5003b040 con 0x7f0f5819ba80 2024-01-31T05:25:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f61bc0740 1 Processor -- start 2024-01-31T05:25:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f61bc0740 1 -- start start 2024-01-31T05:25:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f61bc0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142060 0x55ad80142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f55ffb640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0f480a14f0 0x7f0f480a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f61bc0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 0x55ad8014b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f61bc0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad8014b750 0x55ad8014db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f61bc0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad800b1040 con 0x55ad80142060 2024-01-31T05:25:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f61bc0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad800ae730 con 0x55ad8014b750 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f61bc0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad800aa020 con 0x55ad80142980 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f57fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142060 0x55ad80142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f5cac0640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0f480a14f0 0x7f0f480a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f57fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142060 0x55ad80142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40132/0 (socket says 172.21.15.45:40132) 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f57fff640 1 -- 172.21.15.45:0/974177290 learned_addr learned my addr 172.21.15.45:0/974177290 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f55ffb640 1 -- 172.21.15.45:0/3698840663 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0f500ef780 con 0x7f0f5819ba80 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/974177290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad8014b750 0x55ad8014db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.449+0000 7f0f57fff640 1 --2- 172.21.15.45:0/974177290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 0x55ad8014b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.450+0000 7f0f57fff640 1 -- 172.21.15.45:0/974177290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 msgr2=0x55ad8014b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.450+0000 7f0f57fff640 1 --2- 172.21.15.45:0/974177290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 0x55ad8014b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.450+0000 7f0f57fff640 1 -- 172.21.15.45:0/974177290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad8014b750 msgr2=0x55ad8014db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.450+0000 7f0f57fff640 1 --2- 172.21.15.45:0/974177290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad8014b750 0x55ad8014db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.450+0000 7f0f57fff640 1 -- 172.21.15.45:0/974177290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad800a1e40 con 0x55ad80142060 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.450+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/974177290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad8014b750 0x55ad8014db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.450+0000 7f0f5cac0640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0f480a14f0 0x7f0f480a39b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009fc0 tx=0x7f0f4c005f90 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.450+0000 7f0f57fff640 1 --2- 172.21.15.45:0/974177290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142060 0x55ad80142440 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009fd0 tx=0x7f0f44005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb885c357cb8044 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.450+0000 7f0f42ffd640 1 -- 172.21.15.45:0/974177290 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44013130 con 0x55ad80142060 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.450+0000 7f0f42ffd640 1 -- 172.21.15.45:0/974177290 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f44013290 con 0x55ad80142060 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.450+0000 7f0f42ffd640 1 -- 172.21.15.45:0/974177290 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44004ad0 con 0x55ad80142060 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.451+0000 7f0f61bc0740 1 -- 172.21.15.45:0/974177290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142060 msgr2=0x55ad80142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.451+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/974177290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142060 0x55ad80142440 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009fd0 tx=0x7f0f44005f90 comp rx=0 tx=0).stop 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.451+0000 7f0f61bc0740 1 -- 172.21.15.45:0/974177290 shutdown_connections 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.451+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/974177290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad8014b750 0x55ad8014db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.451+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/974177290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 0x55ad8014b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.451+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/974177290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142060 0x55ad80142440 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.451+0000 7f0f61bc0740 1 -- 172.21.15.45:0/974177290 >> 172.21.15.45:0/974177290 conn(0x55ad8013c980 msgr2=0x55ad8013cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.451+0000 7f0f61bc0740 1 -- 172.21.15.45:0/974177290 shutdown_connections 2024-01-31T05:25:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.451+0000 7f0f61bc0740 1 -- 172.21.15.45:0/974177290 wait complete. 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.451+0000 7f0f61bc0740 1 -- start start 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.451+0000 7f0f61bc0740 1 Processor -- start 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f61bc0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142060 0x55ad801486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f61bc0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142980 0x55ad80148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f61bc0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad8014b750 0x55ad80149130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f61bc0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad800631b0 con 0x55ad80142980 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f61bc0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad80062e50 con 0x55ad8014b750 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f61bc0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad80028090 con 0x55ad80142060 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f61bc0740 10 client.0 _ll_register_callbacks cb 0x55ad8004da90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f5cac0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142060 0x55ad801486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f57fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142980 0x55ad80148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f5cac0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142060 0x55ad801486b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:46080/0 (socket says 172.21.15.45:46080) 2024-01-31T05:25:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f5cac0640 1 -- 172.21.15.45:0/2759702086 learned_addr learned my addr 172.21.15.45:0/2759702086 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad8014b750 0x55ad80149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f57fff640 1 -- 172.21.15.45:0/2759702086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142060 msgr2=0x55ad801486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f57fff640 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142060 0x55ad801486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f57fff640 1 -- 172.21.15.45:0/2759702086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad8014b750 msgr2=0x55ad80149130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f57fff640 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad8014b750 0x55ad80149130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f57fff640 1 -- 172.21.15.45:0/2759702086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4401b040 con 0x55ad80142980 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f5cac0640 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142060 0x55ad801486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad8014b750 0x55ad80149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f57fff640 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142980 0x55ad80148bf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0f44013000 tx=0x7f0f44005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9b3467dcb12aaa server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f37fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f37fff640 1 -- 172.21.15.45:0/2759702086 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44013cb0 con 0x55ad80142980 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f37fff640 1 -- 172.21.15.45:0/2759702086 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0f44005c50 con 0x55ad80142980 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.452+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2759702086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad800a3010 con 0x55ad80142980 2024-01-31T05:25:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.453+0000 7f0f37fff640 1 -- 172.21.15.45:0/2759702086 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f44004a20 con 0x55ad80142980 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.453+0000 7f0f61bc0740 10 client.15603 fetch_fsmap learned FSMap version 23 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.453+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2759702086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ad800a86a0 con 0x55ad80142980 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.453+0000 7f0f37fff640 1 -- 172.21.15.45:0/2759702086 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad800a3010 con 0x55ad80142980 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.453+0000 7f0f37fff640 1 -- 172.21.15.45:0/2759702086 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0f44020440 con 0x55ad80142980 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.453+0000 7f0f61bc0740 10 client.15603 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.453+0000 7f0f61bc0740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.453+0000 7f0f61bc0740 20 client.15603 populate_metadata read hostname 'smithi045' 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.453+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ad8015b380 0x55ad8015d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.453+0000 7f0f61bc0740 4 client.15603 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.453+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2759702086 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad800a2d30 con 0x55ad8015b380 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.453+0000 7f0f5cac0640 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ad8015b380 0x55ad8015d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f5cac0640 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ad8015b380 0x55ad8015d7a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aec75bea89cde9c server_cookie=9c376aa38398e98b in_seq=0 out_seq=0 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f37fff640 10 client.15603 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f37fff640 1 -- 172.21.15.45:0/2759702086 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad800a2d30 con 0x55ad8015b380 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f37fff640 10 client.15603 handle_command_reply: tid=0 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f61bc0740 1 client.15603 shutdown 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2759702086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142980 msgr2=0x55ad80148bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142980 0x55ad80148bf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0f44013000 tx=0x7f0f44005690 comp rx=0 tx=0).stop 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2759702086 shutdown_connections 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ad8015b380 0x55ad8015d7a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad80142980 0x55ad80148bf0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad8014b750 0x55ad80149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f37fff640 0 client.15603 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/2759702086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142060 0x55ad801486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.454+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2759702086 >> 172.21.15.45:0/2759702086 conn(0x55ad8013c980 msgr2=0x55ad8013e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.455+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2759702086 shutdown_connections 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.455+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2759702086 wait complete. 2024-01-31T05:25:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.455+0000 7f0f61bc0740 20 client.15603 trim_cache size 0 max 16384 2024-01-31T05:25:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f61bc0740 1 Processor -- start 2024-01-31T05:25:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f61bc0740 1 -- start start 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f61bc0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 0x55ad8014ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f61bc0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 0x55ad80148f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f61bc0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 0x55ad80146f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f61bc0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad800a2d30 con 0x55ad8014b750 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f61bc0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad800a3010 con 0x55ad80142060 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f61bc0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad80141340 con 0x55ad80142980 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f5d2c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 0x55ad80146f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f5d2c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 0x55ad80146f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40154/0 (socket says 172.21.15.45:40154) 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f5d2c1640 1 -- 172.21.15.45:0/2245438532 learned_addr learned my addr 172.21.15.45:0/2245438532 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f57fff640 1 --2- 172.21.15.45:0/2245438532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 0x55ad80148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f5cac0640 1 --2- 172.21.15.45:0/2245438532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 0x55ad8014ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f5d2c1640 1 -- 172.21.15.45:0/2245438532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 msgr2=0x55ad80148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/2245438532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 0x55ad80148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f5d2c1640 1 -- 172.21.15.45:0/2245438532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 msgr2=0x55ad8014ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/2245438532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 0x55ad8014ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f5d2c1640 1 -- 172.21.15.45:0/2245438532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad80045030 con 0x55ad8014b750 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.468+0000 7f0f5cac0640 1 --2- 172.21.15.45:0/2245438532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 0x55ad8014ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/2245438532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 0x55ad80146f80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0f50003fd0 tx=0x7f0f500a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f896400ebf8d48 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f427fc640 1 -- 172.21.15.45:0/2245438532 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5009b340 con 0x55ad8014b750 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f427fc640 1 -- 172.21.15.45:0/2245438532 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f500bcde0 con 0x55ad8014b750 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f427fc640 1 -- 172.21.15.45:0/2245438532 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f500b8df0 con 0x55ad8014b750 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2245438532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 msgr2=0x55ad80146f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/2245438532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 0x55ad80146f80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0f50003fd0 tx=0x7f0f500a7040 comp rx=0 tx=0).stop 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2245438532 shutdown_connections 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/2245438532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 0x55ad80146f80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/2245438532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 0x55ad80148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/2245438532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 0x55ad8014ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2245438532 >> 172.21.15.45:0/2245438532 conn(0x55ad8013c980 msgr2=0x55ad80143990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2245438532 shutdown_connections 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.469+0000 7f0f61bc0740 1 -- 172.21.15.45:0/2245438532 wait complete. 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f61bc0740 1 -- start start 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f61bc0740 1 Processor -- start 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f61bc0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 0x55ad80150610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f61bc0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 0x55ad80150b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f61bc0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 0x55ad801b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f61bc0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad801b0de0 con 0x55ad8014b750 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f61bc0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad801b0f20 con 0x55ad80142060 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f61bc0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad801b1090 con 0x55ad80142980 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f61bc0740 10 client.0 _ll_register_callbacks cb 0x7f0f44009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f5d2c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 0x55ad801b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f5cac0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 0x55ad80150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f5d2c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 0x55ad801b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40170/0 (socket says 172.21.15.45:40170) 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f5cac0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 0x55ad80150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:49744/0 (socket says 172.21.15.45:49744) 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f5d2c1640 1 -- 172.21.15.45:0/229614524 learned_addr learned my addr 172.21.15.45:0/229614524 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.470+0000 7f0f57fff640 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 0x55ad80150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f5d2c1640 1 -- 172.21.15.45:0/229614524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 msgr2=0x55ad80150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 0x55ad80150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f5d2c1640 1 -- 172.21.15.45:0/229614524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 msgr2=0x55ad80150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 0x55ad80150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f5d2c1640 1 -- 172.21.15.45:0/229614524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f58072f20 con 0x55ad8014b750 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f5cac0640 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 0x55ad80150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f5d2c1640 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 0x55ad801b08a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0f500b8be0 tx=0x7f0f50035f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8893042eb740ad server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f37fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f37fff640 1 -- 172.21.15.45:0/229614524 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f500bc6e0 con 0x55ad8014b750 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f37fff640 1 -- 172.21.15.45:0/229614524 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0f500b8df0 con 0x55ad8014b750 2024-01-31T05:25:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f61bc0740 1 -- 172.21.15.45:0/229614524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad80138080 con 0x55ad8014b750 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f37fff640 1 -- 172.21.15.45:0/229614524 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f500b8fd0 con 0x55ad8014b750 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f61bc0740 10 client.15609 fetch_fsmap learned FSMap version 23 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f61bc0740 1 -- 172.21.15.45:0/229614524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ad8006aa50 con 0x55ad8014b750 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.471+0000 7f0f37fff640 1 -- 172.21.15.45:0/229614524 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad80138080 con 0x55ad8014b750 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.472+0000 7f0f37fff640 1 -- 172.21.15.45:0/229614524 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0f500bbbd0 con 0x55ad8014b750 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.472+0000 7f0f61bc0740 10 client.15609 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.472+0000 7f0f61bc0740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.472+0000 7f0f61bc0740 20 client.15609 populate_metadata read hostname 'smithi045' 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.472+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ad801b23f0 0x55ad801b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.472+0000 7f0f61bc0740 4 client.15609 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.472+0000 7f0f61bc0740 1 -- 172.21.15.45:0/229614524 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f0f44013290 con 0x55ad801b23f0 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.472+0000 7f0f5cac0640 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ad801b23f0 0x55ad801b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.472+0000 7f0f37fff640 10 client.15609 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.472+0000 7f0f5cac0640 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ad801b23f0 0x55ad801b47e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a1d7d8fa4083a server_cookie=3a0c55b9976f72f6 in_seq=0 out_seq=0 2024-01-31T05:25:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f37fff640 1 -- 172.21.15.45:0/229614524 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f0f44013290 con 0x55ad801b23f0 2024-01-31T05:25:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f37fff640 10 client.15609 handle_command_reply: tid=0 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f61bc0740 1 client.15609 shutdown 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f61bc0740 1 -- 172.21.15.45:0/229614524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 msgr2=0x55ad801b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 0x55ad801b08a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0f500b8be0 tx=0x7f0f50035f90 comp rx=0 tx=0).stop 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f61bc0740 1 -- 172.21.15.45:0/229614524 shutdown_connections 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ad801b23f0 0x55ad801b47e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8014b750 0x55ad801b08a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f37fff640 0 client.15609 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad80142980 0x55ad80150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f61bc0740 1 --2- 172.21.15.45:0/229614524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad80142060 0x55ad80150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f61bc0740 1 -- 172.21.15.45:0/229614524 >> 172.21.15.45:0/229614524 conn(0x55ad8013c980 msgr2=0x55ad80146ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f61bc0740 1 -- 172.21.15.45:0/229614524 shutdown_connections 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f61bc0740 1 -- 172.21.15.45:0/229614524 wait complete. 2024-01-31T05:25:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f61bc0740 20 client.15609 trim_cache size 0 max 16384 2024-01-31T05:25:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:25:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:25:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:25:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:25:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.480+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3698840663 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0f480a14f0 msgr2=0x7f0f480a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.481+0000 7f0f5ed24640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0f480a14f0 0x7f0f480a39b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009fc0 tx=0x7f0f4c005f90 comp rx=0 tx=0).stop 2024-01-31T05:25:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.481+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3698840663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f5819ba80 msgr2=0x7f0f5819bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.481+0000 7f0f5ed24640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f5819ba80 0x7f0f5819bf30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009ca0 tx=0x7f0f50032550 comp rx=0 tx=0).stop 2024-01-31T05:25:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.481+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3698840663 shutdown_connections 2024-01-31T05:25:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.481+0000 7f0f5ed24640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0f480a14f0 0x7f0f480a39b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.481+0000 7f0f5ed24640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f5819ba80 0x7f0f5819bf30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.481+0000 7f0f5ed24640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f58106000 0x7f0f58197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.481+0000 7f0f5ed24640 1 --2- 172.21.15.45:0/3698840663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f58100f30 0x7f0f581971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.481+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3698840663 >> 172.21.15.45:0/3698840663 conn(0x7f0f580fc7d0 msgr2=0x7f0f580fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.481+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3698840663 shutdown_connections 2024-01-31T05:25:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:21.481+0000 7f0f5ed24640 1 -- 172.21.15.45:0/3698840663 wait complete. 2024-01-31T05:25:21.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 174591 11.83 MB/sec warmup 81 sec latency 10.479 ms 2024-01-31T05:25:22.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 177608 11.91 MB/sec warmup 82 sec latency 29.533 ms 2024-01-31T05:25:22.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:22 smithi186 ceph-mon[35659]: pgmap v83: 97 pgs: 97 active+clean; 91 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-01-31T05:25:22.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:22 smithi045 ceph-mon[31585]: pgmap v83: 97 pgs: 97 active+clean; 91 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-01-31T05:25:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:22 smithi156 ceph-mon[35720]: pgmap v83: 97 pgs: 97 active+clean; 91 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 226 op/s 2024-01-31T05:25:23.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 179743 11.86 MB/sec warmup 83 sec latency 182.605 ms 2024-01-31T05:25:24.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 181652 11.86 MB/sec warmup 84 sec latency 15.547 ms 2024-01-31T05:25:24.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:24 smithi186 ceph-mon[35659]: pgmap v84: 97 pgs: 97 active+clean; 96 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 346 op/s 2024-01-31T05:25:24.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:24 smithi045 ceph-mon[31585]: pgmap v84: 97 pgs: 97 active+clean; 96 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 346 op/s 2024-01-31T05:25:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:24 smithi156 ceph-mon[35720]: pgmap v84: 97 pgs: 97 active+clean; 96 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 346 op/s 2024-01-31T05:25:25.774 INFO:tasks.workunit.client.0.smithi045.stdout: 1 184670 11.94 MB/sec warmup 85 sec latency 14.890 ms 2024-01-31T05:25:26.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 187037 11.90 MB/sec warmup 86 sec latency 17.009 ms 2024-01-31T05:25:26.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:26 smithi186 ceph-mon[35659]: pgmap v85: 97 pgs: 97 active+clean; 96 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-01-31T05:25:26.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:26 smithi045 ceph-mon[31585]: pgmap v85: 97 pgs: 97 active+clean; 96 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-01-31T05:25:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:26 smithi156 ceph-mon[35720]: pgmap v85: 97 pgs: 97 active+clean; 96 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-01-31T05:25:27.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 189141 11.91 MB/sec warmup 87 sec latency 13.067 ms 2024-01-31T05:25:28.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 191776 11.97 MB/sec warmup 88 sec latency 85.793 ms 2024-01-31T05:25:28.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:28 smithi186 ceph-mon[35659]: pgmap v86: 97 pgs: 97 active+clean; 101 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 348 op/s 2024-01-31T05:25:28.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:28 smithi045 ceph-mon[31585]: pgmap v86: 97 pgs: 97 active+clean; 101 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 348 op/s 2024-01-31T05:25:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:28 smithi156 ceph-mon[35720]: pgmap v86: 97 pgs: 97 active+clean; 101 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 348 op/s 2024-01-31T05:25:29.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 193883 11.92 MB/sec warmup 89 sec latency 96.307 ms 2024-01-31T05:25:30.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 195968 11.93 MB/sec warmup 90 sec latency 11.910 ms 2024-01-31T05:25:30.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:30 smithi186 ceph-mon[35659]: pgmap v87: 97 pgs: 97 active+clean; 101 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-01-31T05:25:30.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:30 smithi045 ceph-mon[31585]: pgmap v87: 97 pgs: 97 active+clean; 101 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-01-31T05:25:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:30 smithi156 ceph-mon[35720]: pgmap v87: 97 pgs: 97 active+clean; 101 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-01-31T05:25:31.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 198955 11.99 MB/sec warmup 91 sec latency 11.087 ms 2024-01-31T05:25:32.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 201826 12.01 MB/sec warmup 92 sec latency 13.006 ms 2024-01-31T05:25:32.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:32 smithi186 ceph-mon[35659]: pgmap v88: 97 pgs: 97 active+clean; 101 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-01-31T05:25:32.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:32 smithi045 ceph-mon[31585]: pgmap v88: 97 pgs: 97 active+clean; 101 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-01-31T05:25:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:32 smithi156 ceph-mon[35720]: pgmap v88: 97 pgs: 97 active+clean; 101 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-01-31T05:25:33.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 203369 11.97 MB/sec warmup 93 sec latency 78.693 ms 2024-01-31T05:25:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.569+0000 7faad0932640 1 -- 172.21.15.45:0/2506721228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faacc106070 msgr2=0x7faacc10dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.569+0000 7faad0932640 1 --2- 172.21.15.45:0/2506721228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faacc106070 0x7faacc10dbf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7faac4009cd0 tx=0x7faac402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.569+0000 7faad0932640 1 -- 172.21.15.45:0/2506721228 shutdown_connections 2024-01-31T05:25:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.569+0000 7faad0932640 1 --2- 172.21.15.45:0/2506721228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faacc106070 0x7faacc10dbf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.569+0000 7faad0932640 1 --2- 172.21.15.45:0/2506721228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faacc0ffdb0 0x7faacc105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.569+0000 7faad0932640 1 --2- 172.21.15.45:0/2506721228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faacc0ff400 0x7faacc0ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.569+0000 7faad0932640 1 -- 172.21.15.45:0/2506721228 >> 172.21.15.45:0/2506721228 conn(0x7faacc076110 msgr2=0x7faacc0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.569+0000 7faad0932640 1 -- 172.21.15.45:0/2506721228 shutdown_connections 2024-01-31T05:25:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.569+0000 7faad0932640 1 -- 172.21.15.45:0/2506721228 wait complete. 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faad0932640 1 Processor -- start 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faad0932640 1 -- start start 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faad0932640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faacc0ff400 0x7faacc197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faad0932640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faacc106070 0x7faacc197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faad0932640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faacc19ba70 0x7faacc19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faad0932640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc19c460 con 0x7faacc106070 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faad0932640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc19c5d0 con 0x7faacc0ff400 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faad0932640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc19c740 con 0x7faacc19ba70 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faac3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faacc106070 0x7faacc197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faac3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faacc106070 0x7faacc197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57168/0 (socket says 172.21.15.45:57168) 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faac3fff640 1 -- 172.21.15.45:0/2318690877 learned_addr learned my addr 172.21.15.45:0/2318690877 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faacad9d640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faacc19ba70 0x7faacc19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.570+0000 7faaca59c640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faacc0ff400 0x7faacc197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.571+0000 7faac3fff640 1 -- 172.21.15.45:0/2318690877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faacc19ba70 msgr2=0x7faacc19bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.571+0000 7faac3fff640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faacc19ba70 0x7faacc19bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.571+0000 7faac3fff640 1 -- 172.21.15.45:0/2318690877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faacc0ff400 msgr2=0x7faacc197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.571+0000 7faac3fff640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faacc0ff400 0x7faacc197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.571+0000 7faac3fff640 1 -- 172.21.15.45:0/2318690877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac4009900 con 0x7faacc106070 2024-01-31T05:25:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.571+0000 7faac3fff640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faacc106070 0x7faacc197740 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faab400d950 tx=0x7faab400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.571+0000 7faac37fe640 1 -- 172.21.15.45:0/2318690877 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4009c30 con 0x7faacc106070 2024-01-31T05:25:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.571+0000 7faad0932640 1 -- 172.21.15.45:0/2318690877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faacc1a3d20 con 0x7faacc106070 2024-01-31T05:25:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.571+0000 7faac37fe640 1 -- 172.21.15.45:0/2318690877 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faab400b600 con 0x7faacc106070 2024-01-31T05:25:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.571+0000 7faac37fe640 1 -- 172.21.15.45:0/2318690877 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab40047a0 con 0x7faacc106070 2024-01-31T05:25:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.571+0000 7faad0932640 1 -- 172.21.15.45:0/2318690877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faacc1a41d0 con 0x7faacc106070 2024-01-31T05:25:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.573+0000 7faac37fe640 1 -- 172.21.15.45:0/2318690877 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7faab4004020 con 0x7faacc106070 2024-01-31T05:25:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.573+0000 7faad37cc740 1 Processor -- start 2024-01-31T05:25:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faad37cc740 1 -- start start 2024-01-31T05:25:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faad37cc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb4a060 0x561f7bb4a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faad37cc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a980 0x561f7bb53210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faac37fe640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7faab80a14f0 0x7faab80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faad37cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb53750 0x561f7bb55b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faad37cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7bab9040 con 0x561f7bb53750 2024-01-31T05:25:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faad37cc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7bab6730 con 0x561f7bb4a980 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faad37cc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7bab2020 con 0x561f7bb4a060 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faacad9d640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7faab80a14f0 0x7faab80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faac3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb53750 0x561f7bb55b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faac3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb53750 0x561f7bb55b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57180/0 (socket says 172.21.15.45:57180) 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faac3fff640 1 -- 172.21.15.45:0/1246804557 learned_addr learned my addr 172.21.15.45:0/1246804557 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faac37fe640 1 -- 172.21.15.45:0/2318690877 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faab40ccab0 con 0x7faacc106070 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faacad9d640 1 --2- 172.21.15.45:0/1246804557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb4a060 0x561f7bb4a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faaca59c640 1 --2- 172.21.15.45:0/1246804557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a980 0x561f7bb53210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faac3fff640 1 -- 172.21.15.45:0/1246804557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb4a060 msgr2=0x561f7bb4a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faac3fff640 1 --2- 172.21.15.45:0/1246804557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb4a060 0x561f7bb4a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faac3fff640 1 -- 172.21.15.45:0/1246804557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a980 msgr2=0x561f7bb53210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faac3fff640 1 --2- 172.21.15.45:0/1246804557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a980 0x561f7bb53210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.574+0000 7faac3fff640 1 -- 172.21.15.45:0/1246804557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7baa9e40 con 0x561f7bb53750 2024-01-31T05:25:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.575+0000 7faacad9d640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7faab80a14f0 0x7faab80a39b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faac4005cc0 tx=0x7faac4032e80 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.575+0000 7faac3fff640 1 --2- 172.21.15.45:0/1246804557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb53750 0x561f7bb55b40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7faab4099890 tx=0x7faab4018cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4df1745b18d62 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.575+0000 7faac0ff9640 1 -- 172.21.15.45:0/1246804557 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab40996d0 con 0x561f7bb53750 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.575+0000 7faac0ff9640 1 -- 172.21.15.45:0/1246804557 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faab4099440 con 0x561f7bb53750 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.575+0000 7faac0ff9640 1 -- 172.21.15.45:0/1246804557 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab409da50 con 0x561f7bb53750 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.575+0000 7faad37cc740 1 -- 172.21.15.45:0/1246804557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb53750 msgr2=0x561f7bb55b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.575+0000 7faad37cc740 1 --2- 172.21.15.45:0/1246804557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb53750 0x561f7bb55b40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7faab4099890 tx=0x7faab4018cf0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.575+0000 7faad37cc740 1 -- 172.21.15.45:0/1246804557 shutdown_connections 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.575+0000 7faad37cc740 1 --2- 172.21.15.45:0/1246804557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb53750 0x561f7bb55b40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 --2- 172.21.15.45:0/1246804557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a980 0x561f7bb53210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 --2- 172.21.15.45:0/1246804557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb4a060 0x561f7bb4a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 -- 172.21.15.45:0/1246804557 >> 172.21.15.45:0/1246804557 conn(0x561f7bb44980 msgr2=0x561f7bb44d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 -- 172.21.15.45:0/1246804557 shutdown_connections 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 -- 172.21.15.45:0/1246804557 wait complete. 2024-01-31T05:25:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 -- start start 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 Processor -- start 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a060 0x561f7bb50710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a980 0x561f7bb50c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 0x561f7bb51190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7ba6b1b0 con 0x561f7bb4a060 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7ba6ae50 con 0x561f7bb4a980 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7ba30090 con 0x561f7bb53750 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faad37cc740 10 client.0 _ll_register_callbacks cb 0x561f7ba55a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faaca59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a060 0x561f7bb50710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faaca59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a060 0x561f7bb50710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57190/0 (socket says 172.21.15.45:57190) 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faaca59c640 1 -- 172.21.15.45:0/3639044466 learned_addr learned my addr 172.21.15.45:0/3639044466 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faacad9d640 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 0x561f7bb51190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.576+0000 7faac3fff640 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a980 0x561f7bb50c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faaca59c640 1 -- 172.21.15.45:0/3639044466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 msgr2=0x561f7bb51190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faaca59c640 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 0x561f7bb51190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faaca59c640 1 -- 172.21.15.45:0/3639044466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a980 msgr2=0x561f7bb50c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faaca59c640 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a980 0x561f7bb50c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faaca59c640 1 -- 172.21.15.45:0/3639044466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faacc1a41d0 con 0x561f7bb4a060 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faaca59c640 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a060 0x561f7bb50710 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7faabc00be90 tx=0x7faabc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422134a31b89ea8c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faab1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faab1ffb640 1 -- 172.21.15.45:0/3639044466 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc016070 con 0x561f7bb4a060 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faab1ffb640 1 -- 172.21.15.45:0/3639044466 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faabc011e40 con 0x561f7bb4a060 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faad37cc740 1 -- 172.21.15.45:0/3639044466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f7baab010 con 0x561f7bb4a060 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faab1ffb640 1 -- 172.21.15.45:0/3639044466 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faabc004020 con 0x561f7bb4a060 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faad37cc740 10 client.15621 fetch_fsmap learned FSMap version 23 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faad37cc740 1 -- 172.21.15.45:0/3639044466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561f7bab06a0 con 0x561f7bb4a060 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faab1ffb640 1 -- 172.21.15.45:0/3639044466 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f7baab010 con 0x561f7bb4a060 2024-01-31T05:25:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.577+0000 7faab1ffb640 1 -- 172.21.15.45:0/3639044466 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faabc0035f0 con 0x561f7bb4a060 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.578+0000 7faad37cc740 10 client.15621 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.578+0000 7faad37cc740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.578+0000 7faad37cc740 20 client.15621 populate_metadata read hostname 'smithi045' 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.578+0000 7faad37cc740 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f7bb636a0 0x561f7bb65ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.578+0000 7faad37cc740 4 client.15621 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.578+0000 7faad37cc740 1 -- 172.21.15.45:0/3639044466 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f7baaad30 con 0x561f7bb636a0 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.578+0000 7faac3fff640 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f7bb636a0 0x561f7bb65ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.578+0000 7faab1ffb640 10 client.15621 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.578+0000 7faac3fff640 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f7bb636a0 0x561f7bb65ae0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82fc64ed049cc569 server_cookie=223c05671c1deb30 in_seq=0 out_seq=0 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faab1ffb640 1 -- 172.21.15.45:0/3639044466 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561f7baaad30 con 0x561f7bb636a0 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faab1ffb640 10 client.15621 handle_command_reply: tid=0 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faad37cc740 1 client.15621 shutdown 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faad37cc740 1 -- 172.21.15.45:0/3639044466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a060 msgr2=0x561f7bb50710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faad37cc740 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a060 0x561f7bb50710 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7faabc00be90 tx=0x7faabc011810 comp rx=0 tx=0).stop 2024-01-31T05:25:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faad37cc740 1 -- 172.21.15.45:0/3639044466 shutdown_connections 2024-01-31T05:25:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faad37cc740 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f7bb636a0 0x561f7bb65ae0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faad37cc740 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 0x561f7bb51190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faab1ffb640 0 client.15621 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faad37cc740 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a980 0x561f7bb50c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faad37cc740 1 --2- 172.21.15.45:0/3639044466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a060 0x561f7bb50710 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faad37cc740 1 -- 172.21.15.45:0/3639044466 >> 172.21.15.45:0/3639044466 conn(0x561f7bb44980 msgr2=0x561f7bb466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faad37cc740 1 -- 172.21.15.45:0/3639044466 shutdown_connections 2024-01-31T05:25:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faad37cc740 1 -- 172.21.15.45:0/3639044466 wait complete. 2024-01-31T05:25:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.579+0000 7faad37cc740 20 client.15621 trim_cache size 0 max 16384 2024-01-31T05:25:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.592+0000 7faad37cc740 1 Processor -- start 2024-01-31T05:25:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.592+0000 7faad37cc740 1 -- start start 2024-01-31T05:25:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.592+0000 7faad37cc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a060 0x561f7bb52e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.592+0000 7faad37cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 0x561f7bb50fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.592+0000 7faad37cc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 0x561f7bb4b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.592+0000 7faad37cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7bb400b0 con 0x561f7bb4a980 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.592+0000 7faad37cc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7baab040 con 0x561f7bb4a060 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.592+0000 7faad37cc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7bb5fa00 con 0x561f7bb53750 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faac3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 0x561f7bb50fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faac3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 0x561f7bb50fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57198/0 (socket says 172.21.15.45:57198) 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faac3fff640 1 -- 172.21.15.45:0/1937726239 learned_addr learned my addr 172.21.15.45:0/1937726239 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faacad9d640 1 --2- 172.21.15.45:0/1937726239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 0x561f7bb4b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faac3fff640 1 -- 172.21.15.45:0/1937726239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 msgr2=0x561f7bb4b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faac3fff640 1 --2- 172.21.15.45:0/1937726239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 0x561f7bb4b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faac3fff640 1 -- 172.21.15.45:0/1937726239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a060 msgr2=0x561f7bb52e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faaca59c640 1 --2- 172.21.15.45:0/1937726239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a060 0x561f7bb52e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faac3fff640 1 --2- 172.21.15.45:0/1937726239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a060 0x561f7bb52e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faac3fff640 1 -- 172.21.15.45:0/1937726239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7ba4d030 con 0x561f7bb4a980 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faac3fff640 1 --2- 172.21.15.45:0/1937726239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 0x561f7bb50fb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7faab4096370 tx=0x7faab401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50710b11198c0c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faab3fff640 1 -- 172.21.15.45:0/1937726239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab40b9130 con 0x561f7bb4a980 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faab3fff640 1 -- 172.21.15.45:0/1937726239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faab40b1070 con 0x561f7bb4a980 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.593+0000 7faab3fff640 1 -- 172.21.15.45:0/1937726239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab409e790 con 0x561f7bb4a980 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 -- 172.21.15.45:0/1937726239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 msgr2=0x561f7bb50fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 --2- 172.21.15.45:0/1937726239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 0x561f7bb50fb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7faab4096370 tx=0x7faab401d040 comp rx=0 tx=0).stop 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 -- 172.21.15.45:0/1937726239 shutdown_connections 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 --2- 172.21.15.45:0/1937726239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 0x561f7bb50fb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 --2- 172.21.15.45:0/1937726239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 0x561f7bb4b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 --2- 172.21.15.45:0/1937726239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a060 0x561f7bb52e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 -- 172.21.15.45:0/1937726239 >> 172.21.15.45:0/1937726239 conn(0x561f7bb44980 msgr2=0x561f7bb49620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 -- 172.21.15.45:0/1937726239 shutdown_connections 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 -- 172.21.15.45:0/1937726239 wait complete. 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 -- start start 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 Processor -- start 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a060 0x561f7bb5a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 0x561f7bb5a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 0x561f7bb58bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7bb59280 con 0x561f7bb4a980 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7bb593f0 con 0x561f7bb4a060 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7bb595b0 con 0x561f7bb53750 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faad37cc740 10 client.0 _ll_register_callbacks cb 0x7faab4009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faac3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 0x561f7bb5a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faac3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 0x561f7bb5a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57214/0 (socket says 172.21.15.45:57214) 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faac3fff640 1 -- 172.21.15.45:0/217670608 learned_addr learned my addr 172.21.15.45:0/217670608 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faaca59c640 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a060 0x561f7bb5a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.594+0000 7faacad9d640 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 0x561f7bb58bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faac3fff640 1 -- 172.21.15.45:0/217670608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 msgr2=0x561f7bb58bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faac3fff640 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 0x561f7bb58bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faac3fff640 1 -- 172.21.15.45:0/217670608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a060 msgr2=0x561f7bb5a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faac3fff640 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a060 0x561f7bb5a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faac3fff640 1 -- 172.21.15.45:0/217670608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f7baa9e40 con 0x561f7bb4a980 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faac3fff640 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 0x561f7bb5a960 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7faab4018000 tx=0x7faab4021fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692822ed6f7b5543 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faab1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faab1ffb640 1 -- 172.21.15.45:0/217670608 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab40a1d00 con 0x561f7bb4a980 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faab1ffb640 1 -- 172.21.15.45:0/217670608 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faab409a000 con 0x561f7bb4a980 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faad37cc740 1 -- 172.21.15.45:0/217670608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faab4099440 con 0x561f7bb4a980 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faab1ffb640 1 -- 172.21.15.45:0/217670608 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faab4099260 con 0x561f7bb4a980 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faad37cc740 10 client.15627 fetch_fsmap learned FSMap version 23 2024-01-31T05:25:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faad37cc740 1 -- 172.21.15.45:0/217670608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561f7ba72a50 con 0x561f7bb4a980 2024-01-31T05:25:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faab1ffb640 1 -- 172.21.15.45:0/217670608 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faab4099440 con 0x561f7bb4a980 2024-01-31T05:25:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faab1ffb640 1 -- 172.21.15.45:0/217670608 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faab4096010 con 0x561f7bb4a980 2024-01-31T05:25:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faad37cc740 10 client.15627 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:25:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faad37cc740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:25:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faad37cc740 20 client.15627 populate_metadata read hostname 'smithi045' 2024-01-31T05:25:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faad37cc740 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f7bbbab90 0x561f7bbbcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.595+0000 7faad37cc740 4 client.15627 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:25:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.596+0000 7faad37cc740 1 -- 172.21.15.45:0/217670608 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f7b7b1600 con 0x561f7bbbab90 2024-01-31T05:25:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.596+0000 7faaca59c640 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f7bbbab90 0x561f7bbbcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.596+0000 7faaca59c640 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f7bbbab90 0x561f7bbbcfb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=871e70fe22a0f990 server_cookie=8fb8993c01c85e4f in_seq=0 out_seq=0 2024-01-31T05:25:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.596+0000 7faab1ffb640 10 client.15627 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faab1ffb640 1 -- 172.21.15.45:0/217670608 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f7b7b1600 con 0x561f7bbbab90 2024-01-31T05:25:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faab1ffb640 10 client.15627 handle_command_reply: tid=0 2024-01-31T05:25:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faad37cc740 1 client.15627 shutdown 2024-01-31T05:25:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faad37cc740 1 -- 172.21.15.45:0/217670608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 msgr2=0x561f7bb5a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faad37cc740 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 0x561f7bb5a960 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7faab4018000 tx=0x7faab4021fc0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faad37cc740 1 -- 172.21.15.45:0/217670608 shutdown_connections 2024-01-31T05:25:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faad37cc740 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561f7bbbab90 0x561f7bbbcfb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faad37cc740 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f7bb4a980 0x561f7bb5a960 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faad37cc740 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f7bb53750 0x561f7bb58bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faab1ffb640 0 client.15627 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faad37cc740 1 --2- 172.21.15.45:0/217670608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561f7bb4a060 0x561f7bb5a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faad37cc740 1 -- 172.21.15.45:0/217670608 >> 172.21.15.45:0/217670608 conn(0x561f7bb44980 msgr2=0x561f7bb4b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faad37cc740 1 -- 172.21.15.45:0/217670608 shutdown_connections 2024-01-31T05:25:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faad37cc740 1 -- 172.21.15.45:0/217670608 wait complete. 2024-01-31T05:25:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.604+0000 7faad37cc740 20 client.15627 trim_cache size 0 max 16384 2024-01-31T05:25:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:25:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:25:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:25:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:25:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.605+0000 7faad0932640 1 -- 172.21.15.45:0/2318690877 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7faab80a14f0 msgr2=0x7faab80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.605+0000 7faad0932640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7faab80a14f0 0x7faab80a39b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faac4005cc0 tx=0x7faac4032e80 comp rx=0 tx=0).stop 2024-01-31T05:25:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.605+0000 7faad0932640 1 -- 172.21.15.45:0/2318690877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faacc106070 msgr2=0x7faacc197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.605+0000 7faad0932640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faacc106070 0x7faacc197740 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faab400d950 tx=0x7faab400de10 comp rx=0 tx=0).stop 2024-01-31T05:25:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.605+0000 7faad0932640 1 -- 172.21.15.45:0/2318690877 shutdown_connections 2024-01-31T05:25:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.605+0000 7faad0932640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7faab80a14f0 0x7faab80a39b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.605+0000 7faad0932640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faacc106070 0x7faacc197740 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.605+0000 7faad0932640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faacc19ba70 0x7faacc19bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.605+0000 7faad0932640 1 --2- 172.21.15.45:0/2318690877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faacc0ff400 0x7faacc197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.605+0000 7faad0932640 1 -- 172.21.15.45:0/2318690877 >> 172.21.15.45:0/2318690877 conn(0x7faacc076110 msgr2=0x7faacc0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.605+0000 7faad0932640 1 -- 172.21.15.45:0/2318690877 shutdown_connections 2024-01-31T05:25:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:34.605+0000 7faad0932640 1 -- 172.21.15.45:0/2318690877 wait complete. 2024-01-31T05:25:34.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 205961 12.02 MB/sec warmup 94 sec latency 107.230 ms 2024-01-31T05:25:34.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:34 smithi186 ceph-mon[35659]: pgmap v89: 97 pgs: 97 active+clean; 106 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 346 op/s 2024-01-31T05:25:34.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:34 smithi045 ceph-mon[31585]: pgmap v89: 97 pgs: 97 active+clean; 106 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 346 op/s 2024-01-31T05:25:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:34 smithi156 ceph-mon[35720]: pgmap v89: 97 pgs: 97 active+clean; 106 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 346 op/s 2024-01-31T05:25:35.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 208299 11.98 MB/sec warmup 95 sec latency 16.728 ms 2024-01-31T05:25:36.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 210541 11.99 MB/sec warmup 96 sec latency 16.479 ms 2024-01-31T05:25:36.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:36 smithi186 ceph-mon[35659]: pgmap v90: 97 pgs: 97 active+clean; 106 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 226 op/s 2024-01-31T05:25:36.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:36 smithi045 ceph-mon[31585]: pgmap v90: 97 pgs: 97 active+clean; 106 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 226 op/s 2024-01-31T05:25:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:36 smithi156 ceph-mon[35720]: pgmap v90: 97 pgs: 97 active+clean; 106 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 226 op/s 2024-01-31T05:25:37.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 213328 12.04 MB/sec warmup 97 sec latency 8.096 ms 2024-01-31T05:25:38.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 216061 12.06 MB/sec warmup 98 sec latency 92.729 ms 2024-01-31T05:25:38.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:38 smithi186 ceph-mon[35659]: pgmap v91: 97 pgs: 97 active+clean; 112 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.5 MiB/s wr, 341 op/s 2024-01-31T05:25:38.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:38 smithi045 ceph-mon[31585]: pgmap v91: 97 pgs: 97 active+clean; 112 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.5 MiB/s wr, 341 op/s 2024-01-31T05:25:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:38 smithi156 ceph-mon[35720]: pgmap v91: 97 pgs: 97 active+clean; 112 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.5 MiB/s wr, 341 op/s 2024-01-31T05:25:39.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 217610 12.02 MB/sec warmup 99 sec latency 132.150 ms 2024-01-31T05:25:40.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 220340 12.07 MB/sec warmup 100 sec latency 31.376 ms 2024-01-31T05:25:40.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:40 smithi186 ceph-mon[35659]: pgmap v92: 97 pgs: 97 active+clean; 112 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-01-31T05:25:40.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:40 smithi045 ceph-mon[31585]: pgmap v92: 97 pgs: 97 active+clean; 112 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-01-31T05:25:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:40 smithi156 ceph-mon[35720]: pgmap v92: 97 pgs: 97 active+clean; 112 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-01-31T05:25:41.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 222986 12.07 MB/sec warmup 101 sec latency 9.930 ms 2024-01-31T05:25:42.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 225181 12.07 MB/sec warmup 102 sec latency 9.273 ms 2024-01-31T05:25:42.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:42 smithi186 ceph-mon[35659]: pgmap v93: 97 pgs: 97 active+clean; 112 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-01-31T05:25:42.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:42 smithi045 ceph-mon[31585]: pgmap v93: 97 pgs: 97 active+clean; 112 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-01-31T05:25:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:42 smithi156 ceph-mon[35720]: pgmap v93: 97 pgs: 97 active+clean; 112 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-01-31T05:25:43.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 227466 12.09 MB/sec warmup 103 sec latency 108.308 ms 2024-01-31T05:25:44.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 229987 12.09 MB/sec warmup 104 sec latency 19.559 ms 2024-01-31T05:25:44.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:44 smithi186 ceph-mon[35659]: pgmap v94: 97 pgs: 97 active+clean; 117 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.6 MiB/s wr, 342 op/s 2024-01-31T05:25:44.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:44 smithi045 ceph-mon[31585]: pgmap v94: 97 pgs: 97 active+clean; 117 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.6 MiB/s wr, 342 op/s 2024-01-31T05:25:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:44 smithi156 ceph-mon[35720]: pgmap v94: 97 pgs: 97 active+clean; 117 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.6 MiB/s wr, 342 op/s 2024-01-31T05:25:45.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 231676 12.06 MB/sec warmup 105 sec latency 75.305 ms 2024-01-31T05:25:46.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 234506 12.10 MB/sec warmup 106 sec latency 13.194 ms 2024-01-31T05:25:46.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:46 smithi186 ceph-mon[35659]: pgmap v95: 97 pgs: 97 active+clean; 117 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-01-31T05:25:46.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:46 smithi045 ceph-mon[31585]: pgmap v95: 97 pgs: 97 active+clean; 117 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-01-31T05:25:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:46 smithi156 ceph-mon[35720]: pgmap v95: 97 pgs: 97 active+clean; 117 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-01-31T05:25:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.697+0000 7f1737e19640 1 -- 172.21.15.45:0/194633182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1730104f00 msgr2=0x7f173010cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.697+0000 7f1737e19640 1 --2- 172.21.15.45:0/194633182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1730104f00 0x7f173010cad0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1728009930 tx=0x7f172802bc30 comp rx=0 tx=0).stop 2024-01-31T05:25:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.697+0000 7f1737e19640 1 -- 172.21.15.45:0/194633182 shutdown_connections 2024-01-31T05:25:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.697+0000 7f1737e19640 1 --2- 172.21.15.45:0/194633182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1730104f00 0x7f173010cad0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.697+0000 7f1737e19640 1 --2- 172.21.15.45:0/194633182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17301007e0 0x7f1730104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.697+0000 7f1737e19640 1 --2- 172.21.15.45:0/194633182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17300ffe30 0x7f1730100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.697+0000 7f1737e19640 1 -- 172.21.15.45:0/194633182 >> 172.21.15.45:0/194633182 conn(0x7f1730076d20 msgr2=0x7f1730077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.697+0000 7f1737e19640 1 -- 172.21.15.45:0/194633182 shutdown_connections 2024-01-31T05:25:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.697+0000 7f1737e19640 1 -- 172.21.15.45:0/194633182 wait complete. 2024-01-31T05:25:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f1737e19640 1 Processor -- start 2024-01-31T05:25:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f1737e19640 1 -- start start 2024-01-31T05:25:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f1737e19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300ffe30 0x7f17301971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f1737e19640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1730104f00 0x7f17301976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f1737e19640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f173019ba70 0x7f173019bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f1737e19640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173019c460 con 0x7f17300ffe30 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f1737e19640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173019c5d0 con 0x7f173019ba70 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f1737e19640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173019c740 con 0x7f1730104f00 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f1735bb5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300ffe30 0x7f17301971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f1735bb5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300ffe30 0x7f17301971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36672/0 (socket says 172.21.15.45:36672) 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f17353b4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1730104f00 0x7f17301976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f1735bb5640 1 -- 172.21.15.45:0/847196571 learned_addr learned my addr 172.21.15.45:0/847196571 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.698+0000 7f17363b6640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f173019ba70 0x7f173019bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.699+0000 7f1735bb5640 1 -- 172.21.15.45:0/847196571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1730104f00 msgr2=0x7f17301976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.699+0000 7f1735bb5640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1730104f00 0x7f17301976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.699+0000 7f1735bb5640 1 -- 172.21.15.45:0/847196571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f173019ba70 msgr2=0x7f173019bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.699+0000 7f1735bb5640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f173019ba70 0x7f173019bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.699+0000 7f1735bb5640 1 -- 172.21.15.45:0/847196571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728009560 con 0x7f17300ffe30 2024-01-31T05:25:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.699+0000 7f1735bb5640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300ffe30 0x7f17301971b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f172000eab0 tx=0x7f172000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.699+0000 7f172effd640 1 -- 172.21.15.45:0/847196571 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000ce40 con 0x7f17300ffe30 2024-01-31T05:25:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.699+0000 7f1737e19640 1 -- 172.21.15.45:0/847196571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17301a3d20 con 0x7f17300ffe30 2024-01-31T05:25:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.699+0000 7f172effd640 1 -- 172.21.15.45:0/847196571 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1720011930 con 0x7f17300ffe30 2024-01-31T05:25:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.699+0000 7f172effd640 1 -- 172.21.15.45:0/847196571 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720010a30 con 0x7f17300ffe30 2024-01-31T05:25:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.699+0000 7f1737e19640 1 -- 172.21.15.45:0/847196571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17301a41d0 con 0x7f17300ffe30 2024-01-31T05:25:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.701+0000 7f172effd640 1 -- 172.21.15.45:0/847196571 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f1720011470 con 0x7f17300ffe30 2024-01-31T05:25:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.702+0000 7f173abd9740 1 Processor -- start 2024-01-31T05:25:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.702+0000 7f173abd9740 1 -- start start 2024-01-31T05:25:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.702+0000 7f172effd640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f17180a14f0 0x7f17180a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f173abd9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1060 0x55a5f38b1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f173abd9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 0x55a5f38ba210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f173abd9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38bcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f173abd9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f3820040 con 0x55a5f38b1980 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f173abd9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f381d730 con 0x55a5f38b1060 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f173abd9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f3819020 con 0x55a5f38ba750 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f17363b6640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f17180a14f0 0x7f17180a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f172effd640 1 -- 172.21.15.45:0/847196571 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f17200cd4d0 con 0x7f17300ffe30 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f17353b4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 0x55a5f38ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f17353b4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 0x55a5f38ba210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36686/0 (socket says 172.21.15.45:36686) 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f17353b4640 1 -- 172.21.15.45:0/2285336208 learned_addr learned my addr 172.21.15.45:0/2285336208 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f1735bb5640 1 --2- 172.21.15.45:0/2285336208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1060 0x55a5f38b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f17363b6640 1 --2- 172.21.15.45:0/2285336208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38bcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f17353b4640 1 -- 172.21.15.45:0/2285336208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 msgr2=0x55a5f38bcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f17353b4640 1 --2- 172.21.15.45:0/2285336208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38bcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f17353b4640 1 -- 172.21.15.45:0/2285336208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1060 msgr2=0x55a5f38b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f17353b4640 1 --2- 172.21.15.45:0/2285336208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1060 0x55a5f38b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f17353b4640 1 -- 172.21.15.45:0/2285336208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5f3810e40 con 0x55a5f38b1980 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f17363b6640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f17180a14f0 0x7f17180a39b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1728005940 tx=0x7f1728004900 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f17353b4640 1 --2- 172.21.15.45:0/2285336208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 0x55a5f38ba210 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1724009c00 tx=0x7f1724011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2ad9d9fddeea1a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f1717fff640 1 -- 172.21.15.45:0/2285336208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724004230 con 0x55a5f38b1980 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f1717fff640 1 -- 172.21.15.45:0/2285336208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1724009e70 con 0x55a5f38b1980 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.703+0000 7f1717fff640 1 -- 172.21.15.45:0/2285336208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724019640 con 0x55a5f38b1980 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.704+0000 7f173abd9740 1 -- 172.21.15.45:0/2285336208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 msgr2=0x55a5f38ba210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.704+0000 7f173abd9740 1 --2- 172.21.15.45:0/2285336208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 0x55a5f38ba210 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1724009c00 tx=0x7f1724011810 comp rx=0 tx=0).stop 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.704+0000 7f173abd9740 1 -- 172.21.15.45:0/2285336208 shutdown_connections 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.704+0000 7f173abd9740 1 --2- 172.21.15.45:0/2285336208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 0x55a5f38ba210 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.704+0000 7f173abd9740 1 --2- 172.21.15.45:0/2285336208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38bcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.704+0000 7f173abd9740 1 --2- 172.21.15.45:0/2285336208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1060 0x55a5f38b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.704+0000 7f173abd9740 1 -- 172.21.15.45:0/2285336208 >> 172.21.15.45:0/2285336208 conn(0x55a5f38ab980 msgr2=0x55a5f38abd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.704+0000 7f173abd9740 1 -- 172.21.15.45:0/2285336208 shutdown_connections 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.704+0000 7f173abd9740 1 -- 172.21.15.45:0/2285336208 wait complete. 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f173abd9740 1 -- start start 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f173abd9740 1 Processor -- start 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f173abd9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 0x55a5f38b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f173abd9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1980 0x55a5f38b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f173abd9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f173abd9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f37d21b0 con 0x55a5f38b1060 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f173abd9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f37d1e50 con 0x55a5f38b1980 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f173abd9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f3797090 con 0x55a5f38ba750 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f173abd9740 10 client.0 _ll_register_callbacks cb 0x55a5f37bca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f1735bb5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 0x55a5f38b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f1735bb5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 0x55a5f38b7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36696/0 (socket says 172.21.15.45:36696) 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f17363b6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f1735bb5640 1 -- 172.21.15.45:0/3179517523 learned_addr learned my addr 172.21.15.45:0/3179517523 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.705+0000 7f17353b4640 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1980 0x55a5f38b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f17363b6640 1 -- 172.21.15.45:0/3179517523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1980 msgr2=0x55a5f38b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f17363b6640 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1980 0x55a5f38b7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f17363b6640 1 -- 172.21.15.45:0/3179517523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 msgr2=0x55a5f38b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f17363b6640 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 0x55a5f38b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f17363b6640 1 -- 172.21.15.45:0/3179517523 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1724009830 con 0x55a5f38ba750 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f1735bb5640 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 0x55a5f38b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f17363b6640 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38b80e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f172802c170 tx=0x7f172802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97535490625eca09 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f17157fa640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f17157fa640 1 -- 172.21.15.45:0/3179517523 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172802cc30 con 0x55a5f38ba750 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f17157fa640 1 -- 172.21.15.45:0/3179517523 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1728005210 con 0x55a5f38ba750 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f173abd9740 1 -- 172.21.15.45:0/3179517523 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5f3812010 con 0x55a5f38ba750 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f17157fa640 1 -- 172.21.15.45:0/3179517523 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17280053d0 con 0x55a5f38ba750 2024-01-31T05:25:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f173abd9740 10 client.25189 fetch_fsmap learned FSMap version 23 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.706+0000 7f173abd9740 1 -- 172.21.15.45:0/3179517523 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a5f38176a0 con 0x55a5f38ba750 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.707+0000 7f17157fa640 1 -- 172.21.15.45:0/3179517523 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a5f3812010 con 0x55a5f38ba750 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.707+0000 7f17157fa640 1 -- 172.21.15.45:0/3179517523 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1728042470 con 0x55a5f38ba750 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.707+0000 7f173abd9740 10 client.25189 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.707+0000 7f173abd9740 10 client.25189 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.707+0000 7f173abd9740 20 client.25189 populate_metadata read hostname 'smithi045' 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.707+0000 7f173abd9740 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a5f38ca540 0x55a5f38cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.707+0000 7f173abd9740 4 client.25189 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.707+0000 7f173abd9740 1 -- 172.21.15.45:0/3179517523 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5f3811d30 con 0x55a5f38ca540 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.707+0000 7f1735bb5640 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a5f38ca540 0x55a5f38cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.707+0000 7f17157fa640 10 client.25189 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.707+0000 7f1735bb5640 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a5f38ca540 0x55a5f38cc960 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b32cc0cdb30d31 server_cookie=236dbfc1e6da1a12 in_seq=0 out_seq=0 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.708+0000 7f17157fa640 1 -- 172.21.15.45:0/3179517523 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5f3811d30 con 0x55a5f38ca540 2024-01-31T05:25:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.708+0000 7f17157fa640 10 client.25189 handle_command_reply: tid=0 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.708+0000 7f173abd9740 1 client.25189 shutdown 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.708+0000 7f173abd9740 1 -- 172.21.15.45:0/3179517523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 msgr2=0x55a5f38b80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.708+0000 7f173abd9740 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38b80e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f172802c170 tx=0x7f172802bc30 comp rx=0 tx=0).stop 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.708+0000 7f173abd9740 1 -- 172.21.15.45:0/3179517523 shutdown_connections 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.708+0000 7f173abd9740 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a5f38ca540 0x55a5f38cc960 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.708+0000 7f173abd9740 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38b80e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.708+0000 7f173abd9740 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1980 0x55a5f38b7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.708+0000 7f17157fa640 0 client.25189 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.708+0000 7f173abd9740 1 --2- 172.21.15.45:0/3179517523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 0x55a5f38b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.709+0000 7f173abd9740 1 -- 172.21.15.45:0/3179517523 >> 172.21.15.45:0/3179517523 conn(0x55a5f38ab980 msgr2=0x55a5f38ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.709+0000 7f173abd9740 1 -- 172.21.15.45:0/3179517523 shutdown_connections 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.709+0000 7f173abd9740 1 -- 172.21.15.45:0/3179517523 wait complete. 2024-01-31T05:25:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.709+0000 7f173abd9740 20 client.25189 trim_cache size 0 max 16384 2024-01-31T05:25:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f173abd9740 1 Processor -- start 2024-01-31T05:25:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f173abd9740 1 -- start start 2024-01-31T05:25:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f173abd9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 0x55a5f38b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f173abd9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1980 0x55a5f38b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f173abd9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38ad440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f173abd9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f38b2f60 con 0x55a5f38b1060 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f173abd9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f3811d30 con 0x55a5f38b1980 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f173abd9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f3812010 con 0x55a5f38ba750 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f1735bb5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 0x55a5f38b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f1735bb5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 0x55a5f38b67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36698/0 (socket says 172.21.15.45:36698) 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f1735bb5640 1 -- 172.21.15.45:0/122597992 learned_addr learned my addr 172.21.15.45:0/122597992 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f17363b6640 1 --2- 172.21.15.45:0/122597992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f17353b4640 1 --2- 172.21.15.45:0/122597992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1980 0x55a5f38b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f1735bb5640 1 -- 172.21.15.45:0/122597992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 msgr2=0x55a5f38ad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f1735bb5640 1 --2- 172.21.15.45:0/122597992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38ad440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f1735bb5640 1 -- 172.21.15.45:0/122597992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1980 msgr2=0x55a5f38b7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f1735bb5640 1 --2- 172.21.15.45:0/122597992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1980 0x55a5f38b7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.724+0000 7f1735bb5640 1 -- 172.21.15.45:0/122597992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5f38b0850 con 0x55a5f38b1060 2024-01-31T05:25:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.725+0000 7f1735bb5640 1 --2- 172.21.15.45:0/122597992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 0x55a5f38b67c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1720096770 tx=0x7f1720000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8e9828a51760ef server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.725+0000 7f17177fe640 1 -- 172.21.15.45:0/122597992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172009e410 con 0x55a5f38b1060 2024-01-31T05:25:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.725+0000 7f17177fe640 1 -- 172.21.15.45:0/122597992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17200a2650 con 0x55a5f38b1060 2024-01-31T05:25:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.725+0000 7f17177fe640 1 -- 172.21.15.45:0/122597992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720011e50 con 0x55a5f38b1060 2024-01-31T05:25:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 -- 172.21.15.45:0/122597992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 msgr2=0x55a5f38b67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 --2- 172.21.15.45:0/122597992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 0x55a5f38b67c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1720096770 tx=0x7f1720000f40 comp rx=0 tx=0).stop 2024-01-31T05:25:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 -- 172.21.15.45:0/122597992 shutdown_connections 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 --2- 172.21.15.45:0/122597992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38ba750 0x55a5f38ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 --2- 172.21.15.45:0/122597992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38b1980 0x55a5f38b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 --2- 172.21.15.45:0/122597992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1060 0x55a5f38b67c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 -- 172.21.15.45:0/122597992 >> 172.21.15.45:0/122597992 conn(0x55a5f38ab980 msgr2=0x55a5f38acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 -- 172.21.15.45:0/122597992 shutdown_connections 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 -- 172.21.15.45:0/122597992 wait complete. 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 -- start start 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 Processor -- start 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38b1060 0x55a5f38c0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 0x55a5f38c07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38ba750 0x55a5f38c0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f3920bf0 con 0x55a5f38b1980 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f3920d60 con 0x55a5f38ba750 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.726+0000 7f173abd9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f3920f20 con 0x55a5f38b1060 2024-01-31T05:25:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f173abd9740 10 client.0 _ll_register_callbacks cb 0x7f1724009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:25:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17353b4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 0x55a5f38c07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f1735bb5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38b1060 0x55a5f38c0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17363b6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38ba750 0x55a5f38c0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17353b4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 0x55a5f38c07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36702/0 (socket says 172.21.15.45:36702) 2024-01-31T05:25:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17353b4640 1 -- 172.21.15.45:0/1004157988 learned_addr learned my addr 172.21.15.45:0/1004157988 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:25:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17353b4640 1 -- 172.21.15.45:0/1004157988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38b1060 msgr2=0x55a5f38c0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17353b4640 1 --2- 172.21.15.45:0/1004157988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38b1060 0x55a5f38c0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17353b4640 1 -- 172.21.15.45:0/1004157988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38ba750 msgr2=0x55a5f38c0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:25:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17353b4640 1 --2- 172.21.15.45:0/1004157988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38ba750 0x55a5f38c0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17353b4640 1 -- 172.21.15.45:0/1004157988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17301a41d0 con 0x55a5f38b1980 2024-01-31T05:25:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17363b6640 1 --2- 172.21.15.45:0/1004157988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38ba750 0x55a5f38c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:25:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17353b4640 1 --2- 172.21.15.45:0/1004157988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 0x55a5f38c07d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x55a5f38b0b10 tx=0x7f1724005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9fad977a007c39 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17157fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17157fa640 1 -- 172.21.15.45:0/1004157988 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724026050 con 0x55a5f38b1980 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17157fa640 1 -- 172.21.15.45:0/1004157988 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1724003a00 con 0x55a5f38b1980 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f173abd9740 1 -- 172.21.15.45:0/1004157988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1724009e70 con 0x55a5f38b1980 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f17157fa640 1 -- 172.21.15.45:0/1004157988 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1724003bc0 con 0x55a5f38b1980 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f173abd9740 10 client.15645 fetch_fsmap learned FSMap version 23 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.727+0000 7f173abd9740 1 -- 172.21.15.45:0/1004157988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a5f37b4030 con 0x55a5f38b1980 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.728+0000 7f17157fa640 1 -- 172.21.15.45:0/1004157988 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1724009e70 con 0x55a5f38b1980 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.728+0000 7f17157fa640 1 -- 172.21.15.45:0/1004157988 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f172401a670 con 0x55a5f38b1980 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.728+0000 7f173abd9740 10 client.15645 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.728+0000 7f173abd9740 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.728+0000 7f173abd9740 20 client.15645 populate_metadata read hostname 'smithi045' 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.728+0000 7f173abd9740 1 --2- 172.21.15.45:0/1004157988 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a5f39226f0 0x55a5f3924b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:25:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.728+0000 7f173abd9740 4 client.15645 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:25:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.728+0000 7f173abd9740 1 -- 172.21.15.45:0/1004157988 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a5f3518600 con 0x55a5f39226f0 2024-01-31T05:25:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.728+0000 7f1735bb5640 1 --2- 172.21.15.45:0/1004157988 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a5f39226f0 0x55a5f3924b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:25:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.729+0000 7f1735bb5640 1 --2- 172.21.15.45:0/1004157988 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a5f39226f0 0x55a5f3924b10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fddf3fe1a03cf3e0 server_cookie=e3ace06946d25506 in_seq=0 out_seq=0 2024-01-31T05:25:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.729+0000 7f17157fa640 10 client.15645 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.736+0000 7f17157fa640 1 -- 172.21.15.45:0/1004157988 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a5f3518600 con 0x55a5f39226f0 2024-01-31T05:25:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.736+0000 7f17157fa640 10 client.15645 handle_command_reply: tid=0 2024-01-31T05:25:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.736+0000 7f173abd9740 1 client.15645 shutdown 2024-01-31T05:25:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f173abd9740 1 -- 172.21.15.45:0/1004157988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 msgr2=0x55a5f38c07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f173abd9740 1 --2- 172.21.15.45:0/1004157988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 0x55a5f38c07d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x55a5f38b0b10 tx=0x7f1724005720 comp rx=0 tx=0).stop 2024-01-31T05:25:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f173abd9740 1 -- 172.21.15.45:0/1004157988 shutdown_connections 2024-01-31T05:25:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f173abd9740 1 --2- 172.21.15.45:0/1004157988 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a5f39226f0 0x55a5f3924b10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f173abd9740 1 --2- 172.21.15.45:0/1004157988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5f38b1980 0x55a5f38c07d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f173abd9740 1 --2- 172.21.15.45:0/1004157988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5f38ba750 0x55a5f38c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f173abd9740 1 --2- 172.21.15.45:0/1004157988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5f38b1060 0x55a5f38c0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f17157fa640 0 client.15645 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:25:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f173abd9740 1 -- 172.21.15.45:0/1004157988 >> 172.21.15.45:0/1004157988 conn(0x55a5f38ab980 msgr2=0x55a5f38aa190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f173abd9740 1 -- 172.21.15.45:0/1004157988 shutdown_connections 2024-01-31T05:25:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f173abd9740 1 -- 172.21.15.45:0/1004157988 wait complete. 2024-01-31T05:25:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f173abd9740 20 client.15645 trim_cache size 0 max 16384 2024-01-31T05:25:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:25:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:25:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:25:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:25:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f1737e19640 1 -- 172.21.15.45:0/847196571 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f17180a14f0 msgr2=0x7f17180a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f1737e19640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f17180a14f0 0x7f17180a39b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1728005940 tx=0x7f1728004900 comp rx=0 tx=0).stop 2024-01-31T05:25:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f1737e19640 1 -- 172.21.15.45:0/847196571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300ffe30 msgr2=0x7f17301971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:25:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f1737e19640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300ffe30 0x7f17301971b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f172000eab0 tx=0x7f172000ef70 comp rx=0 tx=0).stop 2024-01-31T05:25:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f1737e19640 1 -- 172.21.15.45:0/847196571 shutdown_connections 2024-01-31T05:25:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f1737e19640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f17180a14f0 0x7f17180a39b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f1737e19640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f173019ba70 0x7f173019bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f1737e19640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1730104f00 0x7f17301976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f1737e19640 1 --2- 172.21.15.45:0/847196571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300ffe30 0x7f17301971b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:25:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.737+0000 7f1737e19640 1 -- 172.21.15.45:0/847196571 >> 172.21.15.45:0/847196571 conn(0x7f1730076d20 msgr2=0x7f17300697f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:25:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.738+0000 7f1737e19640 1 -- 172.21.15.45:0/847196571 shutdown_connections 2024-01-31T05:25:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:25:47.738+0000 7f1737e19640 1 -- 172.21.15.45:0/847196571 wait complete. 2024-01-31T05:25:47.775 INFO:tasks.workunit.client.0.smithi045.stdout: 1 237314 12.11 MB/sec warmup 107 sec latency 8.760 ms 2024-01-31T05:25:48.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 238917 12.08 MB/sec warmup 108 sec latency 67.411 ms 2024-01-31T05:25:48.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:48 smithi186 ceph-mon[35659]: pgmap v96: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.5 MiB/s wr, 338 op/s 2024-01-31T05:25:48.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:48 smithi045 ceph-mon[31585]: pgmap v96: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.5 MiB/s wr, 338 op/s 2024-01-31T05:25:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:48 smithi156 ceph-mon[35720]: pgmap v96: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.5 MiB/s wr, 338 op/s 2024-01-31T05:25:49.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 241565 12.12 MB/sec warmup 109 sec latency 22.506 ms 2024-01-31T05:25:49.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:49 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:25:49.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:49 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:25:49.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:49 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:25:50.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 244267 12.13 MB/sec warmup 110 sec latency 6.575 ms 2024-01-31T05:25:50.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:50 smithi186 ceph-mon[35659]: pgmap v97: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2024-01-31T05:25:50.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:50 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:25:50.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:50 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:25:50.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:50 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:25:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:50 smithi045 ceph-mon[31585]: pgmap v97: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2024-01-31T05:25:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:50 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:25:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:50 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:25:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:50 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:25:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:50 smithi156 ceph-mon[35720]: pgmap v97: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2024-01-31T05:25:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:50 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:25:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:50 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:25:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:50 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:25:51.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 246877 12.13 MB/sec warmup 111 sec latency 9.242 ms 2024-01-31T05:25:51.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:51 smithi186 ceph-mon[35659]: pgmap v98: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2024-01-31T05:25:51.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:51 smithi045 ceph-mon[31585]: pgmap v98: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2024-01-31T05:25:51.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:51 smithi156 ceph-mon[35720]: pgmap v98: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2024-01-31T05:25:52.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 249110 12.14 MB/sec warmup 112 sec latency 13.894 ms 2024-01-31T05:25:53.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 251993 12.19 MB/sec warmup 113 sec latency 120.881 ms 2024-01-31T05:25:54.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 254358 12.16 MB/sec warmup 114 sec latency 17.906 ms 2024-01-31T05:25:54.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:54 smithi186 ceph-mon[35659]: pgmap v99: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 340 op/s 2024-01-31T05:25:54.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:54 smithi045 ceph-mon[31585]: pgmap v99: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 340 op/s 2024-01-31T05:25:54.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:54 smithi156 ceph-mon[35720]: pgmap v99: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 340 op/s 2024-01-31T05:25:55.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 256493 12.17 MB/sec warmup 115 sec latency 17.808 ms 2024-01-31T05:25:56.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 259084 12.20 MB/sec warmup 116 sec latency 172.229 ms 2024-01-31T05:25:56.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:56 smithi186 ceph-mon[35659]: pgmap v100: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 228 op/s 2024-01-31T05:25:56.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:56 smithi045 ceph-mon[31585]: pgmap v100: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 228 op/s 2024-01-31T05:25:56.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:56 smithi156 ceph-mon[35720]: pgmap v100: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 228 op/s 2024-01-31T05:25:57.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 261437 12.18 MB/sec warmup 117 sec latency 74.741 ms 2024-01-31T05:25:58.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 263201 12.17 MB/sec warmup 118 sec latency 64.821 ms 2024-01-31T05:25:58.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:25:58 smithi186 ceph-mon[35659]: pgmap v101: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 341 op/s 2024-01-31T05:25:58.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:25:58 smithi045 ceph-mon[31585]: pgmap v101: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 341 op/s 2024-01-31T05:25:58.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:25:58 smithi156 ceph-mon[35720]: pgmap v101: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 341 op/s 2024-01-31T05:25:59.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 266129 12.22 MB/sec warmup 119 sec latency 18.976 ms 2024-01-31T05:26:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.825+0000 7fa6c74df640 1 -- 172.21.15.45:0/2612826386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c010c600 msgr2=0x7fa6c01141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.825+0000 7fa6c74df640 1 --2- 172.21.15.45:0/2612826386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c010c600 0x7fa6c01141d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009cd0 tx=0x7fa6b802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.825+0000 7fa6c74df640 1 -- 172.21.15.45:0/2612826386 shutdown_connections 2024-01-31T05:26:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.825+0000 7fa6c74df640 1 --2- 172.21.15.45:0/2612826386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c010c600 0x7fa6c01141d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.825+0000 7fa6c74df640 1 --2- 172.21.15.45:0/2612826386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c0107ee0 0x7fa6c010bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.825+0000 7fa6c74df640 1 --2- 172.21.15.45:0/2612826386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c0107530 0x7fa6c0107910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.825+0000 7fa6c74df640 1 -- 172.21.15.45:0/2612826386 >> 172.21.15.45:0/2612826386 conn(0x7fa6c0075260 msgr2=0x7fa6c0077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.825+0000 7fa6c74df640 1 -- 172.21.15.45:0/2612826386 shutdown_connections 2024-01-31T05:26:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.825+0000 7fa6c74df640 1 -- 172.21.15.45:0/2612826386 wait complete. 2024-01-31T05:26:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c74df640 1 Processor -- start 2024-01-31T05:26:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c74df640 1 -- start start 2024-01-31T05:26:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c74df640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c0107530 0x7fa6c01a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c74df640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c010c600 0x7fa6c01a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c74df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c019d610 0x7fa6c019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c74df640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c019e000 con 0x7fa6c010c600 2024-01-31T05:26:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c74df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c019e170 con 0x7fa6c019d610 2024-01-31T05:26:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c74df640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c019e330 con 0x7fa6c0107530 2024-01-31T05:26:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c4a7a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c010c600 0x7fa6c01a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c527b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c0107530 0x7fa6c01a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c4a7a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c010c600 0x7fa6c01a3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51478/0 (socket says 172.21.15.45:51478) 2024-01-31T05:26:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c4a7a640 1 -- 172.21.15.45:0/356375746 learned_addr learned my addr 172.21.15.45:0/356375746 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.826+0000 7fa6c5a7c640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c019d610 0x7fa6c019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.827+0000 7fa6c4a7a640 1 -- 172.21.15.45:0/356375746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c0107530 msgr2=0x7fa6c01a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.827+0000 7fa6c4a7a640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c0107530 0x7fa6c01a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.827+0000 7fa6c4a7a640 1 -- 172.21.15.45:0/356375746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c019d610 msgr2=0x7fa6c019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.827+0000 7fa6c4a7a640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c019d610 0x7fa6c019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.827+0000 7fa6c4a7a640 1 -- 172.21.15.45:0/356375746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b8009900 con 0x7fa6c010c600 2024-01-31T05:26:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.827+0000 7fa6c527b640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c0107530 0x7fa6c01a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:26:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.827+0000 7fa6c4a7a640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c010c600 0x7fa6c01a3950 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000e9f0 tx=0x7fa6b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.827+0000 7fa6b67fc640 1 -- 172.21.15.45:0/356375746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00096c0 con 0x7fa6c010c600 2024-01-31T05:26:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.827+0000 7fa6c74df640 1 -- 172.21.15.45:0/356375746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c01aa2d0 con 0x7fa6c010c600 2024-01-31T05:26:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.827+0000 7fa6b67fc640 1 -- 172.21.15.45:0/356375746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6b00054e0 con 0x7fa6c010c600 2024-01-31T05:26:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.827+0000 7fa6b67fc640 1 -- 172.21.15.45:0/356375746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0010430 con 0x7fa6c010c600 2024-01-31T05:26:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.827+0000 7fa6c74df640 1 -- 172.21.15.45:0/356375746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c01aa7e0 con 0x7fa6c010c600 2024-01-31T05:26:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.829+0000 7fa6b67fc640 1 -- 172.21.15.45:0/356375746 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fa6b0005020 con 0x7fa6c010c600 2024-01-31T05:26:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6ca384740 1 Processor -- start 2024-01-31T05:26:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6ca384740 1 -- start start 2024-01-31T05:26:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6b67fc640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fa6ac0a14f0 0x7fa6ac0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6ca384740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894060 0x55c2bd894440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6ca384740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894980 0x55c2bd89d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6ca384740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 0x55c2bd89fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6ca384740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bd803040 con 0x55c2bd894980 2024-01-31T05:26:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6ca384740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bd800730 con 0x55c2bd894060 2024-01-31T05:26:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6ca384740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bd7fc020 con 0x55c2bd89d750 2024-01-31T05:26:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6c5a7c640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fa6ac0a14f0 0x7fa6ac0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6c527b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894060 0x55c2bd894440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6c4a7a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894980 0x55c2bd89d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6c527b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894060 0x55c2bd894440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:55934/0 (socket says 172.21.15.45:55934) 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6c527b640 1 -- 172.21.15.45:0/3422102865 learned_addr learned my addr 172.21.15.45:0/3422102865 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6b67fc640 1 -- 172.21.15.45:0/356375746 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa6b0014070 con 0x7fa6c010c600 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6c5a7c640 1 --2- 172.21.15.45:0/3422102865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 0x55c2bd89fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6c527b640 1 -- 172.21.15.45:0/3422102865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 msgr2=0x55c2bd89fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6c527b640 1 --2- 172.21.15.45:0/3422102865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 0x55c2bd89fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6c527b640 1 -- 172.21.15.45:0/3422102865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894980 msgr2=0x55c2bd89d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.830+0000 7fa6c527b640 1 --2- 172.21.15.45:0/3422102865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894980 0x55c2bd89d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.831+0000 7fa6c527b640 1 -- 172.21.15.45:0/3422102865 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2bd7f3e40 con 0x55c2bd894060 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.831+0000 7fa6c4a7a640 1 --2- 172.21.15.45:0/3422102865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894980 0x55c2bd89d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.831+0000 7fa6c5a7c640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fa6ac0a14f0 0x7fa6ac0a39b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005cc0 tx=0x7fa6b8032ee0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.831+0000 7fa6c527b640 1 --2- 172.21.15.45:0/3422102865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894060 0x55c2bd894440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0115f0 tx=0x7fa6bc011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d464873ed92d5f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.831+0000 7fa6ab7fe640 1 -- 172.21.15.45:0/3422102865 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00f640 con 0x55c2bd894060 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.831+0000 7fa6ab7fe640 1 -- 172.21.15.45:0/3422102865 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6bc00fc60 con 0x55c2bd894060 2024-01-31T05:26:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.831+0000 7fa6ab7fe640 1 -- 172.21.15.45:0/3422102865 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc019c40 con 0x55c2bd894060 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.832+0000 7fa6ca384740 1 -- 172.21.15.45:0/3422102865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894060 msgr2=0x55c2bd894440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.832+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3422102865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894060 0x55c2bd894440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0115f0 tx=0x7fa6bc011ab0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.832+0000 7fa6ca384740 1 -- 172.21.15.45:0/3422102865 shutdown_connections 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.832+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3422102865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894980 0x55c2bd89d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.832+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3422102865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 0x55c2bd89fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.832+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3422102865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894060 0x55c2bd894440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.832+0000 7fa6ca384740 1 -- 172.21.15.45:0/3422102865 >> 172.21.15.45:0/3422102865 conn(0x55c2bd88e980 msgr2=0x55c2bd88ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.832+0000 7fa6ca384740 1 -- 172.21.15.45:0/3422102865 shutdown_connections 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.832+0000 7fa6ca384740 1 -- 172.21.15.45:0/3422102865 wait complete. 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.832+0000 7fa6ca384740 1 -- start start 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.832+0000 7fa6ca384740 1 Processor -- start 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6ca384740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 0x55c2bd89a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6ca384740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 0x55c2bd89abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6ca384740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 0x55c2bd89b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6ca384740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bd7b51b0 con 0x55c2bd894060 2024-01-31T05:26:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6ca384740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bd7b4e50 con 0x55c2bd894980 2024-01-31T05:26:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6ca384740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bd77a090 con 0x55c2bd89d750 2024-01-31T05:26:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6ca384740 10 client.0 _ll_register_callbacks cb 0x55c2bd79fa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:26:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6c527b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 0x55c2bd89a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6c527b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 0x55c2bd89a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51500/0 (socket says 172.21.15.45:51500) 2024-01-31T05:26:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6c527b640 1 -- 172.21.15.45:0/3443777881 learned_addr learned my addr 172.21.15.45:0/3443777881 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6c4a7a640 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 0x55c2bd89abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6c5a7c640 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 0x55c2bd89b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6c527b640 1 -- 172.21.15.45:0/3443777881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 msgr2=0x55c2bd89b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6c527b640 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 0x55c2bd89b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6c527b640 1 -- 172.21.15.45:0/3443777881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 msgr2=0x55c2bd89abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6c527b640 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 0x55c2bd89abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6c527b640 1 -- 172.21.15.45:0/3443777881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc009d40 con 0x55c2bd894060 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6c527b640 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 0x55c2bd89a6b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005d70 tx=0x7fa6bc005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a28ab63bf8f663 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6a8ff9640 1 -- 172.21.15.45:0/3443777881 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc01ae70 con 0x55c2bd894060 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6a8ff9640 1 -- 172.21.15.45:0/3443777881 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa6bc00f640 con 0x55c2bd894060 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6ca384740 1 -- 172.21.15.45:0/3443777881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2bd7f5010 con 0x55c2bd894060 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6a8ff9640 1 -- 172.21.15.45:0/3443777881 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6bc00f800 con 0x55c2bd894060 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6ca384740 10 client.15657 fetch_fsmap learned FSMap version 23 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.833+0000 7fa6ca384740 1 -- 172.21.15.45:0/3443777881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c2bd7fa6a0 con 0x55c2bd894060 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.834+0000 7fa6a8ff9640 1 -- 172.21.15.45:0/3443777881 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c2bd7f5010 con 0x55c2bd894060 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.834+0000 7fa6a8ff9640 1 -- 172.21.15.45:0/3443777881 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa6bc0046c0 con 0x55c2bd894060 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.834+0000 7fa6ca384740 10 client.15657 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:26:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.834+0000 7fa6ca384740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.834+0000 7fa6ca384740 20 client.15657 populate_metadata read hostname 'smithi045' 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.834+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c2bd8ad420 0x55c2bd8af860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.834+0000 7fa6ca384740 4 client.15657 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.834+0000 7fa6ca384740 1 -- 172.21.15.45:0/3443777881 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2bd7f4d30 con 0x55c2bd8ad420 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.834+0000 7fa6c4a7a640 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c2bd8ad420 0x55c2bd8af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.834+0000 7fa6c4a7a640 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c2bd8ad420 0x55c2bd8af860 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c702ab99e73b35 server_cookie=af5a58946b74597c in_seq=0 out_seq=0 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.834+0000 7fa6a8ff9640 10 client.15657 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.835+0000 7fa6a8ff9640 1 -- 172.21.15.45:0/3443777881 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2bd7f4d30 con 0x55c2bd8ad420 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.835+0000 7fa6a8ff9640 10 client.15657 handle_command_reply: tid=0 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.835+0000 7fa6ca384740 1 client.15657 shutdown 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.835+0000 7fa6ca384740 1 -- 172.21.15.45:0/3443777881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 msgr2=0x55c2bd89a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.835+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 0x55c2bd89a6b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005d70 tx=0x7fa6bc005da0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.835+0000 7fa6ca384740 1 -- 172.21.15.45:0/3443777881 shutdown_connections 2024-01-31T05:26:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.835+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c2bd8ad420 0x55c2bd8af860 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.835+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 0x55c2bd89b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.835+0000 7fa6a8ff9640 0 client.15657 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.835+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 0x55c2bd89abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.835+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3443777881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 0x55c2bd89a6b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.835+0000 7fa6ca384740 1 -- 172.21.15.45:0/3443777881 >> 172.21.15.45:0/3443777881 conn(0x55c2bd88e980 msgr2=0x55c2bd8906d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.836+0000 7fa6ca384740 1 -- 172.21.15.45:0/3443777881 shutdown_connections 2024-01-31T05:26:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.836+0000 7fa6ca384740 1 -- 172.21.15.45:0/3443777881 wait complete. 2024-01-31T05:26:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.836+0000 7fa6ca384740 20 client.15657 trim_cache size 0 max 16384 2024-01-31T05:26:00.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:00 smithi186 ceph-mon[35659]: pgmap v102: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-01-31T05:26:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.849+0000 7fa6ca384740 1 Processor -- start 2024-01-31T05:26:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.849+0000 7fa6ca384740 1 -- start start 2024-01-31T05:26:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6ca384740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 0x55c2bd89cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6ca384740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 0x55c2bd89af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6ca384740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 0x55c2bd897910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6ca384740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bd88a0b0 con 0x55c2bd894060 2024-01-31T05:26:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6ca384740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bd7f4d60 con 0x55c2bd894980 2024-01-31T05:26:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6ca384740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bd893a80 con 0x55c2bd89d750 2024-01-31T05:26:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6c527b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 0x55c2bd89cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6c527b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 0x55c2bd89cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51516/0 (socket says 172.21.15.45:51516) 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6c527b640 1 -- 172.21.15.45:0/684429900 learned_addr learned my addr 172.21.15.45:0/684429900 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6c4a7a640 1 --2- 172.21.15.45:0/684429900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 0x55c2bd89af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6c5a7c640 1 --2- 172.21.15.45:0/684429900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 0x55c2bd897910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6c527b640 1 -- 172.21.15.45:0/684429900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 msgr2=0x55c2bd897910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6c527b640 1 --2- 172.21.15.45:0/684429900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 0x55c2bd897910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6c527b640 1 -- 172.21.15.45:0/684429900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 msgr2=0x55c2bd89af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6c527b640 1 --2- 172.21.15.45:0/684429900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 0x55c2bd89af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6c527b640 1 -- 172.21.15.45:0/684429900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2bd797030 con 0x55c2bd894060 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6c527b640 1 --2- 172.21.15.45:0/684429900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 0x55c2bd89cde0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005eb0 tx=0x7fa6bc005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01a1398bf947c00 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6aaffd640 1 -- 172.21.15.45:0/684429900 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00f3c0 con 0x55c2bd894060 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6aaffd640 1 -- 172.21.15.45:0/684429900 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6bc004ee0 con 0x55c2bd894060 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.850+0000 7fa6aaffd640 1 -- 172.21.15.45:0/684429900 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc019bd0 con 0x55c2bd894060 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.851+0000 7fa6ca384740 1 -- 172.21.15.45:0/684429900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 msgr2=0x55c2bd89cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.851+0000 7fa6ca384740 1 --2- 172.21.15.45:0/684429900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 0x55c2bd89cde0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005eb0 tx=0x7fa6bc005970 comp rx=0 tx=0).stop 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.851+0000 7fa6ca384740 1 -- 172.21.15.45:0/684429900 shutdown_connections 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.851+0000 7fa6ca384740 1 --2- 172.21.15.45:0/684429900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd89d750 0x55c2bd897910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.851+0000 7fa6ca384740 1 --2- 172.21.15.45:0/684429900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 0x55c2bd89af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.851+0000 7fa6ca384740 1 --2- 172.21.15.45:0/684429900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd894060 0x55c2bd89cde0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.851+0000 7fa6ca384740 1 -- 172.21.15.45:0/684429900 >> 172.21.15.45:0/684429900 conn(0x55c2bd88e980 msgr2=0x55c2bd8b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.851+0000 7fa6ca384740 1 -- 172.21.15.45:0/684429900 shutdown_connections 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.851+0000 7fa6ca384740 1 -- 172.21.15.45:0/684429900 wait complete. 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.851+0000 7fa6ca384740 1 -- start start 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.851+0000 7fa6ca384740 1 Processor -- start 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6ca384740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd894060 0x55c2bd8a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6ca384740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 0x55c2bd8a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6ca384740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd89d750 0x55c2bd8a3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6ca384740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bd8a39d0 con 0x55c2bd89d750 2024-01-31T05:26:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6ca384740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bd8a3b40 con 0x55c2bd894980 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6ca384740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bd903990 con 0x55c2bd894060 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6ca384740 10 client.0 _ll_register_callbacks cb 0x7fa6bc009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6c5a7c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd89d750 0x55c2bd8a3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6c5a7c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd89d750 0x55c2bd8a3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51530/0 (socket says 172.21.15.45:51530) 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6c5a7c640 1 -- 172.21.15.45:0/3064428086 learned_addr learned my addr 172.21.15.45:0/3064428086 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6c4a7a640 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 0x55c2bd8a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6c527b640 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd894060 0x55c2bd8a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6c5a7c640 1 -- 172.21.15.45:0/3064428086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd894060 msgr2=0x55c2bd8a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6c5a7c640 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd894060 0x55c2bd8a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6c5a7c640 1 -- 172.21.15.45:0/3064428086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 msgr2=0x55c2bd8a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6c5a7c640 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 0x55c2bd8a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6c5a7c640 1 -- 172.21.15.45:0/3064428086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc01ae70 con 0x55c2bd89d750 2024-01-31T05:26:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6c5a7c640 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd89d750 0x55c2bd8a3330 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009cd0 tx=0x7fa6b802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34da183ae173f8d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.852+0000 7fa6a8ff9640 1 -- 172.21.15.45:0/3064428086 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8048070 con 0x55c2bd89d750 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6a8ff9640 1 -- 172.21.15.45:0/3064428086 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa6b8003970 con 0x55c2bd89d750 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6ca384740 1 -- 172.21.15.45:0/3064428086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6bc00fc60 con 0x55c2bd89d750 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6a8ff9640 1 -- 172.21.15.45:0/3064428086 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6b8003b30 con 0x55c2bd89d750 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6ca384740 10 client.15663 fetch_fsmap learned FSMap version 23 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6ca384740 1 -- 172.21.15.45:0/3064428086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c2bd7bca50 con 0x55c2bd89d750 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6a8ff9640 1 -- 172.21.15.45:0/3064428086 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6bc00fc60 con 0x55c2bd89d750 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6a8ff9640 1 -- 172.21.15.45:0/3064428086 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa6b8043430 con 0x55c2bd89d750 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6ca384740 10 client.15663 fetch_fsmap finished waiting for FSMap version 23 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6ca384740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6ca384740 20 client.15663 populate_metadata read hostname 'smithi045' 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c2bd905270 0x55c2bd907690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6ca384740 4 client.15663 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:26:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6ca384740 1 -- 172.21.15.45:0/3064428086 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2bd4fb600 con 0x55c2bd905270 2024-01-31T05:26:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.853+0000 7fa6c527b640 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c2bd905270 0x55c2bd907690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.854+0000 7fa6a8ff9640 10 client.15663 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:00.854+0000 7fa6c527b640 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c2bd905270 0x55c2bd907690 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b5a6f55ef1df10a server_cookie=35d29bfc54b406ff in_seq=0 out_seq=0 2024-01-31T05:26:00.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:00 smithi045 ceph-mon[31585]: pgmap v102: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-01-31T05:26:00.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:00 smithi156 ceph-mon[35720]: pgmap v102: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-01-31T05:26:01.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 269847 16.78 MB/sec execute 1 sec latency 61.406 ms 2024-01-31T05:26:02.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 272198 12.57 MB/sec execute 2 sec latency 13.327 ms 2024-01-31T05:26:02.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:02 smithi186 ceph-mon[35659]: pgmap v103: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-01-31T05:26:02.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:02 smithi045 ceph-mon[31585]: pgmap v103: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-01-31T05:26:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:02 smithi156 ceph-mon[35720]: pgmap v103: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-01-31T05:26:03.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 274120 12.36 MB/sec execute 3 sec latency 37.691 ms 2024-01-31T05:26:04.776 INFO:tasks.workunit.client.0.smithi045.stdout: 1 276905 13.75 MB/sec execute 4 sec latency 79.259 ms 2024-01-31T05:26:04.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:04 smithi186 ceph-mon[35659]: pgmap v104: 97 pgs: 97 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 374 op/s 2024-01-31T05:26:04.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:04 smithi045 ceph-mon[31585]: pgmap v104: 97 pgs: 97 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 374 op/s 2024-01-31T05:26:04.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:04 smithi156 ceph-mon[35720]: pgmap v104: 97 pgs: 97 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 374 op/s 2024-01-31T05:26:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.008+0000 7fa6a8ff9640 1 -- 172.21.15.45:0/3064428086 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2bd4fb600 con 0x55c2bd905270 2024-01-31T05:26:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.008+0000 7fa6a8ff9640 10 client.15663 handle_command_reply: tid=0 2024-01-31T05:26:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.008+0000 7fa6ca384740 1 client.15663 shutdown 2024-01-31T05:26:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.008+0000 7fa6ca384740 1 -- 172.21.15.45:0/3064428086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd89d750 msgr2=0x55c2bd8a3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.008+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd89d750 0x55c2bd8a3330 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009cd0 tx=0x7fa6b802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:26:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.008+0000 7fa6ca384740 1 -- 172.21.15.45:0/3064428086 shutdown_connections 2024-01-31T05:26:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.008+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c2bd905270 0x55c2bd907690 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2bd89d750 0x55c2bd8a3330 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2bd894980 0x55c2bd8a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6a8ff9640 0 client.15663 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6ca384740 1 --2- 172.21.15.45:0/3064428086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2bd894060 0x55c2bd8a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6ca384740 1 -- 172.21.15.45:0/3064428086 >> 172.21.15.45:0/3064428086 conn(0x55c2bd88e980 msgr2=0x55c2bd88d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6ca384740 1 -- 172.21.15.45:0/3064428086 shutdown_connections 2024-01-31T05:26:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6ca384740 1 -- 172.21.15.45:0/3064428086 wait complete. 2024-01-31T05:26:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6ca384740 20 client.15663 trim_cache size 0 max 16384 2024-01-31T05:26:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:26:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:26:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:26:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:26:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6c74df640 1 -- 172.21.15.45:0/356375746 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fa6ac0a14f0 msgr2=0x7fa6ac0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6c74df640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fa6ac0a14f0 0x7fa6ac0a39b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005cc0 tx=0x7fa6b8032ee0 comp rx=0 tx=0).stop 2024-01-31T05:26:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6c74df640 1 -- 172.21.15.45:0/356375746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c010c600 msgr2=0x7fa6c01a3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6c74df640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c010c600 0x7fa6c01a3950 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000e9f0 tx=0x7fa6b000eeb0 comp rx=0 tx=0).stop 2024-01-31T05:26:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6c74df640 1 -- 172.21.15.45:0/356375746 shutdown_connections 2024-01-31T05:26:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6c74df640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fa6ac0a14f0 0x7fa6ac0a39b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6c74df640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c010c600 0x7fa6c01a3950 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6c74df640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c019d610 0x7fa6c019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6c74df640 1 --2- 172.21.15.45:0/356375746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c0107530 0x7fa6c01a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6c74df640 1 -- 172.21.15.45:0/356375746 >> 172.21.15.45:0/356375746 conn(0x7fa6c0075260 msgr2=0x7fa6c0077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6c74df640 1 -- 172.21.15.45:0/356375746 shutdown_connections 2024-01-31T05:26:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:05.009+0000 7fa6c74df640 1 -- 172.21.15.45:0/356375746 wait complete. 2024-01-31T05:26:05.477 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: *** Caught signal (Segmentation fault) ** 2024-01-31T05:26:05.477 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: in thread 7f02ad5ec700 thread_name:md_log_replay 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f02be1acd20] 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 2: /usr/bin/ceph-mds(+0x5040a2) [0x558f5820f0a2] 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x558f582112f3] 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 4: (MDLog::_replay_thread()+0x647) [0x558f5826d697] 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 5: (MDLog::ReplayThread::entry()+0x11) [0x558f58276b31] 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f02be1a21ca] 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 7: clone() 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 2024-01-31T05:26:05.162+0000 7f02ad5ec700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: in thread 7f02ad5ec700 thread_name:md_log_replay 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f02be1acd20] 2024-01-31T05:26:05.478 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 2: /usr/bin/ceph-mds(+0x5040a2) [0x558f5820f0a2] 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x558f582112f3] 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 4: (MDLog::_replay_thread()+0x647) [0x558f5826d697] 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 5: (MDLog::ReplayThread::entry()+0x11) [0x558f58276b31] 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f02be1a21ca] 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 7: clone() 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 0> 2024-01-31T05:26:05.162+0000 7f02ad5ec700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: in thread 7f02ad5ec700 thread_name:md_log_replay 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f02be1acd20] 2024-01-31T05:26:05.479 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 2: /usr/bin/ceph-mds(+0x5040a2) [0x558f5820f0a2] 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x558f582112f3] 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 4: (MDLog::_replay_thread()+0x647) [0x558f5826d697] 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 5: (MDLog::ReplayThread::entry()+0x11) [0x558f58276b31] 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f02be1a21ca] 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 7: clone() 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 0> 2024-01-31T05:26:05.162+0000 7f02ad5ec700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: in thread 7f02ad5ec700 thread_name:md_log_replay 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f02be1acd20] 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 2: /usr/bin/ceph-mds(+0x5040a2) [0x558f5820f0a2] 2024-01-31T05:26:05.480 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x558f582112f3] 2024-01-31T05:26:05.481 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 4: (MDLog::_replay_thread()+0x647) [0x558f5826d697] 2024-01-31T05:26:05.481 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 5: (MDLog::ReplayThread::entry()+0x11) [0x558f58276b31] 2024-01-31T05:26:05.481 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f02be1a21ca] 2024-01-31T05:26:05.481 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 7: clone() 2024-01-31T05:26:05.481 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:26:05.481 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[66287]: 2024-01-31T05:26:05.481 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 podman[70119]: 2024-01-31 05:26:05.223933746 +0000 UTC m=+0.021100793 container died cdc5ea8ec148694b4c9a991c5bf8989c167aac325bdbceecdb43202b75771ffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-01-31T05:26:05.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 279275 12.67 MB/sec execute 5 sec latency 7.034 ms 2024-01-31T05:26:06.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 podman[70119]: 2024-01-31 05:26:05.944601335 +0000 UTC m=+0.741768356 container remove cdc5ea8ec148694b4c9a991c5bf8989c167aac325bdbceecdb43202b75771ffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, GIT_CLEAN=True) 2024-01-31T05:26:06.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:05 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-01-31T05:26:06.476 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:06 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-01-31T05:26:06.476 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:06 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Consumed 1.550s CPU time. 2024-01-31T05:26:06.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 281493 12.76 MB/sec execute 6 sec latency 8.515 ms 2024-01-31T05:26:06.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:06 smithi186 ceph-mon[35659]: pgmap v105: 97 pgs: 97 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-01-31T05:26:06.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:06 smithi045 ceph-mon[31585]: pgmap v105: 97 pgs: 97 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-01-31T05:26:06.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:06 smithi156 ceph-mon[35720]: pgmap v105: 97 pgs: 97 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-01-31T05:26:07.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 284225 13.36 MB/sec execute 7 sec latency 9.796 ms 2024-01-31T05:26:08.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 287147 13.51 MB/sec execute 8 sec latency 10.999 ms 2024-01-31T05:26:08.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:08 smithi186 ceph-mon[35659]: pgmap v106: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 368 op/s 2024-01-31T05:26:08.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:08 smithi045 ceph-mon[31585]: pgmap v106: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 368 op/s 2024-01-31T05:26:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:08 smithi156 ceph-mon[35720]: pgmap v106: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 368 op/s 2024-01-31T05:26:09.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 288487 12.75 MB/sec execute 9 sec latency 84.812 ms 2024-01-31T05:26:10.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 291263 13.18 MB/sec execute 10 sec latency 16.353 ms 2024-01-31T05:26:10.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:10 smithi186 ceph-mon[35659]: pgmap v107: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-01-31T05:26:10.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:10 smithi045 ceph-mon[31585]: pgmap v107: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-01-31T05:26:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:10 smithi156 ceph-mon[35720]: pgmap v107: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-01-31T05:26:11.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 293896 13.13 MB/sec execute 11 sec latency 26.656 ms 2024-01-31T05:26:12.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 296286 13.03 MB/sec execute 12 sec latency 10.992 ms 2024-01-31T05:26:12.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:12 smithi186 ceph-mon[35659]: pgmap v108: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-01-31T05:26:12.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:12 smithi045 ceph-mon[31585]: pgmap v108: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-01-31T05:26:12.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:12 smithi156 ceph-mon[35720]: pgmap v108: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-01-31T05:26:13.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 298589 13.08 MB/sec execute 13 sec latency 16.060 ms 2024-01-31T05:26:14.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 301464 13.39 MB/sec execute 14 sec latency 114.781 ms 2024-01-31T05:26:14.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:14 smithi186 ceph-mon[35659]: pgmap v109: 97 pgs: 97 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 371 op/s 2024-01-31T05:26:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:14 smithi045 ceph-mon[31585]: pgmap v109: 97 pgs: 97 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 371 op/s 2024-01-31T05:26:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:14 smithi156 ceph-mon[35720]: pgmap v109: 97 pgs: 97 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 371 op/s 2024-01-31T05:26:15.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 303850 13.07 MB/sec execute 15 sec latency 21.854 ms 2024-01-31T05:26:16.526 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:16 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 1. 2024-01-31T05:26:16.526 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:16 smithi156 systemd[1]: Stopped Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:26:16.526 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:16 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Consumed 1.550s CPU time. 2024-01-31T05:26:16.526 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:16 smithi156 systemd[1]: Starting Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:26:16.526 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:16 smithi156 podman[70222]: 2024-01-31 05:26:16.353277533 +0000 UTC m=+0.017910150 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:26:16.527 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:16 smithi156 podman[70222]: 2024-01-31 05:26:16.527884372 +0000 UTC m=+0.192516979 container create e62386f272badf26e91fc8224e715afc91bc1c1f157b60f20c521561afddbc3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3) 2024-01-31T05:26:16.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 305856 13.04 MB/sec execute 16 sec latency 23.497 ms 2024-01-31T05:26:16.796 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:16 smithi156 ceph-mon[35720]: pgmap v110: 97 pgs: 97 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 227 op/s 2024-01-31T05:26:16.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:16 smithi186 ceph-mon[35659]: pgmap v110: 97 pgs: 97 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 227 op/s 2024-01-31T05:26:16.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:16 smithi045 ceph-mon[31585]: pgmap v110: 97 pgs: 97 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 227 op/s 2024-01-31T05:26:17.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:16 smithi156 podman[70222]: 2024-01-31 05:26:16.798746378 +0000 UTC m=+0.463378993 container init e62386f272badf26e91fc8224e715afc91bc1c1f157b60f20c521561afddbc3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-01-31T05:26:17.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:16 smithi156 podman[70222]: 2024-01-31 05:26:16.802804927 +0000 UTC m=+0.467437527 container start e62386f272badf26e91fc8224e715afc91bc1c1f157b60f20c521561afddbc3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-31T05:26:17.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:16 smithi156 ceph-mds[70242]: -- 172.21.15.156:0/1298782724 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5606ec32f520 con 0x5606ec305000 2024-01-31T05:26:17.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:16 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: starting mds.h at 2024-01-31T05:26:17.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:16 smithi156 bash[70222]: e62386f272badf26e91fc8224e715afc91bc1c1f157b60f20c521561afddbc3f 2024-01-31T05:26:17.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:26:16 smithi156 systemd[1]: Started Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:26:17.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 308864 13.36 MB/sec execute 17 sec latency 14.133 ms 2024-01-31T05:26:17.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:17 smithi186 ceph-mon[35659]: daemon mds.h restarted 2024-01-31T05:26:17.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:17 smithi186 ceph-mon[35659]: osdmap e90: 12 total, 12 up, 12 in 2024-01-31T05:26:17.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:17 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.156:6836/2256765668,v1:172.21.15.156:6837/2256765668] up:boot 2024-01-31T05:26:17.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:17 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:26:17.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:17 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:26:17.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:17 smithi045 ceph-mon[31585]: daemon mds.h restarted 2024-01-31T05:26:17.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:17 smithi045 ceph-mon[31585]: osdmap e90: 12 total, 12 up, 12 in 2024-01-31T05:26:17.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:17 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.156:6836/2256765668,v1:172.21.15.156:6837/2256765668] up:boot 2024-01-31T05:26:17.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:17 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:26:17.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:17 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:26:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:17 smithi156 ceph-mon[35720]: daemon mds.h restarted 2024-01-31T05:26:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:17 smithi156 ceph-mon[35720]: osdmap e90: 12 total, 12 up, 12 in 2024-01-31T05:26:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:17 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.156:6836/2256765668,v1:172.21.15.156:6837/2256765668] up:boot 2024-01-31T05:26:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:17 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:26:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:17 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:26:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.101+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/1765064421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3d4105fe0 msgr2=0x7fe3d410dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.101+0000 7fe3d8bc6640 1 --2- 172.21.15.45:0/1765064421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3d4105fe0 0x7fe3d410dbb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009930 tx=0x7fe3cc02bc30 comp rx=0 tx=0).stop 2024-01-31T05:26:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.102+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/1765064421 shutdown_connections 2024-01-31T05:26:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.102+0000 7fe3d8bc6640 1 --2- 172.21.15.45:0/1765064421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3d4105fe0 0x7fe3d410dbb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.102+0000 7fe3d8bc6640 1 --2- 172.21.15.45:0/1765064421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3d41018c0 0x7fe3d4105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.102+0000 7fe3d8bc6640 1 --2- 172.21.15.45:0/1765064421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d4100f10 0x7fe3d41012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.102+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/1765064421 >> 172.21.15.45:0/1765064421 conn(0x7fe3d40fc7b0 msgr2=0x7fe3d40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.102+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/1765064421 shutdown_connections 2024-01-31T05:26:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.102+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/1765064421 wait complete. 2024-01-31T05:26:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.102+0000 7fe3d8bc6640 1 Processor -- start 2024-01-31T05:26:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.102+0000 7fe3d8bc6640 1 -- start start 2024-01-31T05:26:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d8bc6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3d4100f10 0x7fe3d41971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d8bc6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3d4105fe0 0x7fe3d4197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d8bc6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d419ba40 0x7fe3d419bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d8bc6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d419c430 con 0x7fe3d4105fe0 2024-01-31T05:26:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d8bc6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d419c5a0 con 0x7fe3d419ba40 2024-01-31T05:26:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d8bc6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d419c710 con 0x7fe3d4100f10 2024-01-31T05:26:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3d4105fe0 0x7fe3d4197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d259c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3d4100f10 0x7fe3d41971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3d4105fe0 0x7fe3d4197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37530/0 (socket says 172.21.15.45:37530) 2024-01-31T05:26:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d1d9b640 1 -- 172.21.15.45:0/505641168 learned_addr learned my addr 172.21.15.45:0/505641168 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d2d9d640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d419ba40 0x7fe3d419bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d1d9b640 1 -- 172.21.15.45:0/505641168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3d4100f10 msgr2=0x7fe3d41971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3d4100f10 0x7fe3d41971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d1d9b640 1 -- 172.21.15.45:0/505641168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d419ba40 msgr2=0x7fe3d419bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d419ba40 0x7fe3d419bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d1d9b640 1 -- 172.21.15.45:0/505641168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc009560 con 0x7fe3d4105fe0 2024-01-31T05:26:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d259c640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3d4100f10 0x7fe3d41971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:26:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3d4105fe0 0x7fe3d4197710 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400eb90 tx=0x7fe3c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3cb7fe640 1 -- 172.21.15.45:0/505641168 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c40099b0 con 0x7fe3d4105fe0 2024-01-31T05:26:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3cb7fe640 1 -- 172.21.15.45:0/505641168 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3c400ee90 con 0x7fe3d4105fe0 2024-01-31T05:26:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/505641168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d41a3cf0 con 0x7fe3d4105fe0 2024-01-31T05:26:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3cb7fe640 1 -- 172.21.15.45:0/505641168 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4018a60 con 0x7fe3d4105fe0 2024-01-31T05:26:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.103+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/505641168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d41a41a0 con 0x7fe3d4105fe0 2024-01-31T05:26:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.105+0000 7fe3cb7fe640 1 -- 172.21.15.45:0/505641168 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fe3c4018bc0 con 0x7fe3d4105fe0 2024-01-31T05:26:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.106+0000 7fe3cb7fe640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe3c00a14f0 0x7fe3c00a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.106+0000 7fe3db981740 1 Processor -- start 2024-01-31T05:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.106+0000 7fe3d259c640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe3c00a14f0 0x7fe3c00a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.106+0000 7fe3cb7fe640 1 -- 172.21.15.45:0/505641168 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe3c4014070 con 0x7fe3d4105fe0 2024-01-31T05:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.106+0000 7fe3db981740 1 -- start start 2024-01-31T05:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3db981740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b060 0x560e9be6b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3db981740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 0x560e9be74210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3db981740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be74750 0x560e9be76b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3db981740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9bdda040 con 0x560e9be6b980 2024-01-31T05:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3db981740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9bdd7730 con 0x560e9be6b060 2024-01-31T05:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3db981740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9bdd3020 con 0x560e9be74750 2024-01-31T05:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3d1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 0x560e9be74210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3d259c640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe3c00a14f0 0x7fe3c00a39b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00bfd0 tx=0x7fe3bc00b390 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3d1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 0x560e9be74210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37532/0 (socket says 172.21.15.45:37532) 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3d1d9b640 1 -- 172.21.15.45:0/618871052 learned_addr learned my addr 172.21.15.45:0/618871052 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3d2d9d640 1 --2- 172.21.15.45:0/618871052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be74750 0x560e9be76b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3d259c640 1 --2- 172.21.15.45:0/618871052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b060 0x560e9be6b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3d1d9b640 1 -- 172.21.15.45:0/618871052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be74750 msgr2=0x560e9be76b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/618871052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be74750 0x560e9be76b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3d1d9b640 1 -- 172.21.15.45:0/618871052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b060 msgr2=0x560e9be6b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/618871052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b060 0x560e9be6b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3d1d9b640 1 -- 172.21.15.45:0/618871052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e9bdcae40 con 0x560e9be6b980 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/618871052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 0x560e9be74210 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4096810 tx=0x7fe3c4099f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e8b4912bad1935 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3c8ff9640 1 -- 172.21.15.45:0/618871052 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c40adea0 con 0x560e9be6b980 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3c8ff9640 1 -- 172.21.15.45:0/618871052 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3c40a25f0 con 0x560e9be6b980 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.107+0000 7fe3c8ff9640 1 -- 172.21.15.45:0/618871052 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c40cafe0 con 0x560e9be6b980 2024-01-31T05:26:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.108+0000 7fe3db981740 1 -- 172.21.15.45:0/618871052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 msgr2=0x560e9be74210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.108+0000 7fe3db981740 1 --2- 172.21.15.45:0/618871052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 0x560e9be74210 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4096810 tx=0x7fe3c4099f50 comp rx=0 tx=0).stop 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.108+0000 7fe3db981740 1 -- 172.21.15.45:0/618871052 shutdown_connections 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.108+0000 7fe3db981740 1 --2- 172.21.15.45:0/618871052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 0x560e9be74210 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.108+0000 7fe3db981740 1 --2- 172.21.15.45:0/618871052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be74750 0x560e9be76b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.108+0000 7fe3db981740 1 --2- 172.21.15.45:0/618871052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b060 0x560e9be6b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.108+0000 7fe3db981740 1 -- 172.21.15.45:0/618871052 >> 172.21.15.45:0/618871052 conn(0x560e9be65980 msgr2=0x560e9be65d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.108+0000 7fe3db981740 1 -- 172.21.15.45:0/618871052 shutdown_connections 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.108+0000 7fe3db981740 1 -- 172.21.15.45:0/618871052 wait complete. 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3db981740 1 -- start start 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3db981740 1 Processor -- start 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3db981740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b060 0x560e9be71730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3db981740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 0x560e9be71c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3db981740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be74750 0x560e9be721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3db981740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9bd8c1b0 con 0x560e9be6b980 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3db981740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9bd8be50 con 0x560e9be6b060 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3db981740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9bd51090 con 0x560e9be74750 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3db981740 10 client.0 _ll_register_callbacks cb 0x560e9bd76a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3d1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 0x560e9be71c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3d1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 0x560e9be71c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37538/0 (socket says 172.21.15.45:37538) 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3d1d9b640 1 -- 172.21.15.45:0/1411217004 learned_addr learned my addr 172.21.15.45:0/1411217004 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3d259c640 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b060 0x560e9be71730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3d2d9d640 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be74750 0x560e9be721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3d1d9b640 1 -- 172.21.15.45:0/1411217004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be74750 msgr2=0x560e9be721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be74750 0x560e9be721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3d1d9b640 1 -- 172.21.15.45:0/1411217004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b060 msgr2=0x560e9be71730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b060 0x560e9be71730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3d1d9b640 1 -- 172.21.15.45:0/1411217004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d41a41a0 con 0x560e9be6b980 2024-01-31T05:26:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 0x560e9be71c70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40cac60 tx=0x7fe3c409e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abd97413f35a11c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.109+0000 7fe3b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:26:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3b9ffb640 1 -- 172.21.15.45:0/1411217004 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c40b8070 con 0x560e9be6b980 2024-01-31T05:26:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3b9ffb640 1 -- 172.21.15.45:0/1411217004 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe3c409e160 con 0x560e9be6b980 2024-01-31T05:26:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3db981740 1 -- 172.21.15.45:0/1411217004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e9bdcc010 con 0x560e9be6b980 2024-01-31T05:26:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3b9ffb640 1 -- 172.21.15.45:0/1411217004 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3c40adea0 con 0x560e9be6b980 2024-01-31T05:26:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3db981740 10 client.15681 fetch_fsmap learned FSMap version 24 2024-01-31T05:26:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3db981740 1 -- 172.21.15.45:0/1411217004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560e9bdd16a0 con 0x560e9be6b980 2024-01-31T05:26:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3b9ffb640 1 -- 172.21.15.45:0/1411217004 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e9bdcc010 con 0x560e9be6b980 2024-01-31T05:26:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3b9ffb640 1 -- 172.21.15.45:0/1411217004 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe3c40b2cd0 con 0x560e9be6b980 2024-01-31T05:26:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3db981740 10 client.15681 fetch_fsmap finished waiting for FSMap version 24 2024-01-31T05:26:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3db981740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:26:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3db981740 20 client.15681 populate_metadata read hostname 'smithi045' 2024-01-31T05:26:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3db981740 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560e9be84190 0x560e9be86580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3db981740 4 client.15681 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:26:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3db981740 1 -- 172.21.15.45:0/1411217004 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e9bdcbd30 con 0x560e9be84190 2024-01-31T05:26:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.110+0000 7fe3d259c640 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560e9be84190 0x560e9be86580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3b9ffb640 10 client.15681 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3d259c640 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560e9be84190 0x560e9be86580 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bd6f80d2d130b06 server_cookie=db363ca21eb81fd2 in_seq=0 out_seq=0 2024-01-31T05:26:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3b9ffb640 1 -- 172.21.15.45:0/1411217004 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e9bdcbd30 con 0x560e9be84190 2024-01-31T05:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3b9ffb640 10 client.15681 handle_command_reply: tid=0 2024-01-31T05:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3db981740 1 client.15681 shutdown 2024-01-31T05:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3db981740 1 -- 172.21.15.45:0/1411217004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 msgr2=0x560e9be71c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3db981740 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 0x560e9be71c70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40cac60 tx=0x7fe3c409e530 comp rx=0 tx=0).stop 2024-01-31T05:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3db981740 1 -- 172.21.15.45:0/1411217004 shutdown_connections 2024-01-31T05:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3db981740 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560e9be84190 0x560e9be86580 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3db981740 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b980 0x560e9be71c70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3db981740 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be74750 0x560e9be721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3db981740 1 --2- 172.21.15.45:0/1411217004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b060 0x560e9be71730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3b9ffb640 0 client.15681 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.111+0000 7fe3db981740 1 -- 172.21.15.45:0/1411217004 >> 172.21.15.45:0/1411217004 conn(0x560e9be65980 msgr2=0x560e9be676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.112+0000 7fe3db981740 1 -- 172.21.15.45:0/1411217004 shutdown_connections 2024-01-31T05:26:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.112+0000 7fe3db981740 1 -- 172.21.15.45:0/1411217004 wait complete. 2024-01-31T05:26:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.112+0000 7fe3db981740 20 client.15681 trim_cache size 0 max 16384 2024-01-31T05:26:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3db981740 1 Processor -- start 2024-01-31T05:26:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3db981740 1 -- start start 2024-01-31T05:26:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3db981740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be6b060 0x560e9be73e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3db981740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b980 0x560e9be71fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3db981740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be74750 0x560e9be6e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3db981740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9be612c0 con 0x560e9be74750 2024-01-31T05:26:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3db981740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9be6fc80 con 0x560e9be6b980 2024-01-31T05:26:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3db981740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9be6b770 con 0x560e9be6b060 2024-01-31T05:26:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3d2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be74750 0x560e9be6e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3d2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be74750 0x560e9be6e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37544/0 (socket says 172.21.15.45:37544) 2024-01-31T05:26:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3d2d9d640 1 -- 172.21.15.45:0/3137253634 learned_addr learned my addr 172.21.15.45:0/3137253634 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3d259c640 1 --2- 172.21.15.45:0/3137253634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be6b060 0x560e9be73e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/3137253634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b980 0x560e9be71fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3d2d9d640 1 -- 172.21.15.45:0/3137253634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be6b060 msgr2=0x560e9be73e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3d2d9d640 1 --2- 172.21.15.45:0/3137253634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be6b060 0x560e9be73e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3d2d9d640 1 -- 172.21.15.45:0/3137253634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b980 msgr2=0x560e9be71fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3d2d9d640 1 --2- 172.21.15.45:0/3137253634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b980 0x560e9be71fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.126+0000 7fe3d2d9d640 1 -- 172.21.15.45:0/3137253634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e9bd6e030 con 0x560e9be74750 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3d2d9d640 1 --2- 172.21.15.45:0/3137253634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be74750 0x560e9be6e990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc005ab0 tx=0x7fe3cc02c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6243315d49b088eb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3bbfff640 1 -- 172.21.15.45:0/3137253634 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc045070 con 0x560e9be74750 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3bbfff640 1 -- 172.21.15.45:0/3137253634 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3cc004a20 con 0x560e9be74750 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3bbfff640 1 -- 172.21.15.45:0/3137253634 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc003770 con 0x560e9be74750 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3db981740 1 -- 172.21.15.45:0/3137253634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be74750 msgr2=0x560e9be6e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3db981740 1 --2- 172.21.15.45:0/3137253634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be74750 0x560e9be6e990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc005ab0 tx=0x7fe3cc02c6e0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3db981740 1 -- 172.21.15.45:0/3137253634 shutdown_connections 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3db981740 1 --2- 172.21.15.45:0/3137253634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be74750 0x560e9be6e990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3db981740 1 --2- 172.21.15.45:0/3137253634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be6b980 0x560e9be71fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3db981740 1 --2- 172.21.15.45:0/3137253634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be6b060 0x560e9be73e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3db981740 1 -- 172.21.15.45:0/3137253634 >> 172.21.15.45:0/3137253634 conn(0x560e9be65980 msgr2=0x560e9be6c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3db981740 1 -- 172.21.15.45:0/3137253634 shutdown_connections 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.127+0000 7fe3db981740 1 -- 172.21.15.45:0/3137253634 wait complete. 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3db981740 1 -- start start 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3db981740 1 Processor -- start 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3db981740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b060 0x560e9be7ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3db981740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be6b980 0x560e9be7b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3db981740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be74750 0x560e9bed3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3db981740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9bed3cc0 con 0x560e9be6b060 2024-01-31T05:26:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3db981740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9bed3e30 con 0x560e9be74750 2024-01-31T05:26:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3db981740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9bed3ff0 con 0x560e9be6b980 2024-01-31T05:26:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3db981740 10 client.0 _ll_register_callbacks cb 0x7fe3c4095f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:26:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3d259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b060 0x560e9be7ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3d259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b060 0x560e9be7ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37552/0 (socket says 172.21.15.45:37552) 2024-01-31T05:26:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3d259c640 1 -- 172.21.15.45:0/3467005644 learned_addr learned my addr 172.21.15.45:0/3467005644 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be6b980 0x560e9be7b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3d2d9d640 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be74750 0x560e9bed3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3d259c640 1 -- 172.21.15.45:0/3467005644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be6b980 msgr2=0x560e9be7b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3d259c640 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be6b980 0x560e9be7b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3d259c640 1 -- 172.21.15.45:0/3467005644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be74750 msgr2=0x560e9bed3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3d259c640 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be74750 0x560e9bed3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.128+0000 7fe3d259c640 1 -- 172.21.15.45:0/3467005644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d410e3c0 con 0x560e9be6b060 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3d259c640 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b060 0x560e9be7ab20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x560e9be86e90 tx=0x7fe3bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baefa075a863910d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3b9ffb640 1 -- 172.21.15.45:0/3467005644 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc023040 con 0x560e9be6b060 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3b9ffb640 1 -- 172.21.15.45:0/3467005644 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe3bc017070 con 0x560e9be6b060 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3db981740 1 -- 172.21.15.45:0/3467005644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3c40a25f0 con 0x560e9be6b060 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3b9ffb640 1 -- 172.21.15.45:0/3467005644 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3bc028860 con 0x560e9be6b060 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3db981740 10 client.15687 fetch_fsmap learned FSMap version 24 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3db981740 1 -- 172.21.15.45:0/3467005644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560e9bd93a50 con 0x560e9be6b060 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3b9ffb640 1 -- 172.21.15.45:0/3467005644 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3c40a25f0 con 0x560e9be6b060 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3b9ffb640 1 -- 172.21.15.45:0/3467005644 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe3bc01a040 con 0x560e9be6b060 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3db981740 10 client.15687 fetch_fsmap finished waiting for FSMap version 24 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3db981740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:26:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3db981740 20 client.15687 populate_metadata read hostname 'smithi045' 2024-01-31T05:26:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3db981740 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560e9bedcba0 0x560e9bedefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3db981740 4 client.15687 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:26:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3db981740 1 -- 172.21.15.45:0/3467005644 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560e9bad2600 con 0x560e9bedcba0 2024-01-31T05:26:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.129+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560e9bedcba0 0x560e9bedefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.130+0000 7fe3d1d9b640 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560e9bedcba0 0x560e9bedefc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce1a4234037b22ee server_cookie=d319b1032b051ce4 in_seq=0 out_seq=0 2024-01-31T05:26:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.130+0000 7fe3b9ffb640 10 client.15687 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3b9ffb640 1 -- 172.21.15.45:0/3467005644 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560e9bad2600 con 0x560e9bedcba0 2024-01-31T05:26:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3b9ffb640 10 client.15687 handle_command_reply: tid=0 2024-01-31T05:26:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3db981740 1 client.15687 shutdown 2024-01-31T05:26:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3db981740 1 -- 172.21.15.45:0/3467005644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b060 msgr2=0x560e9be7ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3db981740 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b060 0x560e9be7ab20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x560e9be86e90 tx=0x7fe3bc004730 comp rx=0 tx=0).stop 2024-01-31T05:26:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3db981740 1 -- 172.21.15.45:0/3467005644 shutdown_connections 2024-01-31T05:26:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3db981740 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560e9bedcba0 0x560e9bedefc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3db981740 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9be74750 0x560e9bed3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3db981740 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e9be6b980 0x560e9be7b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3b9ffb640 0 client.15687 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3db981740 1 --2- 172.21.15.45:0/3467005644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e9be6b060 0x560e9be7ab20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3db981740 1 -- 172.21.15.45:0/3467005644 >> 172.21.15.45:0/3467005644 conn(0x560e9be65980 msgr2=0x560e9be66d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3db981740 1 -- 172.21.15.45:0/3467005644 shutdown_connections 2024-01-31T05:26:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3db981740 1 -- 172.21.15.45:0/3467005644 wait complete. 2024-01-31T05:26:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.138+0000 7fe3db981740 20 client.15687 trim_cache size 0 max 16384 2024-01-31T05:26:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:26:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:26:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:26:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:26:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.139+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/505641168 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe3c00a14f0 msgr2=0x7fe3c00a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.139+0000 7fe3d8bc6640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe3c00a14f0 0x7fe3c00a39b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00bfd0 tx=0x7fe3bc00b390 comp rx=0 tx=0).stop 2024-01-31T05:26:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.139+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/505641168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3d4105fe0 msgr2=0x7fe3d4197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.139+0000 7fe3d8bc6640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3d4105fe0 0x7fe3d4197710 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400eb90 tx=0x7fe3c400c6a0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.139+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/505641168 shutdown_connections 2024-01-31T05:26:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.139+0000 7fe3d8bc6640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe3c00a14f0 0x7fe3c00a39b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.139+0000 7fe3d8bc6640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3d4105fe0 0x7fe3d4197710 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.139+0000 7fe3d8bc6640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d419ba40 0x7fe3d419bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.139+0000 7fe3d8bc6640 1 --2- 172.21.15.45:0/505641168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3d4100f10 0x7fe3d41971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.139+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/505641168 >> 172.21.15.45:0/505641168 conn(0x7fe3d40fc7b0 msgr2=0x7fe3d40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.139+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/505641168 shutdown_connections 2024-01-31T05:26:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:18.139+0000 7fe3d8bc6640 1 -- 172.21.15.45:0/505641168 wait complete. 2024-01-31T05:26:18.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 310507 12.94 MB/sec execute 18 sec latency 223.826 ms 2024-01-31T05:26:18.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:18 smithi186 ceph-mon[35659]: pgmap v111: 97 pgs: 97 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 339 op/s 2024-01-31T05:26:18.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:18 smithi045 ceph-mon[31585]: pgmap v111: 97 pgs: 97 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 339 op/s 2024-01-31T05:26:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:18 smithi156 ceph-mon[35720]: pgmap v111: 97 pgs: 97 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 339 op/s 2024-01-31T05:26:19.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 312431 12.96 MB/sec execute 19 sec latency 288.054 ms 2024-01-31T05:26:20.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 314709 12.72 MB/sec execute 20 sec latency 13.453 ms 2024-01-31T05:26:20.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:20 smithi186 ceph-mon[35659]: pgmap v113: 97 pgs: 97 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 274 op/s 2024-01-31T05:26:20.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:20 smithi045 ceph-mon[31585]: pgmap v113: 97 pgs: 97 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 274 op/s 2024-01-31T05:26:20.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:20 smithi156 ceph-mon[35720]: pgmap v113: 97 pgs: 97 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 274 op/s 2024-01-31T05:26:21.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 316742 12.74 MB/sec execute 21 sec latency 8.705 ms 2024-01-31T05:26:22.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 319642 12.94 MB/sec execute 22 sec latency 11.347 ms 2024-01-31T05:26:22.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:22 smithi186 ceph-mon[35659]: pgmap v114: 97 pgs: 97 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 274 op/s 2024-01-31T05:26:22.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:22 smithi045 ceph-mon[31585]: pgmap v114: 97 pgs: 97 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 274 op/s 2024-01-31T05:26:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:22 smithi156 ceph-mon[35720]: pgmap v114: 97 pgs: 97 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 274 op/s 2024-01-31T05:26:23.777 INFO:tasks.workunit.client.0.smithi045.stdout: 1 322124 12.87 MB/sec execute 23 sec latency 83.602 ms 2024-01-31T05:26:24.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 323708 12.69 MB/sec execute 24 sec latency 88.719 ms 2024-01-31T05:26:24.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:24 smithi186 ceph-mon[35659]: pgmap v115: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 267 op/s 2024-01-31T05:26:24.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:24 smithi045 ceph-mon[31585]: pgmap v115: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 267 op/s 2024-01-31T05:26:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:24 smithi156 ceph-mon[35720]: pgmap v115: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 267 op/s 2024-01-31T05:26:25.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 326581 12.91 MB/sec execute 25 sec latency 25.863 ms 2024-01-31T05:26:26.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 329104 12.85 MB/sec execute 26 sec latency 8.589 ms 2024-01-31T05:26:26.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:26 smithi186 ceph-mon[35659]: pgmap v116: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 267 op/s 2024-01-31T05:26:26.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:26 smithi045 ceph-mon[31585]: pgmap v116: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 267 op/s 2024-01-31T05:26:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:26 smithi156 ceph-mon[35720]: pgmap v116: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 267 op/s 2024-01-31T05:26:27.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 331165 12.75 MB/sec execute 27 sec latency 11.427 ms 2024-01-31T05:26:28.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 333871 12.90 MB/sec execute 28 sec latency 10.255 ms 2024-01-31T05:26:28.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:28 smithi186 ceph-mon[35659]: pgmap v117: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 264 op/s 2024-01-31T05:26:28.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:28 smithi045 ceph-mon[31585]: pgmap v117: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 264 op/s 2024-01-31T05:26:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:28 smithi156 ceph-mon[35720]: pgmap v117: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 264 op/s 2024-01-31T05:26:29.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 335989 12.73 MB/sec execute 29 sec latency 77.751 ms 2024-01-31T05:26:30.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 338045 12.74 MB/sec execute 30 sec latency 18.545 ms 2024-01-31T05:26:30.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:30 smithi186 ceph-mon[35659]: pgmap v118: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 221 op/s 2024-01-31T05:26:30.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:30 smithi045 ceph-mon[31585]: pgmap v118: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 221 op/s 2024-01-31T05:26:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:30 smithi156 ceph-mon[35720]: pgmap v118: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 221 op/s 2024-01-31T05:26:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.229+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3395189206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c241018e0 msgr2=0x7f2c24105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.229+0000 7f2c2bfbb640 1 --2- 172.21.15.45:0/3395189206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c241018e0 0x7f2c24105920 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009930 tx=0x7f2c1402bc30 comp rx=0 tx=0).stop 2024-01-31T05:26:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.229+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3395189206 shutdown_connections 2024-01-31T05:26:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.229+0000 7f2c2bfbb640 1 --2- 172.21.15.45:0/3395189206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c241018e0 0x7f2c24105920 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.229+0000 7f2c2bfbb640 1 --2- 172.21.15.45:0/3395189206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c24106000 0x7f2c2410dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.229+0000 7f2c2bfbb640 1 --2- 172.21.15.45:0/3395189206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24100f30 0x7f2c24101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.229+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3395189206 >> 172.21.15.45:0/3395189206 conn(0x7f2c240fc7d0 msgr2=0x7f2c240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.229+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3395189206 shutdown_connections 2024-01-31T05:26:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.229+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3395189206 wait complete. 2024-01-31T05:26:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.229+0000 7f2c2bfbb640 1 Processor -- start 2024-01-31T05:26:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.229+0000 7f2c2bfbb640 1 -- start start 2024-01-31T05:26:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c2bfbb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c24100f30 0x7f2c24072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c2bfbb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24106000 0x7f2c24073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c2bfbb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c24073a70 0x7f2c241a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c2bfbb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c241a59e0 con 0x7f2c24100f30 2024-01-31T05:26:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c2bfbb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c241a5b50 con 0x7f2c24106000 2024-01-31T05:26:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c2bfbb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c241a5d10 con 0x7f2c24073a70 2024-01-31T05:26:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c29d57640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c24100f30 0x7f2c24072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c29d57640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c24100f30 0x7f2c24072ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33592/0 (socket says 172.21.15.45:33592) 2024-01-31T05:26:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c29d57640 1 -- 172.21.15.45:0/3438272715 learned_addr learned my addr 172.21.15.45:0/3438272715 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c29556640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24106000 0x7f2c24073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c2a558640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c24073a70 0x7f2c241a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c29d57640 1 -- 172.21.15.45:0/3438272715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c24073a70 msgr2=0x7f2c241a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c29d57640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c24073a70 0x7f2c241a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c29d57640 1 -- 172.21.15.45:0/3438272715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24106000 msgr2=0x7f2c24073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c29d57640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24106000 0x7f2c24073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c29d57640 1 -- 172.21.15.45:0/3438272715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14009560 con 0x7f2c24100f30 2024-01-31T05:26:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c29d57640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c24100f30 0x7f2c24072ff0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000d950 tx=0x7f2c2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c1affd640 1 -- 172.21.15.45:0/3438272715 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20009c30 con 0x7f2c24100f30 2024-01-31T05:26:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.230+0000 7f2c1affd640 1 -- 172.21.15.45:0/3438272715 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c20009d90 con 0x7f2c24100f30 2024-01-31T05:26:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.231+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3438272715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c241a5ff0 con 0x7f2c24100f30 2024-01-31T05:26:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.231+0000 7f2c1affd640 1 -- 172.21.15.45:0/3438272715 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2000f740 con 0x7f2c24100f30 2024-01-31T05:26:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.231+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3438272715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c241a6420 con 0x7f2c24100f30 2024-01-31T05:26:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.233+0000 7f2c1affd640 1 -- 172.21.15.45:0/3438272715 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f2c20010a20 con 0x7f2c24100f30 2024-01-31T05:26:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.233+0000 7f2c1affd640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2c100a1520 0x7f2c100a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.233+0000 7f2c2ed7c740 1 Processor -- start 2024-01-31T05:26:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.233+0000 7f2c29556640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2c100a1520 0x7f2c100a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.233+0000 7f2c1affd640 1 -- 172.21.15.45:0/3438272715 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2c200cc390 con 0x7f2c24100f30 2024-01-31T05:26:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.233+0000 7f2c2ed7c740 1 -- start start 2024-01-31T05:26:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c2ed7c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa80f060 0x5565fa80f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c2ed7c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa818210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c2ed7c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa818750 0x5565fa81ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c2ed7c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fa77e040 con 0x5565fa80f980 2024-01-31T05:26:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c2ed7c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fa77b730 con 0x5565fa818750 2024-01-31T05:26:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c2ed7c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fa777020 con 0x5565fa80f060 2024-01-31T05:26:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c29d57640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa80f060 0x5565fa80f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c29d57640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa80f060 0x5565fa80f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:56446/0 (socket says 172.21.15.45:56446) 2024-01-31T05:26:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c29d57640 1 -- 172.21.15.45:0/802259120 learned_addr learned my addr 172.21.15.45:0/802259120 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c29556640 1 --2- 172.21.15.45:0/802259120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa818210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c2a558640 1 --2- 172.21.15.45:0/802259120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa818750 0x5565fa81ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c29556640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2c100a1520 0x7f2c100a39e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c14005fd0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c29d57640 1 -- 172.21.15.45:0/802259120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa818750 msgr2=0x5565fa81ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c29d57640 1 --2- 172.21.15.45:0/802259120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa818750 0x5565fa81ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c29d57640 1 -- 172.21.15.45:0/802259120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 msgr2=0x5565fa818210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c29d57640 1 --2- 172.21.15.45:0/802259120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa818210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c29d57640 1 -- 172.21.15.45:0/802259120 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565fa76ee40 con 0x5565fa80f060 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c29d57640 1 --2- 172.21.15.45:0/802259120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa80f060 0x5565fa80f440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2c20095d00 tx=0x7f2c20098bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b5400a482b4e62 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.234+0000 7f2c0ffff640 1 -- 172.21.15.45:0/802259120 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20093ea0 con 0x5565fa80f060 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.235+0000 7f2c0ffff640 1 -- 172.21.15.45:0/802259120 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c2009d120 con 0x5565fa80f060 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.235+0000 7f2c0ffff640 1 -- 172.21.15.45:0/802259120 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200a10e0 con 0x5565fa80f060 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.235+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/802259120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa80f060 msgr2=0x5565fa80f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.235+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/802259120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa80f060 0x5565fa80f440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2c20095d00 tx=0x7f2c20098bf0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.235+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/802259120 shutdown_connections 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.235+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/802259120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa818210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.235+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/802259120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa818750 0x5565fa81ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.235+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/802259120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa80f060 0x5565fa80f440 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.235+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/802259120 >> 172.21.15.45:0/802259120 conn(0x5565fa809980 msgr2=0x5565fa809d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.235+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/802259120 shutdown_connections 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.235+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/802259120 wait complete. 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.236+0000 7f2c2ed7c740 1 -- start start 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.236+0000 7f2c2ed7c740 1 Processor -- start 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.236+0000 7f2c2ed7c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa80f060 0x5565fa8156f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.236+0000 7f2c2ed7c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa815c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.236+0000 7f2c2ed7c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa818750 0x5565fa816170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.236+0000 7f2c2ed7c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fa7301b0 con 0x5565fa80f980 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.236+0000 7f2c2ed7c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fa72fe50 con 0x5565fa80f060 2024-01-31T05:26:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.236+0000 7f2c2ed7c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fa6f5090 con 0x5565fa818750 2024-01-31T05:26:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.236+0000 7f2c2ed7c740 10 client.0 _ll_register_callbacks cb 0x5565fa71aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:26:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.236+0000 7f2c29556640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa815c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.236+0000 7f2c29556640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa815c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33604/0 (socket says 172.21.15.45:33604) 2024-01-31T05:26:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.236+0000 7f2c29556640 1 -- 172.21.15.45:0/1905850065 learned_addr learned my addr 172.21.15.45:0/1905850065 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c29d57640 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa80f060 0x5565fa8156f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c2a558640 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa818750 0x5565fa816170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c29556640 1 -- 172.21.15.45:0/1905850065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa818750 msgr2=0x5565fa816170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c29556640 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa818750 0x5565fa816170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c29556640 1 -- 172.21.15.45:0/1905850065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa80f060 msgr2=0x5565fa8156f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c29556640 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa80f060 0x5565fa8156f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c29556640 1 -- 172.21.15.45:0/1905850065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c241a6420 con 0x5565fa80f980 2024-01-31T05:26:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c29556640 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa815c30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009900 tx=0x7f2c14009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30c995bd4217f82 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:26:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c0d7fa640 1 -- 172.21.15.45:0/1905850065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1402cb10 con 0x5565fa80f980 2024-01-31T05:26:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c0d7fa640 1 -- 172.21.15.45:0/1905850065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2c14005000 con 0x5565fa80f980 2024-01-31T05:26:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/1905850065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565fa770010 con 0x5565fa80f980 2024-01-31T05:26:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c0d7fa640 1 -- 172.21.15.45:0/1905850065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c140051c0 con 0x5565fa80f980 2024-01-31T05:26:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c2ed7c740 10 client.15699 fetch_fsmap learned FSMap version 25 2024-01-31T05:26:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/1905850065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5565fa7756a0 con 0x5565fa80f980 2024-01-31T05:26:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.237+0000 7f2c0d7fa640 1 -- 172.21.15.45:0/1905850065 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565fa770010 con 0x5565fa80f980 2024-01-31T05:26:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.238+0000 7f2c0d7fa640 1 -- 172.21.15.45:0/1905850065 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2c1404d470 con 0x5565fa80f980 2024-01-31T05:26:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.238+0000 7f2c2ed7c740 10 client.15699 fetch_fsmap finished waiting for FSMap version 25 2024-01-31T05:26:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.238+0000 7f2c2ed7c740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.238+0000 7f2c2ed7c740 20 client.15699 populate_metadata read hostname 'smithi045' 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.238+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5565fa828510 0x5565fa82a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.238+0000 7f2c2ed7c740 4 client.15699 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.238+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/1905850065 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565fa76fd30 con 0x5565fa828510 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.238+0000 7f2c29d57640 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5565fa828510 0x5565fa82a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.238+0000 7f2c29d57640 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5565fa828510 0x5565fa82a930 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9046e3e47b1e2d16 server_cookie=9ff2a055e430ca78 in_seq=0 out_seq=0 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.238+0000 7f2c0d7fa640 10 client.15699 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c0d7fa640 1 -- 172.21.15.45:0/1905850065 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565fa76fd30 con 0x5565fa828510 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c0d7fa640 10 client.15699 handle_command_reply: tid=0 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c2ed7c740 1 client.15699 shutdown 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/1905850065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 msgr2=0x5565fa815c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa815c30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009900 tx=0x7f2c14009260 comp rx=0 tx=0).stop 2024-01-31T05:26:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/1905850065 shutdown_connections 2024-01-31T05:26:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5565fa828510 0x5565fa82a930 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa815c30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa818750 0x5565fa816170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/1905850065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa80f060 0x5565fa8156f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c0d7fa640 0 client.15699 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/1905850065 >> 172.21.15.45:0/1905850065 conn(0x5565fa809980 msgr2=0x5565fa80b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/1905850065 shutdown_connections 2024-01-31T05:26:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/1905850065 wait complete. 2024-01-31T05:26:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.239+0000 7f2c2ed7c740 20 client.15699 trim_cache size 0 max 16384 2024-01-31T05:26:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.255+0000 7f2c2ed7c740 1 Processor -- start 2024-01-31T05:26:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.255+0000 7f2c2ed7c740 1 -- start start 2024-01-31T05:26:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.255+0000 7f2c2ed7c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa80f060 0x5565fa815f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.255+0000 7f2c2ed7c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa80b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.255+0000 7f2c2ed7c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa818750 0x5565fa81f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.255+0000 7f2c2ed7c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fa76fd30 con 0x5565fa80f980 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.255+0000 7f2c2ed7c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fa770010 con 0x5565fa80f060 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.255+0000 7f2c2ed7c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fa80a980 con 0x5565fa818750 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.255+0000 7f2c29556640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa80b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.255+0000 7f2c29556640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa80b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33614/0 (socket says 172.21.15.45:33614) 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.255+0000 7f2c29556640 1 -- 172.21.15.45:0/4050407756 learned_addr learned my addr 172.21.15.45:0/4050407756 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.256+0000 7f2c29d57640 1 --2- 172.21.15.45:0/4050407756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa80f060 0x5565fa815f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.256+0000 7f2c2a558640 1 --2- 172.21.15.45:0/4050407756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa818750 0x5565fa81f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.256+0000 7f2c29556640 1 -- 172.21.15.45:0/4050407756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa818750 msgr2=0x5565fa81f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.256+0000 7f2c29556640 1 --2- 172.21.15.45:0/4050407756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa818750 0x5565fa81f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.256+0000 7f2c29556640 1 -- 172.21.15.45:0/4050407756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa80f060 msgr2=0x5565fa815f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.256+0000 7f2c29556640 1 --2- 172.21.15.45:0/4050407756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa80f060 0x5565fa815f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.256+0000 7f2c29556640 1 -- 172.21.15.45:0/4050407756 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565fa712030 con 0x5565fa80f980 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.256+0000 7f2c29556640 1 --2- 172.21.15.45:0/4050407756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa80b4e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009850 tx=0x7f2c14004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa08d0c5a989544 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.256+0000 7f2c0f7fe640 1 -- 172.21.15.45:0/4050407756 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1402cb10 con 0x5565fa80f980 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.256+0000 7f2c0f7fe640 1 -- 172.21.15.45:0/4050407756 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c1404d470 con 0x5565fa80f980 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.256+0000 7f2c0f7fe640 1 -- 172.21.15.45:0/4050407756 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14005600 con 0x5565fa80f980 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.257+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/4050407756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 msgr2=0x5565fa80b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.257+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/4050407756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa80b4e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009850 tx=0x7f2c14004760 comp rx=0 tx=0).stop 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.257+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/4050407756 shutdown_connections 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.257+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/4050407756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa80b4e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.257+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/4050407756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa818750 0x5565fa81f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.257+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/4050407756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa80f060 0x5565fa815f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.257+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/4050407756 >> 172.21.15.45:0/4050407756 conn(0x5565fa809980 msgr2=0x5565fa80add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.257+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/4050407756 shutdown_connections 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.257+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/4050407756 wait complete. 2024-01-31T05:26:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.257+0000 7f2c2ed7c740 1 -- start start 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.257+0000 7f2c2ed7c740 1 Processor -- start 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c2ed7c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa80f060 0x5565fa81e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c2ed7c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa81ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c2ed7c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa818750 0x5565fa878a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c2ed7c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fa878f70 con 0x5565fa80f980 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c2ed7c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fa8790e0 con 0x5565fa818750 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c2ed7c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fa8792a0 con 0x5565fa80f060 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c2ed7c740 10 client.0 _ll_register_callbacks cb 0x7f2c2000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c29556640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa81ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c29556640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa81ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33618/0 (socket says 172.21.15.45:33618) 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c29556640 1 -- 172.21.15.45:0/390805493 learned_addr learned my addr 172.21.15.45:0/390805493 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c29d57640 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa80f060 0x5565fa81e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c2a558640 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa818750 0x5565fa878a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c29556640 1 -- 172.21.15.45:0/390805493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa80f060 msgr2=0x5565fa81e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c29556640 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa80f060 0x5565fa81e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c29556640 1 -- 172.21.15.45:0/390805493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa818750 msgr2=0x5565fa878a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c29556640 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa818750 0x5565fa878a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c29556640 1 -- 172.21.15.45:0/390805493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565fa7756a0 con 0x5565fa80f980 2024-01-31T05:26:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c29556640 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa81ebb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2c1402c170 tx=0x7f2c14005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c93b2805735938 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:26:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c0d7fa640 1 -- 172.21.15.45:0/390805493 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14005000 con 0x5565fa80f980 2024-01-31T05:26:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.258+0000 7f2c0d7fa640 1 -- 172.21.15.45:0/390805493 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2c14005160 con 0x5565fa80f980 2024-01-31T05:26:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/390805493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c2009d120 con 0x5565fa80f980 2024-01-31T05:26:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c0d7fa640 1 -- 172.21.15.45:0/390805493 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c14005320 con 0x5565fa80f980 2024-01-31T05:26:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c2ed7c740 10 client.15705 fetch_fsmap learned FSMap version 25 2024-01-31T05:26:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/390805493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5565fa737a50 con 0x5565fa80f980 2024-01-31T05:26:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c0d7fa640 1 -- 172.21.15.45:0/390805493 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c2009d120 con 0x5565fa80f980 2024-01-31T05:26:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c0d7fa640 1 -- 172.21.15.45:0/390805493 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2c14034cd0 con 0x5565fa80f980 2024-01-31T05:26:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c2ed7c740 10 client.15705 fetch_fsmap finished waiting for FSMap version 25 2024-01-31T05:26:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c2ed7c740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:26:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c2ed7c740 20 client.15705 populate_metadata read hostname 'smithi045' 2024-01-31T05:26:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5565fa880670 0x5565fa882a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c2ed7c740 4 client.15705 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:26:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/390805493 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5565fa476600 con 0x5565fa880670 2024-01-31T05:26:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.259+0000 7f2c29d57640 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5565fa880670 0x5565fa882a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.260+0000 7f2c0d7fa640 10 client.15705 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.260+0000 7f2c29d57640 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5565fa880670 0x5565fa882a90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f69004c4f8eb19 server_cookie=13c0aff94fc2f46 in_seq=0 out_seq=0 2024-01-31T05:26:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c0d7fa640 1 -- 172.21.15.45:0/390805493 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5565fa476600 con 0x5565fa880670 2024-01-31T05:26:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c0d7fa640 10 client.15705 handle_command_reply: tid=0 2024-01-31T05:26:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c2ed7c740 1 client.15705 shutdown 2024-01-31T05:26:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/390805493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 msgr2=0x5565fa81ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa81ebb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2c1402c170 tx=0x7f2c14005940 comp rx=0 tx=0).stop 2024-01-31T05:26:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/390805493 shutdown_connections 2024-01-31T05:26:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5565fa880670 0x5565fa882a90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565fa80f980 0x5565fa81ebb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565fa818750 0x5565fa878a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c0d7fa640 0 client.15705 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c2ed7c740 1 --2- 172.21.15.45:0/390805493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5565fa80f060 0x5565fa81e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/390805493 >> 172.21.15.45:0/390805493 conn(0x5565fa809980 msgr2=0x5565fa80b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/390805493 shutdown_connections 2024-01-31T05:26:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c2ed7c740 1 -- 172.21.15.45:0/390805493 wait complete. 2024-01-31T05:26:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.268+0000 7f2c2ed7c740 20 client.15705 trim_cache size 0 max 16384 2024-01-31T05:26:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:26:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:26:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:26:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:26:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.269+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3438272715 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2c100a1520 msgr2=0x7f2c100a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.269+0000 7f2c2bfbb640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2c100a1520 0x7f2c100a39e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c14005fd0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.269+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3438272715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c24100f30 msgr2=0x7f2c24072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.269+0000 7f2c2bfbb640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c24100f30 0x7f2c24072ff0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000d950 tx=0x7f2c2000de10 comp rx=0 tx=0).stop 2024-01-31T05:26:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.269+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3438272715 shutdown_connections 2024-01-31T05:26:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.269+0000 7f2c2bfbb640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2c100a1520 0x7f2c100a39e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.269+0000 7f2c2bfbb640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c24073a70 0x7f2c241a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.269+0000 7f2c2bfbb640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24106000 0x7f2c24073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.269+0000 7f2c2bfbb640 1 --2- 172.21.15.45:0/3438272715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c24100f30 0x7f2c24072ff0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.269+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3438272715 >> 172.21.15.45:0/3438272715 conn(0x7f2c240fc7d0 msgr2=0x7f2c240feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.269+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3438272715 shutdown_connections 2024-01-31T05:26:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:31.269+0000 7f2c2bfbb640 1 -- 172.21.15.45:0/3438272715 wait complete. 2024-01-31T05:26:31.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 340846 12.88 MB/sec execute 31 sec latency 10.520 ms 2024-01-31T05:26:31.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:31 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:26:31.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:31 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:26:31.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:31 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:26:32.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 343506 12.88 MB/sec execute 32 sec latency 9.077 ms 2024-01-31T05:26:32.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:32 smithi186 ceph-mon[35659]: pgmap v119: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-01-31T05:26:32.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:32 smithi045 ceph-mon[31585]: pgmap v119: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-01-31T05:26:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:32 smithi156 ceph-mon[35720]: pgmap v119: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-01-31T05:26:33.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 345555 12.84 MB/sec execute 33 sec latency 13.320 ms 2024-01-31T05:26:33.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:33 smithi186 ceph-mon[35659]: pgmap v120: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.8 MiB/s wr, 338 op/s 2024-01-31T05:26:33.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:33 smithi045 ceph-mon[31585]: pgmap v120: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.8 MiB/s wr, 338 op/s 2024-01-31T05:26:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:33 smithi156 ceph-mon[35720]: pgmap v120: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.8 MiB/s wr, 338 op/s 2024-01-31T05:26:34.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 347985 12.87 MB/sec execute 34 sec latency 87.807 ms 2024-01-31T05:26:35.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 350065 12.71 MB/sec execute 35 sec latency 93.810 ms 2024-01-31T05:26:36.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 352136 12.71 MB/sec execute 36 sec latency 8.453 ms 2024-01-31T05:26:36.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:36 smithi186 ceph-mon[35659]: pgmap v121: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-01-31T05:26:36.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:36 smithi045 ceph-mon[31585]: pgmap v121: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-01-31T05:26:36.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:36 smithi156 ceph-mon[35720]: pgmap v121: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-01-31T05:26:37.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 355028 12.86 MB/sec execute 37 sec latency 14.053 ms 2024-01-31T05:26:38.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 357302 12.74 MB/sec execute 38 sec latency 51.079 ms 2024-01-31T05:26:38.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:38 smithi186 ceph-mon[35659]: pgmap v122: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.6 MiB/s wr, 333 op/s 2024-01-31T05:26:38.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:38 smithi045 ceph-mon[31585]: pgmap v122: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.6 MiB/s wr, 333 op/s 2024-01-31T05:26:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:38 smithi156 ceph-mon[35720]: pgmap v122: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.6 MiB/s wr, 333 op/s 2024-01-31T05:26:39.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 359229 12.72 MB/sec execute 39 sec latency 60.050 ms 2024-01-31T05:26:40.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 361933 12.82 MB/sec execute 40 sec latency 66.794 ms 2024-01-31T05:26:40.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:40 smithi186 ceph-mon[35659]: pgmap v123: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 223 op/s 2024-01-31T05:26:40.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:40 smithi045 ceph-mon[31585]: pgmap v123: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 223 op/s 2024-01-31T05:26:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:40 smithi156 ceph-mon[35720]: pgmap v123: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 223 op/s 2024-01-31T05:26:41.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 364246 12.72 MB/sec execute 41 sec latency 9.868 ms 2024-01-31T05:26:42.778 INFO:tasks.workunit.client.0.smithi045.stdout: 1 366448 12.75 MB/sec execute 42 sec latency 8.838 ms 2024-01-31T05:26:42.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:42 smithi186 ceph-mon[35659]: pgmap v124: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 223 op/s 2024-01-31T05:26:42.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:42 smithi045 ceph-mon[31585]: pgmap v124: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 223 op/s 2024-01-31T05:26:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:42 smithi156 ceph-mon[35720]: pgmap v124: 97 pgs: 97 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 223 op/s 2024-01-31T05:26:43.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 369278 12.85 MB/sec execute 43 sec latency 15.275 ms 2024-01-31T05:26:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.359+0000 7f7f5165e640 1 -- 172.21.15.45:0/1544596252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f4c100f40 msgr2=0x7f7f4c101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.359+0000 7f7f5165e640 1 --2- 172.21.15.45:0/1544596252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f4c100f40 0x7f7f4c101320 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009930 tx=0x7f7f3c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:26:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.359+0000 7f7f5165e640 1 -- 172.21.15.45:0/1544596252 shutdown_connections 2024-01-31T05:26:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.359+0000 7f7f5165e640 1 --2- 172.21.15.45:0/1544596252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f4c106010 0x7f7f4c10dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.359+0000 7f7f5165e640 1 --2- 172.21.15.45:0/1544596252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f4c1018f0 0x7f7f4c105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.359+0000 7f7f5165e640 1 --2- 172.21.15.45:0/1544596252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f4c100f40 0x7f7f4c101320 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.359+0000 7f7f5165e640 1 -- 172.21.15.45:0/1544596252 >> 172.21.15.45:0/1544596252 conn(0x7f7f4c0fc7a0 msgr2=0x7f7f4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.359+0000 7f7f5165e640 1 -- 172.21.15.45:0/1544596252 shutdown_connections 2024-01-31T05:26:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.359+0000 7f7f5165e640 1 -- 172.21.15.45:0/1544596252 wait complete. 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f5165e640 1 Processor -- start 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f5165e640 1 -- start start 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f5165e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f4c100f40 0x7f7f4c072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f5165e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f4c106010 0x7f7f4c073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f5165e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f4c073a70 0x7f7f4c1a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f5165e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1a5990 con 0x7f7f4c100f40 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f5165e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1a5b00 con 0x7f7f4c073a70 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f5165e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1a5cc0 con 0x7f7f4c106010 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f4affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f4c100f40 0x7f7f4c072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f4affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f4c100f40 0x7f7f4c072ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39944/0 (socket says 172.21.15.45:39944) 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f4affd640 1 -- 172.21.15.45:0/1366850652 learned_addr learned my addr 172.21.15.45:0/1366850652 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f43fff640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f4c106010 0x7f7f4c073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f4b7fe640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f4c073a70 0x7f7f4c1a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f4affd640 1 -- 172.21.15.45:0/1366850652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f4c106010 msgr2=0x7f7f4c073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f4affd640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f4c106010 0x7f7f4c073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f4affd640 1 -- 172.21.15.45:0/1366850652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f4c073a70 msgr2=0x7f7f4c1a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f4affd640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f4c073a70 0x7f7f4c1a53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.360+0000 7f7f4affd640 1 -- 172.21.15.45:0/1366850652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c009560 con 0x7f7f4c100f40 2024-01-31T05:26:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.361+0000 7f7f4affd640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f4c100f40 0x7f7f4c072ff0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c005940 tx=0x7f7f3c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.361+0000 7f7f48ff9640 1 -- 172.21.15.45:0/1366850652 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c004e50 con 0x7f7f4c100f40 2024-01-31T05:26:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.361+0000 7f7f48ff9640 1 -- 172.21.15.45:0/1366850652 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f3c0054c0 con 0x7f7f4c100f40 2024-01-31T05:26:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.361+0000 7f7f5165e640 1 -- 172.21.15.45:0/1366850652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c1a5f40 con 0x7f7f4c100f40 2024-01-31T05:26:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.361+0000 7f7f48ff9640 1 -- 172.21.15.45:0/1366850652 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c003b20 con 0x7f7f4c100f40 2024-01-31T05:26:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.361+0000 7f7f5165e640 1 -- 172.21.15.45:0/1366850652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4c1a6410 con 0x7f7f4c100f40 2024-01-31T05:26:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.363+0000 7f7f48ff9640 1 -- 172.21.15.45:0/1366850652 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f7f3c003c80 con 0x7f7f4c100f40 2024-01-31T05:26:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.363+0000 7f7f54428740 1 Processor -- start 2024-01-31T05:26:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.363+0000 7f7f54428740 1 -- start start 2024-01-31T05:26:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f54428740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659060 0x56541c659440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f54428740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659980 0x56541c662210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f54428740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 0x56541c664b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f54428740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541c5c8040 con 0x56541c662750 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f54428740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541c5c5730 con 0x56541c659060 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f54428740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541c5c1020 con 0x56541c659980 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f48ff9640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7f380a15a0 0x7f7f380a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 0x56541c664b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f4affd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659980 0x56541c662210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 0x56541c664b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39952/0 (socket says 172.21.15.45:39952) 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f43fff640 1 -- 172.21.15.45:0/511813882 learned_addr learned my addr 172.21.15.45:0/511813882 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f4b7fe640 1 --2- 172.21.15.45:0/511813882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659060 0x56541c659440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f4b7fe640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7f380a15a0 0x7f7f380a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f48ff9640 1 -- 172.21.15.45:0/1366850652 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7f3c0ef500 con 0x7f7f4c100f40 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f43fff640 1 -- 172.21.15.45:0/511813882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659980 msgr2=0x56541c662210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f43fff640 1 --2- 172.21.15.45:0/511813882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659980 0x56541c662210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f43fff640 1 -- 172.21.15.45:0/511813882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659060 msgr2=0x56541c659440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f43fff640 1 --2- 172.21.15.45:0/511813882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659060 0x56541c659440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f43fff640 1 -- 172.21.15.45:0/511813882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56541c5b8e40 con 0x56541c662750 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.364+0000 7f7f43fff640 1 --2- 172.21.15.45:0/511813882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 0x56541c664b40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009c60 tx=0x7f7f34011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f9975894995461 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.365+0000 7f7f4b7fe640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7f380a15a0 0x7f7f380a3a60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005fd0 tx=0x7f7f44009500 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.365+0000 7f7f417fa640 1 -- 172.21.15.45:0/511813882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f34004230 con 0x56541c662750 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.365+0000 7f7f417fa640 1 -- 172.21.15.45:0/511813882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f34004390 con 0x56541c662750 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.365+0000 7f7f417fa640 1 -- 172.21.15.45:0/511813882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f34004d20 con 0x56541c662750 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.365+0000 7f7f54428740 1 -- 172.21.15.45:0/511813882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 msgr2=0x56541c664b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.365+0000 7f7f54428740 1 --2- 172.21.15.45:0/511813882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 0x56541c664b40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009c60 tx=0x7f7f34011810 comp rx=0 tx=0).stop 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 -- 172.21.15.45:0/511813882 shutdown_connections 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 --2- 172.21.15.45:0/511813882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 0x56541c664b40 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 --2- 172.21.15.45:0/511813882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659980 0x56541c662210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 --2- 172.21.15.45:0/511813882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659060 0x56541c659440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 -- 172.21.15.45:0/511813882 >> 172.21.15.45:0/511813882 conn(0x56541c653980 msgr2=0x56541c653d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 -- 172.21.15.45:0/511813882 shutdown_connections 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 -- 172.21.15.45:0/511813882 wait complete. 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 -- start start 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 Processor -- start 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659060 0x56541c65f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659980 0x56541c65fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c662750 0x56541c660070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541c57a1b0 con 0x56541c659060 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541c579e50 con 0x56541c659980 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541c53f090 con 0x56541c662750 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f54428740 10 client.0 _ll_register_callbacks cb 0x56541c564a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f4affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659060 0x56541c65f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f4affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659060 0x56541c65f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39956/0 (socket says 172.21.15.45:39956) 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f4affd640 1 -- 172.21.15.45:0/3605343266 learned_addr learned my addr 172.21.15.45:0/3605343266 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f4b7fe640 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c662750 0x56541c660070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.366+0000 7f7f43fff640 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659980 0x56541c65fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f4affd640 1 -- 172.21.15.45:0/3605343266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c662750 msgr2=0x56541c660070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f4affd640 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c662750 0x56541c660070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f4affd640 1 -- 172.21.15.45:0/3605343266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659980 msgr2=0x56541c65fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f4affd640 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659980 0x56541c65fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f4affd640 1 -- 172.21.15.45:0/3605343266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f34009890 con 0x56541c659060 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f4affd640 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659060 0x56541c65f5f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c000c00 tx=0x7f7f3c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ae1a8653e6e294 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f327fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f327fc640 1 -- 172.21.15.45:0/3605343266 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c0c0550 con 0x56541c659060 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f327fc640 1 -- 172.21.15.45:0/3605343266 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7f3c0c4790 con 0x56541c659060 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f54428740 1 -- 172.21.15.45:0/3605343266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56541c5ba010 con 0x56541c659060 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f327fc640 1 -- 172.21.15.45:0/3605343266 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f3c0c0180 con 0x56541c659060 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f54428740 10 client.15717 fetch_fsmap learned FSMap version 25 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f54428740 1 -- 172.21.15.45:0/3605343266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56541c5bf6a0 con 0x56541c659060 2024-01-31T05:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f327fc640 1 -- 172.21.15.45:0/3605343266 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56541c5ba010 con 0x56541c659060 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f327fc640 1 -- 172.21.15.45:0/3605343266 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7f3c03c070 con 0x56541c659060 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f54428740 10 client.15717 fetch_fsmap finished waiting for FSMap version 25 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f54428740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f54428740 20 client.15717 populate_metadata read hostname 'smithi045' 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f54428740 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56541c672320 0x56541c674710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f54428740 4 client.15717 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.367+0000 7f7f54428740 1 -- 172.21.15.45:0/3605343266 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56541c5b9d30 con 0x56541c672320 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.368+0000 7f7f43fff640 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56541c672320 0x56541c674710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.368+0000 7f7f327fc640 10 client.15717 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.368+0000 7f7f43fff640 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56541c672320 0x56541c674710 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eaa7ef99c4409f8 server_cookie=2802f698150bbc47 in_seq=0 out_seq=0 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f327fc640 1 -- 172.21.15.45:0/3605343266 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56541c5b9d30 con 0x56541c672320 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f327fc640 10 client.15717 handle_command_reply: tid=0 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f54428740 1 client.15717 shutdown 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f54428740 1 -- 172.21.15.45:0/3605343266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659060 msgr2=0x56541c65f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f54428740 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659060 0x56541c65f5f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c000c00 tx=0x7f7f3c0035f0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f54428740 1 -- 172.21.15.45:0/3605343266 shutdown_connections 2024-01-31T05:26:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f54428740 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56541c672320 0x56541c674710 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f54428740 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c662750 0x56541c660070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f54428740 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659980 0x56541c65fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f327fc640 0 client.15717 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f54428740 1 --2- 172.21.15.45:0/3605343266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659060 0x56541c65f5f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f54428740 1 -- 172.21.15.45:0/3605343266 >> 172.21.15.45:0/3605343266 conn(0x56541c653980 msgr2=0x56541c6556d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f54428740 1 -- 172.21.15.45:0/3605343266 shutdown_connections 2024-01-31T05:26:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f54428740 1 -- 172.21.15.45:0/3605343266 wait complete. 2024-01-31T05:26:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.369+0000 7f7f54428740 20 client.15717 trim_cache size 0 max 16384 2024-01-31T05:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.383+0000 7f7f54428740 1 Processor -- start 2024-01-31T05:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.383+0000 7f7f54428740 1 -- start start 2024-01-31T05:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f54428740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659060 0x56541c661d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f54428740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659980 0x56541c65e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f54428740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 0x56541c65fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f54428740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541c5b9d30 con 0x56541c662750 2024-01-31T05:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f54428740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541c5ba010 con 0x56541c659980 2024-01-31T05:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f54428740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541c654980 con 0x56541c659060 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 0x56541c65fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 0x56541c65fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39964/0 (socket says 172.21.15.45:39964) 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f4b7fe640 1 -- 172.21.15.45:0/1804715483 learned_addr learned my addr 172.21.15.45:0/1804715483 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f4affd640 1 --2- 172.21.15.45:0/1804715483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659060 0x56541c661d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f43fff640 1 --2- 172.21.15.45:0/1804715483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659980 0x56541c65e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f4b7fe640 1 -- 172.21.15.45:0/1804715483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659060 msgr2=0x56541c661d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f4b7fe640 1 --2- 172.21.15.45:0/1804715483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659060 0x56541c661d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f4b7fe640 1 -- 172.21.15.45:0/1804715483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659980 msgr2=0x56541c65e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f4b7fe640 1 --2- 172.21.15.45:0/1804715483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659980 0x56541c65e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.384+0000 7f7f4b7fe640 1 -- 172.21.15.45:0/1804715483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56541c55c030 con 0x56541c662750 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.385+0000 7f7f4b7fe640 1 --2- 172.21.15.45:0/1804715483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 0x56541c65fe90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7f44014330 tx=0x7f7f440098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1078689ce056e6ee server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.385+0000 7f7f40ff9640 1 -- 172.21.15.45:0/1804715483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f44011e90 con 0x56541c662750 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.385+0000 7f7f40ff9640 1 -- 172.21.15.45:0/1804715483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f4401a070 con 0x56541c662750 2024-01-31T05:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.385+0000 7f7f40ff9640 1 -- 172.21.15.45:0/1804715483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4400f6a0 con 0x56541c662750 2024-01-31T05:26:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.385+0000 7f7f54428740 1 -- 172.21.15.45:0/1804715483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 msgr2=0x56541c65fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.385+0000 7f7f54428740 1 --2- 172.21.15.45:0/1804715483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 0x56541c65fe90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7f44014330 tx=0x7f7f440098e0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 -- 172.21.15.45:0/1804715483 shutdown_connections 2024-01-31T05:26:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 --2- 172.21.15.45:0/1804715483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c662750 0x56541c65fe90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 --2- 172.21.15.45:0/1804715483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c659980 0x56541c65e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 --2- 172.21.15.45:0/1804715483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659060 0x56541c661d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 -- 172.21.15.45:0/1804715483 >> 172.21.15.45:0/1804715483 conn(0x56541c653980 msgr2=0x56541c65dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 -- 172.21.15.45:0/1804715483 shutdown_connections 2024-01-31T05:26:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 -- 172.21.15.45:0/1804715483 wait complete. 2024-01-31T05:26:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 -- start start 2024-01-31T05:26:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 Processor -- start 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659060 0x56541c6698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659980 0x56541c667d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c662750 0x56541c6683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541c668aa0 con 0x56541c659980 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541c6c87d0 con 0x56541c662750 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541c6c8910 con 0x56541c659060 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f54428740 10 client.0 _ll_register_callbacks cb 0x7f7f340097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659980 0x56541c667d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659980 0x56541c667d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39968/0 (socket says 172.21.15.45:39968) 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f43fff640 1 -- 172.21.15.45:0/3469155328 learned_addr learned my addr 172.21.15.45:0/3469155328 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.386+0000 7f7f4affd640 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659060 0x56541c6698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f4b7fe640 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c662750 0x56541c6683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f43fff640 1 -- 172.21.15.45:0/3469155328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659060 msgr2=0x56541c6698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f43fff640 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659060 0x56541c6698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f43fff640 1 -- 172.21.15.45:0/3469155328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c662750 msgr2=0x56541c6683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f43fff640 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c662750 0x56541c6683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f43fff640 1 -- 172.21.15.45:0/3469155328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f44014040 con 0x56541c659980 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f4b7fe640 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c662750 0x56541c6683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f4affd640 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659060 0x56541c6698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f43fff640 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659980 0x56541c667d20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009b80 tx=0x7f7f34027680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f82a6f237563ff server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f327fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f327fc640 1 -- 172.21.15.45:0/3469155328 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f34005cd0 con 0x56541c659980 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f327fc640 1 -- 172.21.15.45:0/3469155328 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7f34027c00 con 0x56541c659980 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f54428740 1 -- 172.21.15.45:0/3469155328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f34004390 con 0x56541c659980 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f327fc640 1 -- 172.21.15.45:0/3469155328 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f34009ed0 con 0x56541c659980 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f54428740 10 client.15723 fetch_fsmap learned FSMap version 25 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f54428740 1 -- 172.21.15.45:0/3469155328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56541c581a50 con 0x56541c659980 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.387+0000 7f7f327fc640 1 -- 172.21.15.45:0/3469155328 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f34004390 con 0x56541c659980 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.388+0000 7f7f327fc640 1 -- 172.21.15.45:0/3469155328 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7f340193f0 con 0x56541c659980 2024-01-31T05:26:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.388+0000 7f7f54428740 10 client.15723 fetch_fsmap finished waiting for FSMap version 25 2024-01-31T05:26:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.388+0000 7f7f54428740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:26:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.388+0000 7f7f54428740 20 client.15723 populate_metadata read hostname 'smithi045' 2024-01-31T05:26:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.388+0000 7f7f54428740 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56541c6ca0e0 0x56541c6cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.388+0000 7f7f54428740 4 client.15723 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:26:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.388+0000 7f7f54428740 1 -- 172.21.15.45:0/3469155328 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56541c2c0600 con 0x56541c6ca0e0 2024-01-31T05:26:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.388+0000 7f7f4affd640 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56541c6ca0e0 0x56541c6cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.388+0000 7f7f4affd640 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56541c6ca0e0 0x56541c6cc500 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be7f294ea2e82e6e server_cookie=6dcc76e4ddf34611 in_seq=0 out_seq=0 2024-01-31T05:26:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.388+0000 7f7f327fc640 10 client.15723 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.396+0000 7f7f327fc640 1 -- 172.21.15.45:0/3469155328 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56541c2c0600 con 0x56541c6ca0e0 2024-01-31T05:26:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.396+0000 7f7f327fc640 10 client.15723 handle_command_reply: tid=0 2024-01-31T05:26:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.396+0000 7f7f54428740 1 client.15723 shutdown 2024-01-31T05:26:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f54428740 1 -- 172.21.15.45:0/3469155328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659980 msgr2=0x56541c667d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f54428740 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659980 0x56541c667d20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009b80 tx=0x7f7f34027680 comp rx=0 tx=0).stop 2024-01-31T05:26:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f54428740 1 -- 172.21.15.45:0/3469155328 shutdown_connections 2024-01-31T05:26:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f54428740 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56541c6ca0e0 0x56541c6cc500 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f54428740 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56541c659980 0x56541c667d20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f54428740 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56541c662750 0x56541c6683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f327fc640 0 client.15723 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f54428740 1 --2- 172.21.15.45:0/3469155328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56541c659060 0x56541c6698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f54428740 1 -- 172.21.15.45:0/3469155328 >> 172.21.15.45:0/3469155328 conn(0x56541c653980 msgr2=0x56541c654dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f54428740 1 -- 172.21.15.45:0/3469155328 shutdown_connections 2024-01-31T05:26:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f54428740 1 -- 172.21.15.45:0/3469155328 wait complete. 2024-01-31T05:26:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f54428740 20 client.15723 trim_cache size 0 max 16384 2024-01-31T05:26:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:26:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:26:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:26:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:26:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f5165e640 1 -- 172.21.15.45:0/1366850652 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7f380a15a0 msgr2=0x7f7f380a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f5165e640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7f380a15a0 0x7f7f380a3a60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005fd0 tx=0x7f7f44009500 comp rx=0 tx=0).stop 2024-01-31T05:26:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f5165e640 1 -- 172.21.15.45:0/1366850652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f4c100f40 msgr2=0x7f7f4c072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.397+0000 7f7f5165e640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f4c100f40 0x7f7f4c072ff0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c005940 tx=0x7f7f3c005d50 comp rx=0 tx=0).stop 2024-01-31T05:26:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.398+0000 7f7f5165e640 1 -- 172.21.15.45:0/1366850652 shutdown_connections 2024-01-31T05:26:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.398+0000 7f7f5165e640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7f380a15a0 0x7f7f380a3a60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.398+0000 7f7f5165e640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f4c073a70 0x7f7f4c1a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.398+0000 7f7f5165e640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f4c106010 0x7f7f4c073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.398+0000 7f7f5165e640 1 --2- 172.21.15.45:0/1366850652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f4c100f40 0x7f7f4c072ff0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.398+0000 7f7f5165e640 1 -- 172.21.15.45:0/1366850652 >> 172.21.15.45:0/1366850652 conn(0x7f7f4c0fc7a0 msgr2=0x7f7f4c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.398+0000 7f7f5165e640 1 -- 172.21.15.45:0/1366850652 shutdown_connections 2024-01-31T05:26:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:44.398+0000 7f7f5165e640 1 -- 172.21.15.45:0/1366850652 wait complete. 2024-01-31T05:26:44.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 371806 12.84 MB/sec execute 44 sec latency 72.018 ms 2024-01-31T05:26:44.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:44 smithi186 ceph-mon[35659]: pgmap v125: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 330 op/s 2024-01-31T05:26:44.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:44 smithi045 ceph-mon[31585]: pgmap v125: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 330 op/s 2024-01-31T05:26:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:44 smithi156 ceph-mon[35720]: pgmap v125: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 330 op/s 2024-01-31T05:26:45.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 373567 12.75 MB/sec execute 45 sec latency 43.044 ms 2024-01-31T05:26:46.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 376298 12.83 MB/sec execute 46 sec latency 17.621 ms 2024-01-31T05:26:46.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:46 smithi186 ceph-mon[35659]: pgmap v126: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 212 op/s 2024-01-31T05:26:46.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:46 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:26:46.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:46 smithi045 ceph-mon[31585]: pgmap v126: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 212 op/s 2024-01-31T05:26:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:46 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:26:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:46 smithi156 ceph-mon[35720]: pgmap v126: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 212 op/s 2024-01-31T05:26:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:46 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:26:47.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 378942 12.84 MB/sec execute 47 sec latency 8.769 ms 2024-01-31T05:26:48.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 380998 12.81 MB/sec execute 48 sec latency 15.609 ms 2024-01-31T05:26:48.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:48 smithi186 ceph-mon[35659]: pgmap v127: 97 pgs: 97 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 331 op/s 2024-01-31T05:26:48.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:48 smithi045 ceph-mon[31585]: pgmap v127: 97 pgs: 97 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 331 op/s 2024-01-31T05:26:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:48 smithi156 ceph-mon[35720]: pgmap v127: 97 pgs: 97 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 331 op/s 2024-01-31T05:26:49.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 383585 12.84 MB/sec execute 49 sec latency 8.308 ms 2024-01-31T05:26:49.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:49 smithi045 ceph-mon[31585]: pgmap v128: 97 pgs: 97 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-01-31T05:26:49.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:49 smithi156 ceph-mon[35720]: pgmap v128: 97 pgs: 97 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-01-31T05:26:50.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:49 smithi186 ceph-mon[35659]: pgmap v128: 97 pgs: 97 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-01-31T05:26:50.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 386027 12.82 MB/sec execute 50 sec latency 44.770 ms 2024-01-31T05:26:50.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:50 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:26:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:50 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:26:51.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:50 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:26:51.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 387745 12.75 MB/sec execute 51 sec latency 32.021 ms 2024-01-31T05:26:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:51 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:26:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:51 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:26:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:51 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:26:51.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:51 smithi045 ceph-mon[31585]: pgmap v129: 97 pgs: 97 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-01-31T05:26:51.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:51 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:26:51.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:51 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:26:51.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:51 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:26:51.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:51 smithi156 ceph-mon[35720]: pgmap v129: 97 pgs: 97 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-01-31T05:26:52.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:51 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:26:52.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:51 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:26:52.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:51 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:26:52.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:51 smithi186 ceph-mon[35659]: pgmap v129: 97 pgs: 97 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-01-31T05:26:52.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 390604 12.83 MB/sec execute 52 sec latency 8.293 ms 2024-01-31T05:26:53.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 393406 12.84 MB/sec execute 53 sec latency 13.791 ms 2024-01-31T05:26:54.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 395350 12.81 MB/sec execute 54 sec latency 16.314 ms 2024-01-31T05:26:54.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:54 smithi186 ceph-mon[35659]: pgmap v130: 97 pgs: 97 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 334 op/s 2024-01-31T05:26:54.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:54 smithi045 ceph-mon[31585]: pgmap v130: 97 pgs: 97 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 334 op/s 2024-01-31T05:26:54.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:54 smithi156 ceph-mon[35720]: pgmap v130: 97 pgs: 97 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 334 op/s 2024-01-31T05:26:55.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 397692 12.83 MB/sec execute 55 sec latency 43.296 ms 2024-01-31T05:26:56.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 400497 12.84 MB/sec execute 56 sec latency 11.700 ms 2024-01-31T05:26:56.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:56 smithi186 ceph-mon[35659]: pgmap v131: 97 pgs: 97 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-01-31T05:26:56.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:56 smithi045 ceph-mon[31585]: pgmap v131: 97 pgs: 97 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-01-31T05:26:56.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:56 smithi156 ceph-mon[35720]: pgmap v131: 97 pgs: 97 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-01-31T05:26:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.493+0000 7f9f59fc4640 1 -- 172.21.15.45:0/3019722541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f54100f30 msgr2=0x7f9f54101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.493+0000 7f9f59fc4640 1 --2- 172.21.15.45:0/3019722541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f54100f30 0x7f9f54101310 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009930 tx=0x7f9f4402bc30 comp rx=0 tx=0).stop 2024-01-31T05:26:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.493+0000 7f9f59fc4640 1 -- 172.21.15.45:0/3019722541 shutdown_connections 2024-01-31T05:26:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.493+0000 7f9f59fc4640 1 --2- 172.21.15.45:0/3019722541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f54106000 0x7f9f5410dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.493+0000 7f9f59fc4640 1 --2- 172.21.15.45:0/3019722541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f541018e0 0x7f9f54105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.493+0000 7f9f59fc4640 1 --2- 172.21.15.45:0/3019722541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f54100f30 0x7f9f54101310 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.493+0000 7f9f59fc4640 1 -- 172.21.15.45:0/3019722541 >> 172.21.15.45:0/3019722541 conn(0x7f9f540fc7d0 msgr2=0x7f9f540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.493+0000 7f9f59fc4640 1 -- 172.21.15.45:0/3019722541 shutdown_connections 2024-01-31T05:26:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.493+0000 7f9f59fc4640 1 -- 172.21.15.45:0/3019722541 wait complete. 2024-01-31T05:26:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f59fc4640 1 Processor -- start 2024-01-31T05:26:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f59fc4640 1 -- start start 2024-01-31T05:26:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f59fc4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f54100f30 0x7f9f541971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f59fc4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54106000 0x7f9f54197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f59fc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f5419baa0 0x7f9f5419bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f59fc4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419c490 con 0x7f9f5419baa0 2024-01-31T05:26:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f59fc4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419c600 con 0x7f9f54106000 2024-01-31T05:26:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f59fc4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419c770 con 0x7f9f54100f30 2024-01-31T05:26:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f54100f30 0x7f9f541971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f5419baa0 0x7f9f5419bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f54100f30 0x7f9f541971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:59960/0 (socket says 172.21.15.45:59960) 2024-01-31T05:26:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f537fe640 1 -- 172.21.15.45:0/2095635268 learned_addr learned my addr 172.21.15.45:0/2095635268 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.494+0000 7f9f52ffd640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54106000 0x7f9f54197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.495+0000 7f9f537fe640 1 -- 172.21.15.45:0/2095635268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54106000 msgr2=0x7f9f54197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.495+0000 7f9f537fe640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54106000 0x7f9f54197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.495+0000 7f9f537fe640 1 -- 172.21.15.45:0/2095635268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f5419baa0 msgr2=0x7f9f5419bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.495+0000 7f9f537fe640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f5419baa0 0x7f9f5419bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.495+0000 7f9f537fe640 1 -- 172.21.15.45:0/2095635268 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44009560 con 0x7f9f54100f30 2024-01-31T05:26:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.495+0000 7f9f537fe640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f54100f30 0x7f9f541971e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f44005940 tx=0x7f9f44005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.495+0000 7f9f50ff9640 1 -- 172.21.15.45:0/2095635268 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44004e50 con 0x7f9f54100f30 2024-01-31T05:26:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.495+0000 7f9f50ff9640 1 -- 172.21.15.45:0/2095635268 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f440054c0 con 0x7f9f54100f30 2024-01-31T05:26:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.495+0000 7f9f59fc4640 1 -- 172.21.15.45:0/2095635268 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f541a3d50 con 0x7f9f54100f30 2024-01-31T05:26:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.495+0000 7f9f50ff9640 1 -- 172.21.15.45:0/2095635268 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44003b20 con 0x7f9f54100f30 2024-01-31T05:26:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.495+0000 7f9f59fc4640 1 -- 172.21.15.45:0/2095635268 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f541a41b0 con 0x7f9f54100f30 2024-01-31T05:26:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.497+0000 7f9f50ff9640 1 -- 172.21.15.45:0/2095635268 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f9f44003c80 con 0x7f9f54100f30 2024-01-31T05:26:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f5cd84740 1 Processor -- start 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f5cd84740 1 -- start start 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f50ff9640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f9f3c0a1570 0x7f9f3c0a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f5cd84740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 0x5632a5099440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f5cd84740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099980 0x5632a50a2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f5cd84740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a50a2750 0x5632a50a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f5cd84740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a5008040 con 0x5632a5099060 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f5cd84740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a5005730 con 0x5632a50a2750 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f5cd84740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a5001020 con 0x5632a5099980 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f53fff640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f9f3c0a1570 0x7f9f3c0a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f50ff9640 1 -- 172.21.15.45:0/2095635268 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9f440ef460 con 0x7f9f54100f30 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 0x5632a5099440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099980 0x5632a50a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 0x5632a5099440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50328/0 (socket says 172.21.15.45:50328) 2024-01-31T05:26:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f537fe640 1 -- 172.21.15.45:0/3275683494 learned_addr learned my addr 172.21.15.45:0/3275683494 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.498+0000 7f9f53fff640 1 --2- 172.21.15.45:0/3275683494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a50a2750 0x5632a50a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.499+0000 7f9f537fe640 1 -- 172.21.15.45:0/3275683494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099980 msgr2=0x5632a50a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.499+0000 7f9f537fe640 1 --2- 172.21.15.45:0/3275683494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099980 0x5632a50a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.499+0000 7f9f537fe640 1 -- 172.21.15.45:0/3275683494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a50a2750 msgr2=0x5632a50a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.499+0000 7f9f537fe640 1 --2- 172.21.15.45:0/3275683494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a50a2750 0x5632a50a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.499+0000 7f9f537fe640 1 -- 172.21.15.45:0/3275683494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632a4ff8e40 con 0x5632a5099060 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.499+0000 7f9f52ffd640 1 --2- 172.21.15.45:0/3275683494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099980 0x5632a50a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.499+0000 7f9f53fff640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f9f3c0a1570 0x7f9f3c0a3a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c005fd0 tx=0x7f9f4c011b40 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.499+0000 7f9f537fe640 1 --2- 172.21.15.45:0/3275683494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 0x5632a5099440 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9f440b8dd0 tx=0x7f9f440bbf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7f6283803e32a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.499+0000 7f9f39ffb640 1 -- 172.21.15.45:0/3275683494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4403a240 con 0x5632a5099060 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.499+0000 7f9f39ffb640 1 -- 172.21.15.45:0/3275683494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f440bca60 con 0x5632a5099060 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.499+0000 7f9f39ffb640 1 -- 172.21.15.45:0/3275683494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44048d80 con 0x5632a5099060 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.500+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3275683494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 msgr2=0x5632a5099440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.500+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/3275683494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 0x5632a5099440 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9f440b8dd0 tx=0x7f9f440bbf60 comp rx=0 tx=0).stop 2024-01-31T05:26:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.500+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3275683494 shutdown_connections 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.500+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/3275683494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a50a2750 0x5632a50a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.500+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/3275683494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099980 0x5632a50a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.500+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/3275683494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 0x5632a5099440 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.500+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3275683494 >> 172.21.15.45:0/3275683494 conn(0x5632a5093980 msgr2=0x5632a5093d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.500+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3275683494 shutdown_connections 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.500+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3275683494 wait complete. 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.500+0000 7f9f5cd84740 1 -- start start 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.500+0000 7f9f5cd84740 1 Processor -- start 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f5cd84740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 0x5632a509f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f5cd84740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a5099980 0x5632a509fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f5cd84740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a50a2750 0x5632a50a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f5cd84740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a4fba1b0 con 0x5632a5099060 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f5cd84740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a4fb9e50 con 0x5632a5099980 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f5cd84740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a4f7f090 con 0x5632a50a2750 2024-01-31T05:26:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f5cd84740 10 client.0 _ll_register_callbacks cb 0x5632a4fa4a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 0x5632a509f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 0x5632a509f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50336/0 (socket says 172.21.15.45:50336) 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a5099980 0x5632a509fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f537fe640 1 -- 172.21.15.45:0/3395523085 learned_addr learned my addr 172.21.15.45:0/3395523085 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f53fff640 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a50a2750 0x5632a50a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f52ffd640 1 -- 172.21.15.45:0/3395523085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a50a2750 msgr2=0x5632a50a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f52ffd640 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a50a2750 0x5632a50a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f52ffd640 1 -- 172.21.15.45:0/3395523085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 msgr2=0x5632a509f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f52ffd640 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 0x5632a509f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f52ffd640 1 -- 172.21.15.45:0/3395523085 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f541a41b0 con 0x5632a5099980 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f537fe640 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 0x5632a509f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f52ffd640 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a5099980 0x5632a509fc60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f5406a320 tx=0x7f9f48012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42903460887dd045 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.501+0000 7f9f32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:26:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f32ffd640 1 -- 172.21.15.45:0/3395523085 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48017070 con 0x5632a5099980 2024-01-31T05:26:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3395523085 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632a4ffa010 con 0x5632a5099980 2024-01-31T05:26:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f32ffd640 1 -- 172.21.15.45:0/3395523085 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9f48004030 con 0x5632a5099980 2024-01-31T05:26:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f32ffd640 1 -- 172.21.15.45:0/3395523085 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f480041f0 con 0x5632a5099980 2024-01-31T05:26:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f5cd84740 10 client.25256 fetch_fsmap learned FSMap version 26 2024-01-31T05:26:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3395523085 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5632a4fff6a0 con 0x5632a5099980 2024-01-31T05:26:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f32ffd640 1 -- 172.21.15.45:0/3395523085 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632a4ffa010 con 0x5632a5099980 2024-01-31T05:26:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f32ffd640 1 -- 172.21.15.45:0/3395523085 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9f48003920 con 0x5632a5099980 2024-01-31T05:26:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f5cd84740 10 client.25256 fetch_fsmap finished waiting for FSMap version 26 2024-01-31T05:26:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f5cd84740 10 client.25256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:26:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f5cd84740 20 client.25256 populate_metadata read hostname 'smithi045' 2024-01-31T05:26:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5632a50b6790 0x5632a50b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f5cd84740 4 client.25256 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:26:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3395523085 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632a4ff9d30 con 0x5632a50b6790 2024-01-31T05:26:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.502+0000 7f9f537fe640 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5632a50b6790 0x5632a50b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.503+0000 7f9f537fe640 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5632a50b6790 0x5632a50b8bb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2536b36a3f9fc41c server_cookie=94a7bbfbcdb0aedf in_seq=0 out_seq=0 2024-01-31T05:26:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.503+0000 7f9f32ffd640 10 client.25256 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.503+0000 7f9f32ffd640 1 -- 172.21.15.45:0/3395523085 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632a4ff9d30 con 0x5632a50b6790 2024-01-31T05:26:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.503+0000 7f9f32ffd640 10 client.25256 handle_command_reply: tid=0 2024-01-31T05:26:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.503+0000 7f9f5cd84740 1 client.25256 shutdown 2024-01-31T05:26:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.503+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3395523085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a5099980 msgr2=0x5632a509fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.503+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a5099980 0x5632a509fc60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f5406a320 tx=0x7f9f48012b60 comp rx=0 tx=0).stop 2024-01-31T05:26:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.504+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3395523085 shutdown_connections 2024-01-31T05:26:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.504+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5632a50b6790 0x5632a50b8bb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.504+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a50a2750 0x5632a50a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.504+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a5099980 0x5632a509fc60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.504+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/3395523085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099060 0x5632a509f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.504+0000 7f9f32ffd640 0 client.25256 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.504+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3395523085 >> 172.21.15.45:0/3395523085 conn(0x5632a5093980 msgr2=0x5632a50956d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.504+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3395523085 shutdown_connections 2024-01-31T05:26:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.504+0000 7f9f5cd84740 1 -- 172.21.15.45:0/3395523085 wait complete. 2024-01-31T05:26:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.504+0000 7f9f5cd84740 20 client.25256 trim_cache size 0 max 16384 2024-01-31T05:26:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f5cd84740 1 Processor -- start 2024-01-31T05:26:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f5cd84740 1 -- start start 2024-01-31T05:26:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f5cd84740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099060 0x5632a50a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f5cd84740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099980 0x5632a509ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f5cd84740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a50a2750 0x5632a509c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f5cd84740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a508f080 con 0x5632a5099980 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f5cd84740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a5094550 con 0x5632a50a2750 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f5cd84740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a5094980 con 0x5632a5099060 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099980 0x5632a509ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a50a2750 0x5632a509c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099060 0x5632a50a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099980 0x5632a509ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50342/0 (socket says 172.21.15.45:50342) 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f52ffd640 1 -- 172.21.15.45:0/227729855 learned_addr learned my addr 172.21.15.45:0/227729855 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f52ffd640 1 -- 172.21.15.45:0/227729855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099060 msgr2=0x5632a50a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f52ffd640 1 --2- 172.21.15.45:0/227729855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099060 0x5632a50a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.520+0000 7f9f52ffd640 1 -- 172.21.15.45:0/227729855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a50a2750 msgr2=0x5632a509c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.521+0000 7f9f52ffd640 1 --2- 172.21.15.45:0/227729855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a50a2750 0x5632a509c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.521+0000 7f9f52ffd640 1 -- 172.21.15.45:0/227729855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632a4f9c030 con 0x5632a5099980 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.521+0000 7f9f53fff640 1 --2- 172.21.15.45:0/227729855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a50a2750 0x5632a509c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.521+0000 7f9f52ffd640 1 --2- 172.21.15.45:0/227729855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099980 0x5632a509ffc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9f480098d0 tx=0x7f9f48009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d02a74f3e647814 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.521+0000 7f9f397fa640 1 -- 172.21.15.45:0/227729855 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48017070 con 0x5632a5099980 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.521+0000 7f9f397fa640 1 -- 172.21.15.45:0/227729855 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f48018040 con 0x5632a5099980 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.521+0000 7f9f397fa640 1 -- 172.21.15.45:0/227729855 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48003ea0 con 0x5632a5099980 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 -- 172.21.15.45:0/227729855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099980 msgr2=0x5632a509ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/227729855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099980 0x5632a509ffc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9f480098d0 tx=0x7f9f48009900 comp rx=0 tx=0).stop 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 -- 172.21.15.45:0/227729855 shutdown_connections 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/227729855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a5099980 0x5632a509ffc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/227729855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a50a2750 0x5632a509c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/227729855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099060 0x5632a50a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 -- 172.21.15.45:0/227729855 >> 172.21.15.45:0/227729855 conn(0x5632a5093980 msgr2=0x5632a509dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 -- 172.21.15.45:0/227729855 shutdown_connections 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 -- 172.21.15.45:0/227729855 wait complete. 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 -- start start 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 Processor -- start 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a5099060 0x5632a50a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099980 0x5632a50a80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a50a2750 0x5632a50a8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a50a9b40 con 0x5632a50a2750 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a510ccc0 con 0x5632a5099060 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a510ce30 con 0x5632a5099980 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.522+0000 7f9f5cd84740 10 client.0 _ll_register_callbacks cb 0x7f9f4403bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a50a2750 0x5632a50a8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a50a2750 0x5632a50a8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50350/0 (socket says 172.21.15.45:50350) 2024-01-31T05:26:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f53fff640 1 -- 172.21.15.45:0/1417887472 learned_addr learned my addr 172.21.15.45:0/1417887472 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f52ffd640 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099980 0x5632a50a80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f537fe640 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a5099060 0x5632a50a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f53fff640 1 -- 172.21.15.45:0/1417887472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099980 msgr2=0x5632a50a80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f53fff640 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099980 0x5632a50a80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f53fff640 1 -- 172.21.15.45:0/1417887472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a5099060 msgr2=0x5632a50a7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f53fff640 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a5099060 0x5632a50a7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f53fff640 1 -- 172.21.15.45:0/1417887472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4c00fa80 con 0x5632a50a2750 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f537fe640 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a5099060 0x5632a50a7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f53fff640 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a50a2750 0x5632a50a8620 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00f790 tx=0x7f9f4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca75a65dba274fb0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f32ffd640 1 -- 172.21.15.45:0/1417887472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c026070 con 0x5632a50a2750 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f5cd84740 1 -- 172.21.15.45:0/1417887472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f440bca60 con 0x5632a50a2750 2024-01-31T05:26:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f32ffd640 1 -- 172.21.15.45:0/1417887472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9f4c004b20 con 0x5632a50a2750 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f32ffd640 1 -- 172.21.15.45:0/1417887472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4c004d20 con 0x5632a50a2750 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f5cd84740 10 client.15741 fetch_fsmap learned FSMap version 26 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.523+0000 7f9f5cd84740 1 -- 172.21.15.45:0/1417887472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5632a4fff6a0 con 0x5632a50a2750 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.524+0000 7f9f32ffd640 1 -- 172.21.15.45:0/1417887472 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f440bca60 con 0x5632a50a2750 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.524+0000 7f9f32ffd640 1 -- 172.21.15.45:0/1417887472 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9f4c0053c0 con 0x5632a50a2750 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.524+0000 7f9f5cd84740 10 client.15741 fetch_fsmap finished waiting for FSMap version 26 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.524+0000 7f9f5cd84740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.524+0000 7f9f5cd84740 20 client.15741 populate_metadata read hostname 'smithi045' 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.524+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5632a510e5e0 0x5632a5110a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.524+0000 7f9f5cd84740 4 client.15741 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.524+0000 7f9f5cd84740 1 -- 172.21.15.45:0/1417887472 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5632a4d00600 con 0x5632a510e5e0 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.524+0000 7f9f537fe640 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5632a510e5e0 0x5632a5110a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:26:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.524+0000 7f9f32ffd640 10 client.15741 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.524+0000 7f9f537fe640 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5632a510e5e0 0x5632a5110a00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582d3f002452d8b1 server_cookie=9875a31b447aa061 in_seq=0 out_seq=0 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f32ffd640 1 -- 172.21.15.45:0/1417887472 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5632a4d00600 con 0x5632a510e5e0 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f32ffd640 10 client.15741 handle_command_reply: tid=0 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f5cd84740 1 client.15741 shutdown 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f5cd84740 1 -- 172.21.15.45:0/1417887472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a50a2750 msgr2=0x5632a50a8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a50a2750 0x5632a50a8620 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00f790 tx=0x7f9f4c004ab0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f5cd84740 1 -- 172.21.15.45:0/1417887472 shutdown_connections 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5632a510e5e0 0x5632a5110a00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5632a50a2750 0x5632a50a8620 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632a5099980 0x5632a50a80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f32ffd640 0 client.15741 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f5cd84740 1 --2- 172.21.15.45:0/1417887472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632a5099060 0x5632a50a7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f5cd84740 1 -- 172.21.15.45:0/1417887472 >> 172.21.15.45:0/1417887472 conn(0x5632a5093980 msgr2=0x5632a509e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f5cd84740 1 -- 172.21.15.45:0/1417887472 shutdown_connections 2024-01-31T05:26:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f5cd84740 1 -- 172.21.15.45:0/1417887472 wait complete. 2024-01-31T05:26:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.533+0000 7f9f5cd84740 20 client.15741 trim_cache size 0 max 16384 2024-01-31T05:26:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:26:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:26:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:26:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:26:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.534+0000 7f9f59fc4640 1 -- 172.21.15.45:0/2095635268 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f9f3c0a1570 msgr2=0x7f9f3c0a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.534+0000 7f9f59fc4640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f9f3c0a1570 0x7f9f3c0a3a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c005fd0 tx=0x7f9f4c011b40 comp rx=0 tx=0).stop 2024-01-31T05:26:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.534+0000 7f9f59fc4640 1 -- 172.21.15.45:0/2095635268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f54100f30 msgr2=0x7f9f541971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:26:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.534+0000 7f9f59fc4640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f54100f30 0x7f9f541971e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f44005940 tx=0x7f9f44005d50 comp rx=0 tx=0).stop 2024-01-31T05:26:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.534+0000 7f9f59fc4640 1 -- 172.21.15.45:0/2095635268 shutdown_connections 2024-01-31T05:26:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.534+0000 7f9f59fc4640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f9f3c0a1570 0x7f9f3c0a3a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.534+0000 7f9f59fc4640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f5419baa0 0x7f9f5419bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.534+0000 7f9f59fc4640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54106000 0x7f9f54197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.534+0000 7f9f59fc4640 1 --2- 172.21.15.45:0/2095635268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f54100f30 0x7f9f541971e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:26:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.534+0000 7f9f59fc4640 1 -- 172.21.15.45:0/2095635268 >> 172.21.15.45:0/2095635268 conn(0x7f9f540fc7d0 msgr2=0x7f9f540fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:26:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.534+0000 7f9f59fc4640 1 -- 172.21.15.45:0/2095635268 shutdown_connections 2024-01-31T05:26:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:26:57.534+0000 7f9f59fc4640 1 -- 172.21.15.45:0/2095635268 wait complete. 2024-01-31T05:26:57.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 402635 12.81 MB/sec execute 57 sec latency 15.300 ms 2024-01-31T05:26:58.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 405111 12.83 MB/sec execute 58 sec latency 13.717 ms 2024-01-31T05:26:58.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:26:58 smithi186 ceph-mon[35659]: pgmap v132: 97 pgs: 97 active+clean; 210 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 338 op/s 2024-01-31T05:26:58.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:26:58 smithi045 ceph-mon[31585]: pgmap v132: 97 pgs: 97 active+clean; 210 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 338 op/s 2024-01-31T05:26:58.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:26:58 smithi156 ceph-mon[35720]: pgmap v132: 97 pgs: 97 active+clean; 210 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 338 op/s 2024-01-31T05:26:59.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 408104 12.91 MB/sec execute 59 sec latency 13.680 ms 2024-01-31T05:27:00.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 408984 12.75 MB/sec execute 60 sec latency 76.396 ms 2024-01-31T05:27:00.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:00 smithi186 ceph-mon[35659]: pgmap v133: 97 pgs: 97 active+clean; 210 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 218 op/s 2024-01-31T05:27:00.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:00 smithi045 ceph-mon[31585]: pgmap v133: 97 pgs: 97 active+clean; 210 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 218 op/s 2024-01-31T05:27:00.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:00 smithi156 ceph-mon[35720]: pgmap v133: 97 pgs: 97 active+clean; 210 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 218 op/s 2024-01-31T05:27:01.779 INFO:tasks.workunit.client.0.smithi045.stdout: 1 411735 12.81 MB/sec execute 61 sec latency 36.504 ms 2024-01-31T05:27:02.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 413862 12.72 MB/sec execute 62 sec latency 184.292 ms 2024-01-31T05:27:02.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:02 smithi186 ceph-mon[35659]: pgmap v134: 97 pgs: 97 active+clean; 210 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 218 op/s 2024-01-31T05:27:02.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:02 smithi045 ceph-mon[31585]: pgmap v134: 97 pgs: 97 active+clean; 210 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 218 op/s 2024-01-31T05:27:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:02 smithi156 ceph-mon[35720]: pgmap v134: 97 pgs: 97 active+clean; 210 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 218 op/s 2024-01-31T05:27:03.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 415440 12.72 MB/sec execute 63 sec latency 310.681 ms 2024-01-31T05:27:04.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 418335 12.74 MB/sec execute 64 sec latency 9.326 ms 2024-01-31T05:27:04.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:04 smithi186 ceph-mon[35659]: pgmap v135: 97 pgs: 97 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.7 MiB/s wr, 369 op/s 2024-01-31T05:27:04.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:04 smithi045 ceph-mon[31585]: pgmap v135: 97 pgs: 97 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.7 MiB/s wr, 369 op/s 2024-01-31T05:27:04.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:04 smithi156 ceph-mon[35720]: pgmap v135: 97 pgs: 97 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.7 MiB/s wr, 369 op/s 2024-01-31T05:27:05.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 420008 12.70 MB/sec execute 65 sec latency 77.378 ms 2024-01-31T05:27:06.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 422502 12.72 MB/sec execute 66 sec latency 23.201 ms 2024-01-31T05:27:06.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:06 smithi186 ceph-mon[35659]: pgmap v136: 97 pgs: 97 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2024-01-31T05:27:06.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:06 smithi045 ceph-mon[31585]: pgmap v136: 97 pgs: 97 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2024-01-31T05:27:06.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:06 smithi156 ceph-mon[35720]: pgmap v136: 97 pgs: 97 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2024-01-31T05:27:07.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 424909 12.69 MB/sec execute 67 sec latency 62.454 ms 2024-01-31T05:27:08.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 426852 12.65 MB/sec execute 68 sec latency 31.304 ms 2024-01-31T05:27:08.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:08 smithi186 ceph-mon[35659]: pgmap v137: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 366 op/s 2024-01-31T05:27:08.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:08 smithi045 ceph-mon[31585]: pgmap v137: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 366 op/s 2024-01-31T05:27:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:08 smithi156 ceph-mon[35720]: pgmap v137: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 366 op/s 2024-01-31T05:27:09.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 429618 12.72 MB/sec execute 69 sec latency 13.133 ms 2024-01-31T05:27:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.625+0000 7f593d542640 1 -- 172.21.15.45:0/3819648323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5938068370 msgr2=0x7f59380687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.625+0000 7f593d542640 1 --2- 172.21.15.45:0/3819648323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5938068370 0x7f59380687f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5920009900 tx=0x7f592002bcd0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.625+0000 7f593d542640 1 -- 172.21.15.45:0/3819648323 shutdown_connections 2024-01-31T05:27:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.625+0000 7f593d542640 1 --2- 172.21.15.45:0/3819648323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5938068370 0x7f59380687f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.625+0000 7f593d542640 1 --2- 172.21.15.45:0/3819648323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5938068d30 0x7f593810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.625+0000 7f593d542640 1 --2- 172.21.15.45:0/3819648323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5938106e00 0x7f59381071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.625+0000 7f593d542640 1 -- 172.21.15.45:0/3819648323 >> 172.21.15.45:0/3819648323 conn(0x7f5938076a40 msgr2=0x7f5938076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.625+0000 7f593d542640 1 -- 172.21.15.45:0/3819648323 shutdown_connections 2024-01-31T05:27:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.625+0000 7f593d542640 1 -- 172.21.15.45:0/3819648323 wait complete. 2024-01-31T05:27:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f593d542640 1 Processor -- start 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f593d542640 1 -- start start 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f593d542640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5938068370 0x7f59381971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f593d542640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5938068d30 0x7f59381976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f593d542640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5938106e00 0x7f593819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f593d542640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593819c060 con 0x7f5938068d30 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f593d542640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593819c1d0 con 0x7f5938106e00 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f593d542640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593819c390 con 0x7f5938068370 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f592ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5938068d30 0x7f59381976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f592ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5938068d30 0x7f59381976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47518/0 (socket says 172.21.15.45:47518) 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f592ffff640 1 -- 172.21.15.45:0/114855178 learned_addr learned my addr 172.21.15.45:0/114855178 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f5936ffd640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5938068370 0x7f59381971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.626+0000 7f59377fe640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5938106e00 0x7f593819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.627+0000 7f592ffff640 1 -- 172.21.15.45:0/114855178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5938068370 msgr2=0x7f59381971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.627+0000 7f592ffff640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5938068370 0x7f59381971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.627+0000 7f592ffff640 1 -- 172.21.15.45:0/114855178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5938106e00 msgr2=0x7f593819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.627+0000 7f592ffff640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5938106e00 0x7f593819ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.627+0000 7f592ffff640 1 -- 172.21.15.45:0/114855178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5920009560 con 0x7f5938068d30 2024-01-31T05:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.627+0000 7f592ffff640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5938068d30 0x7f59381976e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5920005940 tx=0x7f592002c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.627+0000 7f5934ff9640 1 -- 172.21.15.45:0/114855178 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920004e50 con 0x7f5938068d30 2024-01-31T05:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.627+0000 7f5934ff9640 1 -- 172.21.15.45:0/114855178 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5920005000 con 0x7f5938068d30 2024-01-31T05:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.627+0000 7f5934ff9640 1 -- 172.21.15.45:0/114855178 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200039f0 con 0x7f5938068d30 2024-01-31T05:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.627+0000 7f593d542640 1 -- 172.21.15.45:0/114855178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f593819c610 con 0x7f5938068d30 2024-01-31T05:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.627+0000 7f593d542640 1 -- 172.21.15.45:0/114855178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59381a3ef0 con 0x7f5938068d30 2024-01-31T05:27:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.629+0000 7f5934ff9640 1 -- 172.21.15.45:0/114855178 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f5920005170 con 0x7f5938068d30 2024-01-31T05:27:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.629+0000 7f59403e4740 1 Processor -- start 2024-01-31T05:27:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.629+0000 7f59403e4740 1 -- start start 2024-01-31T05:27:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f59403e4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac62ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f59403e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac63420 0x55fd9ac70620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f59403e4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac70b60 0x55fd9ac72f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f59403e4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9abd2040 con 0x55fd9ac70b60 2024-01-31T05:27:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f59403e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9abcf730 con 0x55fd9ac63420 2024-01-31T05:27:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f59403e4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9abcb020 con 0x55fd9ac62b00 2024-01-31T05:27:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f5934ff9640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f59240a14f0 0x7f59240a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f5936ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac63420 0x55fd9ac70620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f59377fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac62ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f59377fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac62ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:38416/0 (socket says 172.21.15.45:38416) 2024-01-31T05:27:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f5936ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac63420 0x55fd9ac70620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:53504/0 (socket says 172.21.15.45:53504) 2024-01-31T05:27:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f59377fe640 1 -- 172.21.15.45:0/2893216712 learned_addr learned my addr 172.21.15.45:0/2893216712 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f592ffff640 1 --2- 172.21.15.45:0/2893216712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac70b60 0x55fd9ac72f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f59377fe640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f59240a14f0 0x7f59240a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f5934ff9640 1 -- 172.21.15.45:0/114855178 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f59200eefc0 con 0x7f5938068d30 2024-01-31T05:27:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f5936ffd640 1 -- 172.21.15.45:0/2893216712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 msgr2=0x55fd9ac62ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f5936ffd640 1 --2- 172.21.15.45:0/2893216712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac62ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f5936ffd640 1 -- 172.21.15.45:0/2893216712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac70b60 msgr2=0x55fd9ac72f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f5936ffd640 1 --2- 172.21.15.45:0/2893216712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac70b60 0x55fd9ac72f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.630+0000 7f5936ffd640 1 -- 172.21.15.45:0/2893216712 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd9abc2e40 con 0x55fd9ac63420 2024-01-31T05:27:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.631+0000 7f5936ffd640 1 --2- 172.21.15.45:0/2893216712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac63420 0x55fd9ac70620 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5928009c50 tx=0x7f5928011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca2e65122f4b446 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.631+0000 7f59377fe640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f59240a14f0 0x7f59240a39b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5930005c40 tx=0x7f5930004ab0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.631+0000 7f592d7fa640 1 -- 172.21.15.45:0/2893216712 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928004050 con 0x55fd9ac63420 2024-01-31T05:27:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.631+0000 7f592d7fa640 1 -- 172.21.15.45:0/2893216712 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5928005020 con 0x55fd9ac63420 2024-01-31T05:27:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.631+0000 7f592d7fa640 1 -- 172.21.15.45:0/2893216712 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928004c00 con 0x55fd9ac63420 2024-01-31T05:27:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 -- 172.21.15.45:0/2893216712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac63420 msgr2=0x55fd9ac70620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 --2- 172.21.15.45:0/2893216712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac63420 0x55fd9ac70620 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5928009c50 tx=0x7f5928011810 comp rx=0 tx=0).stop 2024-01-31T05:27:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 -- 172.21.15.45:0/2893216712 shutdown_connections 2024-01-31T05:27:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 --2- 172.21.15.45:0/2893216712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac70b60 0x55fd9ac72f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 --2- 172.21.15.45:0/2893216712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac63420 0x55fd9ac70620 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 --2- 172.21.15.45:0/2893216712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac62ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 -- 172.21.15.45:0/2893216712 >> 172.21.15.45:0/2893216712 conn(0x55fd9ac5d980 msgr2=0x55fd9ac5dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 -- 172.21.15.45:0/2893216712 shutdown_connections 2024-01-31T05:27:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 -- 172.21.15.45:0/2893216712 wait complete. 2024-01-31T05:27:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 -- start start 2024-01-31T05:27:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 Processor -- start 2024-01-31T05:27:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac6da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac6df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9ac6e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9ab841b0 con 0x55fd9ac63420 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9ab83e50 con 0x55fd9ac70b60 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9ab49090 con 0x55fd9ac62b00 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.632+0000 7f59403e4740 10 client.0 _ll_register_callbacks cb 0x55fd9ab6ea90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f592ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac6df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f59377fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9ac6e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f5936ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac6da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f592ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac6df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47540/0 (socket says 172.21.15.45:47540) 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f592ffff640 1 -- 172.21.15.45:0/1109679585 learned_addr learned my addr 172.21.15.45:0/1109679585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f59377fe640 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9ac6e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:53506/0 (socket says 172.21.15.45:53506) 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f5936ffd640 1 -- 172.21.15.45:0/1109679585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 msgr2=0x55fd9ac6e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f5936ffd640 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9ac6e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f5936ffd640 1 -- 172.21.15.45:0/1109679585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 msgr2=0x55fd9ac6df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f5936ffd640 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac6df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f5936ffd640 1 -- 172.21.15.45:0/1109679585 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5928009850 con 0x55fd9ac62b00 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f592ffff640 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac6df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f5936ffd640 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac6da30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5928000c00 tx=0x7f5928005d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26183fc0eba2db2a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f591e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f591e7fc640 1 -- 172.21.15.45:0/1109679585 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928004050 con 0x55fd9ac62b00 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f591e7fc640 1 -- 172.21.15.45:0/1109679585 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f59280041b0 con 0x55fd9ac62b00 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.633+0000 7f59403e4740 1 -- 172.21.15.45:0/1109679585 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd9abc4010 con 0x55fd9ac62b00 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f591e7fc640 1 -- 172.21.15.45:0/1109679585 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5928004370 con 0x55fd9ac62b00 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f59403e4740 10 client.25303 fetch_fsmap learned FSMap version 26 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f59403e4740 1 -- 172.21.15.45:0/1109679585 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fd9abc96a0 con 0x55fd9ac62b00 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f591e7fc640 1 -- 172.21.15.45:0/1109679585 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd9abc4010 con 0x55fd9ac62b00 2024-01-31T05:27:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f591e7fc640 1 -- 172.21.15.45:0/1109679585 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5928021040 con 0x55fd9ac62b00 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f59403e4740 10 client.25303 fetch_fsmap finished waiting for FSMap version 26 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f59403e4740 10 client.25303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f59403e4740 20 client.25303 populate_metadata read hostname 'smithi045' 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f59403e4740 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fd9ac808a0 0x55fd9ac82cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f59403e4740 4 client.25303 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f59403e4740 1 -- 172.21.15.45:0/1109679585 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd9abc3d30 con 0x55fd9ac808a0 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f592ffff640 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fd9ac808a0 0x55fd9ac82cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f591e7fc640 10 client.25303 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.634+0000 7f592ffff640 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fd9ac808a0 0x55fd9ac82cb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a794304545a209d server_cookie=73ac19ab80e3ecf8 in_seq=0 out_seq=0 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f591e7fc640 1 -- 172.21.15.45:0/1109679585 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd9abc3d30 con 0x55fd9ac808a0 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f591e7fc640 10 client.25303 handle_command_reply: tid=0 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f59403e4740 1 client.25303 shutdown 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f59403e4740 1 -- 172.21.15.45:0/1109679585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 msgr2=0x55fd9ac6da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f59403e4740 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac6da30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5928000c00 tx=0x7f5928005d10 comp rx=0 tx=0).stop 2024-01-31T05:27:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f59403e4740 1 -- 172.21.15.45:0/1109679585 shutdown_connections 2024-01-31T05:27:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f59403e4740 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fd9ac808a0 0x55fd9ac82cb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f59403e4740 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac6df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f59403e4740 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9ac6e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f591e7fc640 0 client.25303 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f59403e4740 1 --2- 172.21.15.45:0/1109679585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac6da30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f59403e4740 1 -- 172.21.15.45:0/1109679585 >> 172.21.15.45:0/1109679585 conn(0x55fd9ac5d980 msgr2=0x55fd9ac5f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f59403e4740 1 -- 172.21.15.45:0/1109679585 shutdown_connections 2024-01-31T05:27:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f59403e4740 1 -- 172.21.15.45:0/1109679585 wait complete. 2024-01-31T05:27:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.635+0000 7f59403e4740 20 client.25303 trim_cache size 0 max 16384 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.649+0000 7f59403e4740 1 Processor -- start 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.649+0000 7f59403e4740 1 -- start start 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.649+0000 7f59403e4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac70160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.649+0000 7f59403e4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac6e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.649+0000 7f59403e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9ac6acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.649+0000 7f59403e4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9ac590b0 con 0x55fd9ac63420 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.649+0000 7f59403e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9ac806a0 con 0x55fd9ac70b60 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.649+0000 7f59403e4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9ac6ee50 con 0x55fd9ac62b00 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f59377fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9ac6acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f592ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac6e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f59377fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9ac6acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:53522/0 (socket says 172.21.15.45:53522) 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f592ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac6e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47554/0 (socket says 172.21.15.45:47554) 2024-01-31T05:27:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f59377fe640 1 -- 172.21.15.45:0/1820202722 learned_addr learned my addr 172.21.15.45:0/1820202722 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f5936ffd640 1 --2- 172.21.15.45:0/1820202722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac70160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f59377fe640 1 -- 172.21.15.45:0/1820202722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 msgr2=0x55fd9ac70160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f59377fe640 1 --2- 172.21.15.45:0/1820202722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac70160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f59377fe640 1 -- 172.21.15.45:0/1820202722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 msgr2=0x55fd9ac6e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f59377fe640 1 --2- 172.21.15.45:0/1820202722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac6e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f59377fe640 1 -- 172.21.15.45:0/1820202722 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd9ab66030 con 0x55fd9ac70b60 2024-01-31T05:27:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f592ffff640 1 --2- 172.21.15.45:0/1820202722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac6e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:27:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f59377fe640 1 --2- 172.21.15.45:0/1820202722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9ac6acb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5930013430 tx=0x7f5930009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=898cafc7c0a03ecd server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f592cff9640 1 -- 172.21.15.45:0/1820202722 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930004d20 con 0x55fd9ac70b60 2024-01-31T05:27:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f592cff9640 1 -- 172.21.15.45:0/1820202722 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5930004e80 con 0x55fd9ac70b60 2024-01-31T05:27:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.650+0000 7f592cff9640 1 -- 172.21.15.45:0/1820202722 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930010670 con 0x55fd9ac70b60 2024-01-31T05:27:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.651+0000 7f59403e4740 1 -- 172.21.15.45:0/1820202722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 msgr2=0x55fd9ac6acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.651+0000 7f59403e4740 1 --2- 172.21.15.45:0/1820202722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9ac6acb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5930013430 tx=0x7f5930009bd0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.651+0000 7f59403e4740 1 -- 172.21.15.45:0/1820202722 shutdown_connections 2024-01-31T05:27:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.651+0000 7f59403e4740 1 --2- 172.21.15.45:0/1820202722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac6e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.651+0000 7f59403e4740 1 --2- 172.21.15.45:0/1820202722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9ac6acb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.651+0000 7f59403e4740 1 --2- 172.21.15.45:0/1820202722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac70160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.651+0000 7f59403e4740 1 -- 172.21.15.45:0/1820202722 >> 172.21.15.45:0/1820202722 conn(0x55fd9ac5d980 msgr2=0x55fd9ac87620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.651+0000 7f59403e4740 1 -- 172.21.15.45:0/1820202722 shutdown_connections 2024-01-31T05:27:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.651+0000 7f59403e4740 1 -- 172.21.15.45:0/1820202722 wait complete. 2024-01-31T05:27:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.651+0000 7f59403e4740 1 -- start start 2024-01-31T05:27:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f59403e4740 1 Processor -- start 2024-01-31T05:27:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f59403e4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac76d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f59403e4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac77260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f59403e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9acd0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f59403e4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9acd13a0 con 0x55fd9ac63420 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f59403e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9acd1510 con 0x55fd9ac70b60 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f59403e4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9acd16d0 con 0x55fd9ac62b00 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f59403e4740 10 client.0 _ll_register_callbacks cb 0x7f59280097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f592ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac77260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f592ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac77260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47566/0 (socket says 172.21.15.45:47566) 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f5936ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac76d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f592ffff640 1 -- 172.21.15.45:0/1460280 learned_addr learned my addr 172.21.15.45:0/1460280 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f59377fe640 1 --2- 172.21.15.45:0/1460280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9acd0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f592ffff640 1 -- 172.21.15.45:0/1460280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 msgr2=0x55fd9ac76d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f592ffff640 1 --2- 172.21.15.45:0/1460280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac76d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f592ffff640 1 -- 172.21.15.45:0/1460280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 msgr2=0x55fd9acd0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f592ffff640 1 --2- 172.21.15.45:0/1460280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9acd0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f592ffff640 1 -- 172.21.15.45:0/1460280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59381a3ef0 con 0x55fd9ac63420 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f592ffff640 1 --2- 172.21.15.45:0/1460280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac77260 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f59200ecd60 tx=0x7f5920044d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7789febe2b40e16e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f591e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f591e7fc640 1 -- 172.21.15.45:0/1460280 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200c0070 con 0x55fd9ac63420 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.652+0000 7f591e7fc640 1 -- 172.21.15.45:0/1460280 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f59200c42b0 con 0x55fd9ac63420 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f59403e4740 1 -- 172.21.15.45:0/1460280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd9a8ca600 con 0x55fd9ac63420 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f591e7fc640 1 -- 172.21.15.45:0/1460280 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59200ec9e0 con 0x55fd9ac63420 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f59403e4740 10 client.15759 fetch_fsmap learned FSMap version 26 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f59403e4740 1 -- 172.21.15.45:0/1460280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f5930013170 con 0x55fd9ac63420 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f591e7fc640 1 -- 172.21.15.45:0/1460280 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd9a8ca600 con 0x55fd9ac63420 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f591e7fc640 1 -- 172.21.15.45:0/1460280 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f592003c070 con 0x55fd9ac63420 2024-01-31T05:27:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f59403e4740 10 client.15759 fetch_fsmap finished waiting for FSMap version 26 2024-01-31T05:27:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f59403e4740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:27:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f59403e4740 20 client.15759 populate_metadata read hostname 'smithi045' 2024-01-31T05:27:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f59403e4740 1 --2- 172.21.15.45:0/1460280 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fd9acd8cf0 0x55fd9acdb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f59403e4740 4 client.15759 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:27:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f59403e4740 1 -- 172.21.15.45:0/1460280 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd9a91dc50 con 0x55fd9acd8cf0 2024-01-31T05:27:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.653+0000 7f5936ffd640 1 --2- 172.21.15.45:0/1460280 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fd9acd8cf0 0x55fd9acdb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.654+0000 7f5936ffd640 1 --2- 172.21.15.45:0/1460280 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fd9acd8cf0 0x55fd9acdb110 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f890d2bdf5c06c7 server_cookie=20ebbc002558fd11 in_seq=0 out_seq=0 2024-01-31T05:27:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.654+0000 7f591e7fc640 10 client.15759 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.663+0000 7f591e7fc640 1 -- 172.21.15.45:0/1460280 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd9a91dc50 con 0x55fd9acd8cf0 2024-01-31T05:27:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.663+0000 7f591e7fc640 10 client.15759 handle_command_reply: tid=0 2024-01-31T05:27:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.663+0000 7f59403e4740 1 client.15759 shutdown 2024-01-31T05:27:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.663+0000 7f59403e4740 1 -- 172.21.15.45:0/1460280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 msgr2=0x55fd9ac77260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.663+0000 7f59403e4740 1 --2- 172.21.15.45:0/1460280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac77260 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f59200ecd60 tx=0x7f5920044d30 comp rx=0 tx=0).stop 2024-01-31T05:27:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f59403e4740 1 -- 172.21.15.45:0/1460280 shutdown_connections 2024-01-31T05:27:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f59403e4740 1 --2- 172.21.15.45:0/1460280 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fd9acd8cf0 0x55fd9acdb110 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f59403e4740 1 --2- 172.21.15.45:0/1460280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd9ac63420 0x55fd9ac77260 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f591e7fc640 0 client.15759 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f59403e4740 1 --2- 172.21.15.45:0/1460280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd9ac70b60 0x55fd9acd0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f59403e4740 1 --2- 172.21.15.45:0/1460280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd9ac62b00 0x55fd9ac76d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f59403e4740 1 -- 172.21.15.45:0/1460280 >> 172.21.15.45:0/1460280 conn(0x55fd9ac5d980 msgr2=0x55fd9ac5c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f59403e4740 1 -- 172.21.15.45:0/1460280 shutdown_connections 2024-01-31T05:27:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f59403e4740 1 -- 172.21.15.45:0/1460280 wait complete. 2024-01-31T05:27:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f59403e4740 20 client.15759 trim_cache size 0 max 16384 2024-01-31T05:27:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:27:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:27:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:27:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:27:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f593d542640 1 -- 172.21.15.45:0/114855178 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f59240a14f0 msgr2=0x7f59240a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f593d542640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f59240a14f0 0x7f59240a39b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5930005c40 tx=0x7f5930004ab0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f593d542640 1 -- 172.21.15.45:0/114855178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5938068d30 msgr2=0x7f59381976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f593d542640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5938068d30 0x7f59381976e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5920005940 tx=0x7f592002c690 comp rx=0 tx=0).stop 2024-01-31T05:27:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f593d542640 1 -- 172.21.15.45:0/114855178 shutdown_connections 2024-01-31T05:27:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f593d542640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f59240a14f0 0x7f59240a39b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f593d542640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5938068d30 0x7f59381976e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f593d542640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5938106e00 0x7f593819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f593d542640 1 --2- 172.21.15.45:0/114855178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5938068370 0x7f59381971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f593d542640 1 -- 172.21.15.45:0/114855178 >> 172.21.15.45:0/114855178 conn(0x7f5938076a40 msgr2=0x7f59380fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f593d542640 1 -- 172.21.15.45:0/114855178 shutdown_connections 2024-01-31T05:27:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:10.664+0000 7f593d542640 1 -- 172.21.15.45:0/114855178 wait complete. 2024-01-31T05:27:10.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 431999 12.69 MB/sec execute 70 sec latency 72.559 ms 2024-01-31T05:27:10.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:10 smithi186 ceph-mon[35659]: pgmap v138: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 255 op/s 2024-01-31T05:27:10.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:10 smithi045 ceph-mon[31585]: pgmap v138: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 255 op/s 2024-01-31T05:27:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:10 smithi156 ceph-mon[35720]: pgmap v138: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 255 op/s 2024-01-31T05:27:11.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 433920 12.66 MB/sec execute 71 sec latency 22.291 ms 2024-01-31T05:27:12.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 436688 12.72 MB/sec execute 72 sec latency 12.599 ms 2024-01-31T05:27:12.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:12 smithi186 ceph-mon[35659]: pgmap v139: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 255 op/s 2024-01-31T05:27:12.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:12 smithi045 ceph-mon[31585]: pgmap v139: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 255 op/s 2024-01-31T05:27:12.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:12 smithi156 ceph-mon[35720]: pgmap v139: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 255 op/s 2024-01-31T05:27:13.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 439095 12.70 MB/sec execute 73 sec latency 64.259 ms 2024-01-31T05:27:14.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 441106 12.66 MB/sec execute 74 sec latency 23.645 ms 2024-01-31T05:27:14.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:14 smithi186 ceph-mon[35659]: pgmap v140: 97 pgs: 97 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 373 op/s 2024-01-31T05:27:14.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:14 smithi045 ceph-mon[31585]: pgmap v140: 97 pgs: 97 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 373 op/s 2024-01-31T05:27:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:14 smithi156 ceph-mon[35720]: pgmap v140: 97 pgs: 97 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 373 op/s 2024-01-31T05:27:15.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 443671 12.72 MB/sec execute 75 sec latency 69.265 ms 2024-01-31T05:27:16.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 446012 12.66 MB/sec execute 76 sec latency 20.765 ms 2024-01-31T05:27:16.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:16 smithi186 ceph-mon[35659]: pgmap v141: 97 pgs: 97 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-01-31T05:27:16.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:16 smithi045 ceph-mon[31585]: pgmap v141: 97 pgs: 97 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-01-31T05:27:16.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:16 smithi156 ceph-mon[35720]: pgmap v141: 97 pgs: 97 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-01-31T05:27:17.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 448178 12.67 MB/sec execute 77 sec latency 7.833 ms 2024-01-31T05:27:18.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 450918 12.72 MB/sec execute 78 sec latency 14.218 ms 2024-01-31T05:27:18.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:18 smithi186 ceph-mon[35659]: pgmap v142: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.3 MiB/s wr, 333 op/s 2024-01-31T05:27:18.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:18 smithi045 ceph-mon[31585]: pgmap v142: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.3 MiB/s wr, 333 op/s 2024-01-31T05:27:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:18 smithi156 ceph-mon[35720]: pgmap v142: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.3 MiB/s wr, 333 op/s 2024-01-31T05:27:19.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 453268 12.70 MB/sec execute 79 sec latency 95.272 ms 2024-01-31T05:27:20.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 455027 12.65 MB/sec execute 80 sec latency 110.224 ms 2024-01-31T05:27:20.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:20 smithi186 ceph-mon[35659]: pgmap v143: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 229 op/s 2024-01-31T05:27:20.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:20 smithi045 ceph-mon[31585]: pgmap v143: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 229 op/s 2024-01-31T05:27:20.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:20 smithi156 ceph-mon[35720]: pgmap v143: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 229 op/s 2024-01-31T05:27:21.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 457879 12.72 MB/sec execute 81 sec latency 21.042 ms 2024-01-31T05:27:22.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 459595 12.85 MB/sec execute 82 sec latency 75.608 ms 2024-01-31T05:27:22.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:22 smithi186 ceph-mon[35659]: pgmap v144: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 229 op/s 2024-01-31T05:27:22.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:22 smithi045 ceph-mon[31585]: pgmap v144: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 229 op/s 2024-01-31T05:27:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:22 smithi156 ceph-mon[35720]: pgmap v144: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 229 op/s 2024-01-31T05:27:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.751+0000 7fb8eff6b640 1 -- 172.21.15.45:0/2021914002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e8100800 msgr2=0x7fb8e810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.751+0000 7fb8eff6b640 1 --2- 172.21.15.45:0/2021914002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e8100800 0x7fb8e810ca20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009ca0 tx=0x7fb8e002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 -- 172.21.15.45:0/2021914002 shutdown_connections 2024-01-31T05:27:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 --2- 172.21.15.45:0/2021914002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e8100800 0x7fb8e810ca20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 --2- 172.21.15.45:0/2021914002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80ffe40 0x7fb8e81002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 --2- 172.21.15.45:0/2021914002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8e8104b90 0x7fb8e8104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 -- 172.21.15.45:0/2021914002 >> 172.21.15.45:0/2021914002 conn(0x7fb8e8076a40 msgr2=0x7fb8e8076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 -- 172.21.15.45:0/2021914002 shutdown_connections 2024-01-31T05:27:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 -- 172.21.15.45:0/2021914002 wait complete. 2024-01-31T05:27:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 Processor -- start 2024-01-31T05:27:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 -- start start 2024-01-31T05:27:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80ffe40 0x7fb8e8196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e8100800 0x7fb8e81965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8e8104b90 0x7fb8e819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e819af50 con 0x7fb8e8100800 2024-01-31T05:27:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e819b0c0 con 0x7fb8e80ffe40 2024-01-31T05:27:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.752+0000 7fb8eff6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e819b280 con 0x7fb8e8104b90 2024-01-31T05:27:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8ed506640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e8100800 0x7fb8e81965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8edd07640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80ffe40 0x7fb8e8196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8edd07640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80ffe40 0x7fb8e8196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:46354/0 (socket says 172.21.15.45:46354) 2024-01-31T05:27:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8ed506640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e8100800 0x7fb8e81965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40950/0 (socket says 172.21.15.45:40950) 2024-01-31T05:27:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8edd07640 1 -- 172.21.15.45:0/1331870473 learned_addr learned my addr 172.21.15.45:0/1331870473 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8ee508640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8e8104b90 0x7fb8e819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8ed506640 1 -- 172.21.15.45:0/1331870473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8e8104b90 msgr2=0x7fb8e819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8ed506640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8e8104b90 0x7fb8e819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8ed506640 1 -- 172.21.15.45:0/1331870473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80ffe40 msgr2=0x7fb8e8196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8ed506640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80ffe40 0x7fb8e8196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8ed506640 1 -- 172.21.15.45:0/1331870473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e0009900 con 0x7fb8e8100800 2024-01-31T05:27:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8ed506640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e8100800 0x7fb8e81965d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00d6d0 tx=0x7fb8dc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8daffd640 1 -- 172.21.15.45:0/1331870473 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc005220 con 0x7fb8e8100800 2024-01-31T05:27:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8daffd640 1 -- 172.21.15.45:0/1331870473 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8dc009ce0 con 0x7fb8e8100800 2024-01-31T05:27:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8eff6b640 1 -- 172.21.15.45:0/1331870473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e819b560 con 0x7fb8e8100800 2024-01-31T05:27:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8daffd640 1 -- 172.21.15.45:0/1331870473 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc004d90 con 0x7fb8e8100800 2024-01-31T05:27:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.753+0000 7fb8eff6b640 1 -- 172.21.15.45:0/1331870473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e81a2de0 con 0x7fb8e8100800 2024-01-31T05:27:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.756+0000 7fb8daffd640 1 -- 172.21.15.45:0/1331870473 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fb8dc005480 con 0x7fb8e8100800 2024-01-31T05:27:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.756+0000 7fb8f2e0c740 1 Processor -- start 2024-01-31T05:27:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.756+0000 7fb8f2e0c740 1 -- start start 2024-01-31T05:27:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8daffd640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb8d40a1570 0x7fb8d40a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8f2e0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9060 0x561c085e9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8f2e0c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8f2e0c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085f2750 0x561c085f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8f2e0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c08558040 con 0x561c085e9060 2024-01-31T05:27:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8f2e0c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c08555730 con 0x561c085f2750 2024-01-31T05:27:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8f2e0c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c08551020 con 0x561c085e9980 2024-01-31T05:27:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8edd07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9060 0x561c085e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8ee508640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb8d40a1570 0x7fb8d40a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8ed506640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8edd07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9060 0x561c085e9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40956/0 (socket says 172.21.15.45:40956) 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8daffd640 1 -- 172.21.15.45:0/1331870473 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb8dc0cc8e0 con 0x7fb8e8100800 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8edd07640 1 -- 172.21.15.45:0/3248817840 learned_addr learned my addr 172.21.15.45:0/3248817840 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8ee508640 1 --2- 172.21.15.45:0/3248817840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085f2750 0x561c085f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8edd07640 1 -- 172.21.15.45:0/3248817840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 msgr2=0x561c085f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8edd07640 1 --2- 172.21.15.45:0/3248817840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8edd07640 1 -- 172.21.15.45:0/3248817840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085f2750 msgr2=0x561c085f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8edd07640 1 --2- 172.21.15.45:0/3248817840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085f2750 0x561c085f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8edd07640 1 -- 172.21.15.45:0/3248817840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c08548e40 con 0x561c085e9060 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8ed506640 1 --2- 172.21.15.45:0/3248817840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:27:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.757+0000 7fb8ee508640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb8d40a1570 0x7fb8d40a3a30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0004af0 tx=0x7fb8e0034440 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.758+0000 7fb8edd07640 1 --2- 172.21.15.45:0/3248817840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9060 0x561c085e9440 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009d90 tx=0x7fb8e40119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95eeb7d3ff15049e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.758+0000 7fb8d3fff640 1 -- 172.21.15.45:0/3248817840 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4016070 con 0x561c085e9060 2024-01-31T05:27:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.758+0000 7fb8d3fff640 1 -- 172.21.15.45:0/3248817840 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8e400e930 con 0x561c085e9060 2024-01-31T05:27:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.758+0000 7fb8d3fff640 1 -- 172.21.15.45:0/3248817840 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e401a8e0 con 0x561c085e9060 2024-01-31T05:27:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.758+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/3248817840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9060 msgr2=0x561c085e9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.758+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/3248817840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9060 0x561c085e9440 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009d90 tx=0x7fb8e40119a0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.758+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/3248817840 shutdown_connections 2024-01-31T05:27:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.758+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/3248817840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085f2750 0x561c085f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.758+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/3248817840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.758+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/3248817840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9060 0x561c085e9440 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.758+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/3248817840 >> 172.21.15.45:0/3248817840 conn(0x561c085e3980 msgr2=0x561c085e3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.759+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/3248817840 shutdown_connections 2024-01-31T05:27:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.759+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/3248817840 wait complete. 2024-01-31T05:27:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.759+0000 7fb8f2e0c740 1 -- start start 2024-01-31T05:27:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.759+0000 7fb8f2e0c740 1 Processor -- start 2024-01-31T05:27:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.759+0000 7fb8f2e0c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.759+0000 7fb8f2e0c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.759+0000 7fb8f2e0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 0x561c085f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.759+0000 7fb8f2e0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c0850a1b0 con 0x561c085f2750 2024-01-31T05:27:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.759+0000 7fb8f2e0c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c08509e50 con 0x561c085e9060 2024-01-31T05:27:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.759+0000 7fb8f2e0c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c084cf090 con 0x561c085e9980 2024-01-31T05:27:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.759+0000 7fb8f2e0c740 10 client.0 _ll_register_callbacks cb 0x561c084f4a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:27:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.759+0000 7fb8ee508640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 0x561c085f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8edd07640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8ed506640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8ee508640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 0x561c085f0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40970/0 (socket says 172.21.15.45:40970) 2024-01-31T05:27:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8ee508640 1 -- 172.21.15.45:0/1029409055 learned_addr learned my addr 172.21.15.45:0/1029409055 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8ee508640 1 -- 172.21.15.45:0/1029409055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 msgr2=0x561c085efbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8ee508640 1 --2- 172.21.15.45:0/1029409055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085efbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8ee508640 1 -- 172.21.15.45:0/1029409055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 msgr2=0x561c085ef6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8ee508640 1 --2- 172.21.15.45:0/1029409055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085ef6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8ee508640 1 -- 172.21.15.45:0/1029409055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e4009a70 con 0x561c085f2750 2024-01-31T05:27:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8edd07640 1 --2- 172.21.15.45:0/1029409055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085ef6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8ee508640 1 --2- 172.21.15.45:0/1029409055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 0x561c085f0120 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009ca0 tx=0x7fb8e002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6bfca38580672b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8d17fa640 1 -- 172.21.15.45:0/1029409055 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0032a10 con 0x561c085f2750 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8d17fa640 1 -- 172.21.15.45:0/1029409055 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb8e0003970 con 0x561c085f2750 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1029409055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c0854a010 con 0x561c085f2750 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8d17fa640 1 -- 172.21.15.45:0/1029409055 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8e0003b30 con 0x561c085f2750 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8f2e0c740 10 client.15771 fetch_fsmap learned FSMap version 26 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1029409055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561c0854f6a0 con 0x561c085f2750 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.760+0000 7fb8d17fa640 1 -- 172.21.15.45:0/1029409055 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c0854a010 con 0x561c085f2750 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.761+0000 7fb8d17fa640 1 -- 172.21.15.45:0/1029409055 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb8e004d440 con 0x561c085f2750 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.761+0000 7fb8f2e0c740 10 client.15771 fetch_fsmap finished waiting for FSMap version 26 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.761+0000 7fb8f2e0c740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:27:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.761+0000 7fb8f2e0c740 20 client.15771 populate_metadata read hostname 'smithi045' 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.761+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/1029409055 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561c08602600 0x561c08604a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.761+0000 7fb8f2e0c740 4 client.15771 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.761+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1029409055 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c08549d30 con 0x561c08602600 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.761+0000 7fb8edd07640 1 --2- 172.21.15.45:0/1029409055 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561c08602600 0x561c08604a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.761+0000 7fb8d17fa640 10 client.15771 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.761+0000 7fb8edd07640 1 --2- 172.21.15.45:0/1029409055 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561c08602600 0x561c08604a20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65df279ca3c8397a server_cookie=a7ba135033ae3e5c in_seq=0 out_seq=0 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8d17fa640 1 -- 172.21.15.45:0/1029409055 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561c08549d30 con 0x561c08602600 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8d17fa640 10 client.15771 handle_command_reply: tid=0 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8f2e0c740 1 client.15771 shutdown 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1029409055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 msgr2=0x561c085f0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/1029409055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 0x561c085f0120 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009ca0 tx=0x7fb8e002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1029409055 shutdown_connections 2024-01-31T05:27:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/1029409055 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561c08602600 0x561c08604a20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/1029409055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 0x561c085f0120 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/1029409055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085efbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8d17fa640 0 client.15771 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/1029409055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085ef6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1029409055 >> 172.21.15.45:0/1029409055 conn(0x561c085e3980 msgr2=0x561c085e56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1029409055 shutdown_connections 2024-01-31T05:27:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1029409055 wait complete. 2024-01-31T05:27:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.762+0000 7fb8f2e0c740 20 client.15771 trim_cache size 0 max 16384 2024-01-31T05:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8f2e0c740 1 Processor -- start 2024-01-31T05:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8f2e0c740 1 -- start start 2024-01-31T05:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8f2e0c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8f2e0c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085ec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8f2e0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 0x561c085ece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8f2e0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c08549d30 con 0x561c085f2750 2024-01-31T05:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8f2e0c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c0854a010 con 0x561c085e9060 2024-01-31T05:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8f2e0c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c085ddf20 con 0x561c085e9980 2024-01-31T05:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8ee508640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 0x561c085ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8ee508640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 0x561c085ece80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40974/0 (socket says 172.21.15.45:40974) 2024-01-31T05:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8ee508640 1 -- 172.21.15.45:0/252772707 learned_addr learned my addr 172.21.15.45:0/252772707 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8ed506640 1 --2- 172.21.15.45:0/252772707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085ec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8edd07640 1 --2- 172.21.15.45:0/252772707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8ee508640 1 -- 172.21.15.45:0/252772707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 msgr2=0x561c085ec940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8ee508640 1 --2- 172.21.15.45:0/252772707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085ec940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8ee508640 1 -- 172.21.15.45:0/252772707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 msgr2=0x561c085eff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8ee508640 1 --2- 172.21.15.45:0/252772707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085eff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8ee508640 1 -- 172.21.15.45:0/252772707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c084ec030 con 0x561c085f2750 2024-01-31T05:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.776+0000 7fb8edd07640 1 --2- 172.21.15.45:0/252772707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8ee508640 1 --2- 172.21.15.45:0/252772707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 0x561c085ece80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb8e002c630 tx=0x7fb8e0032520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b40fe400319e2a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8d37fe640 1 -- 172.21.15.45:0/252772707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e004d640 con 0x561c085f2750 2024-01-31T05:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8d37fe640 1 -- 172.21.15.45:0/252772707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8e0003e30 con 0x561c085f2750 2024-01-31T05:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8d37fe640 1 -- 172.21.15.45:0/252772707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00042a0 con 0x561c085f2750 2024-01-31T05:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/252772707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 msgr2=0x561c085ece80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/252772707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 0x561c085ece80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb8e002c630 tx=0x7fb8e0032520 comp rx=0 tx=0).stop 2024-01-31T05:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/252772707 shutdown_connections 2024-01-31T05:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/252772707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085f2750 0x561c085ece80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/252772707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085e9980 0x561c085ec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/252772707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/252772707 >> 172.21.15.45:0/252772707 conn(0x561c085e3980 msgr2=0x561c085edf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/252772707 shutdown_connections 2024-01-31T05:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.777+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/252772707 wait complete. 2024-01-31T05:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8f2e0c740 1 -- start start 2024-01-31T05:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8f2e0c740 1 Processor -- start 2024-01-31T05:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8f2e0c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8f2e0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9980 0x561c085f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8f2e0c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085f2750 0x561c085f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8f2e0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c085f84a0 con 0x561c085e9980 2024-01-31T05:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8f2e0c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c085f8610 con 0x561c085e9060 2024-01-31T05:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8f2e0c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c085f87d0 con 0x561c085f2750 2024-01-31T05:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8f2e0c740 10 client.0 _ll_register_callbacks cb 0x7fb8e4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8ed506640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9980 0x561c085f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8ed506640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9980 0x561c085f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40986/0 (socket says 172.21.15.45:40986) 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8ed506640 1 -- 172.21.15.45:0/1915505045 learned_addr learned my addr 172.21.15.45:0/1915505045 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8ee508640 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085f2750 0x561c085f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.778+0000 7fb8edd07640 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8ed506640 1 -- 172.21.15.45:0/1915505045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085f2750 msgr2=0x561c085f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8ed506640 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085f2750 0x561c085f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8ed506640 1 -- 172.21.15.45:0/1915505045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 msgr2=0x561c085f94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8ed506640 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085f94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8ed506640 1 -- 172.21.15.45:0/1915505045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c0854f6a0 con 0x561c085e9980 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8edd07640 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8ed506640 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9980 0x561c085f7a20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0ca680 tx=0x7fb8dc0bc160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af830f06ccaecfd server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8d17fa640 1 -- 172.21.15.45:0/1915505045 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0ca300 con 0x561c085e9980 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8d17fa640 1 -- 172.21.15.45:0/1915505045 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb8dc0ca460 con 0x561c085e9980 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1915505045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8e400e930 con 0x561c085e9980 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8d17fa640 1 -- 172.21.15.45:0/1915505045 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8dc095ef0 con 0x561c085e9980 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8f2e0c740 10 client.15777 fetch_fsmap learned FSMap version 26 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1915505045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561c08511a50 con 0x561c085e9980 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8d17fa640 1 -- 172.21.15.45:0/1915505045 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8e400e930 con 0x561c085e9980 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8d17fa640 1 -- 172.21.15.45:0/1915505045 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb8dc0bc370 con 0x561c085e9980 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8f2e0c740 10 client.15777 fetch_fsmap finished waiting for FSMap version 26 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8f2e0c740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.779+0000 7fb8f2e0c740 20 client.15777 populate_metadata read hostname 'smithi045' 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.780+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561c0865a090 0x561c0865c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.780+0000 7fb8f2e0c740 4 client.15777 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:27:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.780+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1915505045 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561c08250600 con 0x561c0865a090 2024-01-31T05:27:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.780+0000 7fb8edd07640 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561c0865a090 0x561c0865c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:23.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 462372 12.90 MB/sec execute 83 sec latency 8.191 ms 2024-01-31T05:27:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.780+0000 7fb8d17fa640 10 client.15777 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.780+0000 7fb8edd07640 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561c0865a090 0x561c0865c4b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4869f617a7907a2b server_cookie=417f0f37f3851652 in_seq=0 out_seq=0 2024-01-31T05:27:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8d17fa640 1 -- 172.21.15.45:0/1915505045 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561c08250600 con 0x561c0865a090 2024-01-31T05:27:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8d17fa640 10 client.15777 handle_command_reply: tid=0 2024-01-31T05:27:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8f2e0c740 1 client.15777 shutdown 2024-01-31T05:27:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1915505045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9980 msgr2=0x561c085f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9980 0x561c085f7a20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0ca680 tx=0x7fb8dc0bc160 comp rx=0 tx=0).stop 2024-01-31T05:27:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1915505045 shutdown_connections 2024-01-31T05:27:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561c0865a090 0x561c0865c4b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c085e9980 0x561c085f7a20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c085f2750 0x561c085f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8f2e0c740 1 --2- 172.21.15.45:0/1915505045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c085e9060 0x561c085f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1915505045 >> 172.21.15.45:0/1915505045 conn(0x561c085e3980 msgr2=0x561c085e4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8d17fa640 0 client.15777 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1915505045 shutdown_connections 2024-01-31T05:27:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8f2e0c740 1 -- 172.21.15.45:0/1915505045 wait complete. 2024-01-31T05:27:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.789+0000 7fb8f2e0c740 20 client.15777 trim_cache size 0 max 16384 2024-01-31T05:27:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:27:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:27:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:27:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:27:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.790+0000 7fb8eff6b640 1 -- 172.21.15.45:0/1331870473 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb8d40a1570 msgr2=0x7fb8d40a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.790+0000 7fb8eff6b640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb8d40a1570 0x7fb8d40a3a30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0004af0 tx=0x7fb8e0034440 comp rx=0 tx=0).stop 2024-01-31T05:27:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.790+0000 7fb8eff6b640 1 -- 172.21.15.45:0/1331870473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e8100800 msgr2=0x7fb8e81965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.790+0000 7fb8eff6b640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e8100800 0x7fb8e81965d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00d6d0 tx=0x7fb8dc00db90 comp rx=0 tx=0).stop 2024-01-31T05:27:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.790+0000 7fb8eff6b640 1 -- 172.21.15.45:0/1331870473 shutdown_connections 2024-01-31T05:27:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.790+0000 7fb8eff6b640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb8d40a1570 0x7fb8d40a3a30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.790+0000 7fb8eff6b640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e8100800 0x7fb8e81965d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.790+0000 7fb8eff6b640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8e8104b90 0x7fb8e819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.790+0000 7fb8eff6b640 1 --2- 172.21.15.45:0/1331870473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80ffe40 0x7fb8e8196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.790+0000 7fb8eff6b640 1 -- 172.21.15.45:0/1331870473 >> 172.21.15.45:0/1331870473 conn(0x7fb8e8076a40 msgr2=0x7fb8e80fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.790+0000 7fb8eff6b640 1 -- 172.21.15.45:0/1331870473 shutdown_connections 2024-01-31T05:27:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:23.790+0000 7fb8eff6b640 1 -- 172.21.15.45:0/1331870473 wait complete. 2024-01-31T05:27:24.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 464877 12.90 MB/sec execute 84 sec latency 94.198 ms 2024-01-31T05:27:24.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:24 smithi186 ceph-mon[35659]: pgmap v145: 97 pgs: 97 active+clean; 247 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 340 op/s 2024-01-31T05:27:24.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:24 smithi045 ceph-mon[31585]: pgmap v145: 97 pgs: 97 active+clean; 247 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 340 op/s 2024-01-31T05:27:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:24 smithi156 ceph-mon[35720]: pgmap v145: 97 pgs: 97 active+clean; 247 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 340 op/s 2024-01-31T05:27:25.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 466663 12.85 MB/sec execute 85 sec latency 95.954 ms 2024-01-31T05:27:26.780 INFO:tasks.workunit.client.0.smithi045.stdout: 1 469429 12.90 MB/sec execute 86 sec latency 26.488 ms 2024-01-31T05:27:26.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:26 smithi186 ceph-mon[35659]: pgmap v146: 97 pgs: 97 active+clean; 247 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 221 op/s 2024-01-31T05:27:26.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:26 smithi045 ceph-mon[31585]: pgmap v146: 97 pgs: 97 active+clean; 247 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 221 op/s 2024-01-31T05:27:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:26 smithi156 ceph-mon[35720]: pgmap v146: 97 pgs: 97 active+clean; 247 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 221 op/s 2024-01-31T05:27:27.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 472603 12.95 MB/sec execute 87 sec latency 7.050 ms 2024-01-31T05:27:28.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 475110 12.91 MB/sec execute 88 sec latency 8.231 ms 2024-01-31T05:27:28.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:28 smithi186 ceph-mon[35659]: pgmap v147: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 336 op/s 2024-01-31T05:27:28.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:28 smithi045 ceph-mon[31585]: pgmap v147: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 336 op/s 2024-01-31T05:27:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:28 smithi156 ceph-mon[35720]: pgmap v147: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 336 op/s 2024-01-31T05:27:29.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 477211 12.91 MB/sec execute 89 sec latency 57.389 ms 2024-01-31T05:27:30.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 479993 12.96 MB/sec execute 90 sec latency 71.564 ms 2024-01-31T05:27:30.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:30 smithi186 ceph-mon[35659]: pgmap v148: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 225 op/s 2024-01-31T05:27:30.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:30 smithi045 ceph-mon[31585]: pgmap v148: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 225 op/s 2024-01-31T05:27:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:30 smithi156 ceph-mon[35720]: pgmap v148: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 225 op/s 2024-01-31T05:27:31.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 482278 12.91 MB/sec execute 91 sec latency 66.749 ms 2024-01-31T05:27:32.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 484715 12.94 MB/sec execute 92 sec latency 7.165 ms 2024-01-31T05:27:32.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:32 smithi186 ceph-mon[35659]: pgmap v149: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 225 op/s 2024-01-31T05:27:32.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:32 smithi045 ceph-mon[31585]: pgmap v149: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 225 op/s 2024-01-31T05:27:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:32 smithi156 ceph-mon[35720]: pgmap v149: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 225 op/s 2024-01-31T05:27:33.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 487363 12.95 MB/sec execute 93 sec latency 7.345 ms 2024-01-31T05:27:34.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 490433 13.01 MB/sec execute 94 sec latency 27.237 ms 2024-01-31T05:27:34.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:34 smithi186 ceph-mon[35659]: pgmap v150: 97 pgs: 97 active+clean; 262 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 326 op/s 2024-01-31T05:27:34.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:34 smithi045 ceph-mon[31585]: pgmap v150: 97 pgs: 97 active+clean; 262 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 326 op/s 2024-01-31T05:27:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:34 smithi156 ceph-mon[35720]: pgmap v150: 97 pgs: 97 active+clean; 262 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 326 op/s 2024-01-31T05:27:35.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 492677 12.96 MB/sec execute 95 sec latency 73.180 ms 2024-01-31T05:27:35.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:35 smithi186 ceph-mon[35659]: evicting unresponsive client smithi045:x (15291), after 301.164 seconds 2024-01-31T05:27:35.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:35 smithi186 ceph-mon[35659]: Evicting (and blocklisting) client session 15291 (172.21.15.45:0/1441130788) 2024-01-31T05:27:35.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:35 smithi186 ceph-mon[35659]: from='mds.? [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384]' entity='mds.b' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.45:0/1441130788"}]: dispatch 2024-01-31T05:27:35.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:35 smithi186 ceph-mon[35659]: from='mds.? ' entity='mds.b' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.45:0/1441130788"}]: dispatch 2024-01-31T05:27:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:35 smithi045 ceph-mon[31585]: evicting unresponsive client smithi045:x (15291), after 301.164 seconds 2024-01-31T05:27:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:35 smithi045 ceph-mon[31585]: Evicting (and blocklisting) client session 15291 (172.21.15.45:0/1441130788) 2024-01-31T05:27:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:35 smithi045 ceph-mon[31585]: from='mds.? [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384]' entity='mds.b' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.45:0/1441130788"}]: dispatch 2024-01-31T05:27:35.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:35 smithi045 ceph-mon[31585]: from='mds.? ' entity='mds.b' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.45:0/1441130788"}]: dispatch 2024-01-31T05:27:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:35 smithi156 ceph-mon[35720]: evicting unresponsive client smithi045:x (15291), after 301.164 seconds 2024-01-31T05:27:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:35 smithi156 ceph-mon[35720]: Evicting (and blocklisting) client session 15291 (172.21.15.45:0/1441130788) 2024-01-31T05:27:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:35 smithi156 ceph-mon[35720]: from='mds.? [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384]' entity='mds.b' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.45:0/1441130788"}]: dispatch 2024-01-31T05:27:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:35 smithi156 ceph-mon[35720]: from='mds.? ' entity='mds.b' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.45:0/1441130788"}]: dispatch 2024-01-31T05:27:36.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 494403 12.95 MB/sec execute 96 sec latency 88.453 ms 2024-01-31T05:27:36.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:36 smithi186 ceph-mon[35659]: pgmap v151: 97 pgs: 97 active+clean; 262 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 215 op/s 2024-01-31T05:27:36.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:36 smithi186 ceph-mon[35659]: from='mds.? ' entity='mds.b' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.45:0/1441130788"}]': finished 2024-01-31T05:27:36.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:36 smithi186 ceph-mon[35659]: osdmap e91: 12 total, 12 up, 12 in 2024-01-31T05:27:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.886+0000 7f668651f640 1 -- 172.21.15.45:0/3556915844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6680101900 msgr2=0x7f6680105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.886+0000 7f668651f640 1 --2- 172.21.15.45:0/3556915844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6680101900 0x7f6680105940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6674009860 tx=0x7f667402bb70 comp rx=0 tx=0).stop 2024-01-31T05:27:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.886+0000 7f668651f640 1 -- 172.21.15.45:0/3556915844 shutdown_connections 2024-01-31T05:27:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.886+0000 7f668651f640 1 --2- 172.21.15.45:0/3556915844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6680106020 0x7f668010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.886+0000 7f668651f640 1 --2- 172.21.15.45:0/3556915844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6680101900 0x7f6680105940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.886+0000 7f668651f640 1 --2- 172.21.15.45:0/3556915844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680100f50 0x7f6680101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.886+0000 7f668651f640 1 -- 172.21.15.45:0/3556915844 >> 172.21.15.45:0/3556915844 conn(0x7f66800fc7b0 msgr2=0x7f66800febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.886+0000 7f668651f640 1 -- 172.21.15.45:0/3556915844 shutdown_connections 2024-01-31T05:27:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.886+0000 7f668651f640 1 -- 172.21.15.45:0/3556915844 wait complete. 2024-01-31T05:27:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.886+0000 7f668651f640 1 Processor -- start 2024-01-31T05:27:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f668651f640 1 -- start start 2024-01-31T05:27:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f668651f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6680100f50 0x7f6680072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f668651f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680106020 0x7f6680073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f668651f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6680073a70 0x7f66801a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f668651f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66801a5960 con 0x7f6680106020 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f668651f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66801a5ad0 con 0x7f6680073a70 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f668651f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66801a5c90 con 0x7f6680100f50 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f667f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680106020 0x7f6680073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f667ffff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6680100f50 0x7f6680072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f667f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680106020 0x7f6680073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39518/0 (socket says 172.21.15.45:39518) 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f667ffff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6680100f50 0x7f6680072ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:38046/0 (socket says 172.21.15.45:38046) 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f667f7fe640 1 -- 172.21.15.45:0/2226943320 learned_addr learned my addr 172.21.15.45:0/2226943320 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f6684abc640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6680073a70 0x7f66801a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f667f7fe640 1 -- 172.21.15.45:0/2226943320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6680100f50 msgr2=0x7f6680072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f667f7fe640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6680100f50 0x7f6680072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f667f7fe640 1 -- 172.21.15.45:0/2226943320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6680073a70 msgr2=0x7f66801a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f667f7fe640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6680073a70 0x7f66801a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.887+0000 7f667f7fe640 1 -- 172.21.15.45:0/2226943320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6670009560 con 0x7f6680106020 2024-01-31T05:27:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.888+0000 7f667ffff640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6680100f50 0x7f6680072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:27:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.888+0000 7f667f7fe640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680106020 0x7f6680073530 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6674037040 tx=0x7f66740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.888+0000 7f667d7fa640 1 -- 172.21.15.45:0/2226943320 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674034850 con 0x7f6680106020 2024-01-31T05:27:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.888+0000 7f668651f640 1 -- 172.21.15.45:0/2226943320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6678009840 con 0x7f6680106020 2024-01-31T05:27:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.888+0000 7f667d7fa640 1 -- 172.21.15.45:0/2226943320 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66740044e0 con 0x7f6680106020 2024-01-31T05:27:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.888+0000 7f667d7fa640 1 -- 172.21.15.45:0/2226943320 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674032c40 con 0x7f6680106020 2024-01-31T05:27:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.888+0000 7f668651f640 1 -- 172.21.15.45:0/2226943320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6674009490 con 0x7f6680106020 2024-01-31T05:27:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.891+0000 7f667d7fa640 1 -- 172.21.15.45:0/2226943320 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f667403b040 con 0x7f6680106020 2024-01-31T05:27:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.892+0000 7f667d7fa640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f66680a1570 0x7f66680a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.892+0000 7f667d7fa640 1 -- 172.21.15.45:0/2226943320 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f66740ee780 con 0x7f6680106020 2024-01-31T05:27:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.892+0000 7f66893b1740 1 Processor -- start 2024-01-31T05:27:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.892+0000 7f667ffff640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f66680a1570 0x7f66680a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.892+0000 7f66893b1740 1 -- start start 2024-01-31T05:27:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.892+0000 7f66893b1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff6699fb80 0x55ff6699ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.892+0000 7f66893b1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669a04a0 0x55ff669b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.892+0000 7f66893b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669b1f60 0x55ff669b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.892+0000 7f66893b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6690f040 con 0x55ff669b1f60 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.892+0000 7f66893b1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6690c730 con 0x55ff669a04a0 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.892+0000 7f66893b1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff66908020 con 0x55ff6699fb80 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f667f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669a04a0 0x55ff669b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f667ffff640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f66680a1570 0x7f66680a3a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6670009530 tx=0x7f6670009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f667ffff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff6699fb80 0x55ff6699ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f667f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669a04a0 0x55ff669b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:44232/0 (socket says 172.21.15.45:44232) 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f667f7fe640 1 -- 172.21.15.45:0/697102800 learned_addr learned my addr 172.21.15.45:0/697102800 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f667ffff640 1 --2- 172.21.15.45:0/697102800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff6699fb80 0x55ff6699ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:38048/0 (socket says 172.21.15.45:38048) 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f6684abc640 1 --2- 172.21.15.45:0/697102800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669b1f60 0x55ff669b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f667f7fe640 1 -- 172.21.15.45:0/697102800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff6699fb80 msgr2=0x55ff6699ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f667f7fe640 1 --2- 172.21.15.45:0/697102800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff6699fb80 0x55ff6699ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f667f7fe640 1 -- 172.21.15.45:0/697102800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669b1f60 msgr2=0x55ff669b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f667f7fe640 1 --2- 172.21.15.45:0/697102800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669b1f60 0x55ff669b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f667f7fe640 1 -- 172.21.15.45:0/697102800 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff668ffe40 con 0x55ff669a04a0 2024-01-31T05:27:36.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:36 smithi045 ceph-mon[31585]: pgmap v151: 97 pgs: 97 active+clean; 262 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 215 op/s 2024-01-31T05:27:36.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:36 smithi045 ceph-mon[31585]: from='mds.? ' entity='mds.b' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.45:0/1441130788"}]': finished 2024-01-31T05:27:36.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:36 smithi045 ceph-mon[31585]: osdmap e91: 12 total, 12 up, 12 in 2024-01-31T05:27:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f667f7fe640 1 --2- 172.21.15.45:0/697102800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669a04a0 0x55ff669b1a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6674005cf0 tx=0x7f66740bb4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99bb5383cbe51964 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f66667fc640 1 -- 172.21.15.45:0/697102800 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66740bb650 con 0x55ff669a04a0 2024-01-31T05:27:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f66667fc640 1 -- 172.21.15.45:0/697102800 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66740bf7a0 con 0x55ff669a04a0 2024-01-31T05:27:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.893+0000 7f66667fc640 1 -- 172.21.15.45:0/697102800 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66740b7d60 con 0x55ff669a04a0 2024-01-31T05:27:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.894+0000 7f66893b1740 1 -- 172.21.15.45:0/697102800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669a04a0 msgr2=0x55ff669b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.894+0000 7f66893b1740 1 --2- 172.21.15.45:0/697102800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669a04a0 0x55ff669b1a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6674005cf0 tx=0x7f66740bb4e0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.894+0000 7f66893b1740 1 -- 172.21.15.45:0/697102800 shutdown_connections 2024-01-31T05:27:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.894+0000 7f66893b1740 1 --2- 172.21.15.45:0/697102800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669b1f60 0x55ff669b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.894+0000 7f66893b1740 1 --2- 172.21.15.45:0/697102800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669a04a0 0x55ff669b1a20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.894+0000 7f66893b1740 1 --2- 172.21.15.45:0/697102800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff6699fb80 0x55ff6699ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.894+0000 7f66893b1740 1 -- 172.21.15.45:0/697102800 >> 172.21.15.45:0/697102800 conn(0x55ff6699a980 msgr2=0x55ff6699ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.895+0000 7f66893b1740 1 -- 172.21.15.45:0/697102800 shutdown_connections 2024-01-31T05:27:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.895+0000 7f66893b1740 1 -- 172.21.15.45:0/697102800 wait complete. 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.895+0000 7f66893b1740 1 -- start start 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.895+0000 7f66893b1740 1 Processor -- start 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.895+0000 7f66893b1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff6699fb80 0x55ff669aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.895+0000 7f66893b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 0x55ff669af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.895+0000 7f66893b1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff669b1f60 0x55ff669af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.895+0000 7f66893b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff668c11b0 con 0x55ff669a04a0 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.895+0000 7f66893b1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff668c0e50 con 0x55ff6699fb80 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.895+0000 7f66893b1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff66886090 con 0x55ff669b1f60 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f66893b1740 10 client.0 _ll_register_callbacks cb 0x55ff668aba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f667ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff6699fb80 0x55ff669aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f667f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 0x55ff669af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f667ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff6699fb80 0x55ff669aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:44236/0 (socket says 172.21.15.45:44236) 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f667ffff640 1 -- 172.21.15.45:0/469408355 learned_addr learned my addr 172.21.15.45:0/469408355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f6684abc640 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff669b1f60 0x55ff669af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f667ffff640 1 -- 172.21.15.45:0/469408355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff669b1f60 msgr2=0x55ff669af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f667ffff640 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff669b1f60 0x55ff669af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f667ffff640 1 -- 172.21.15.45:0/469408355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 msgr2=0x55ff669af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f667ffff640 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 0x55ff669af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f667ffff640 1 -- 172.21.15.45:0/469408355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6674009490 con 0x55ff6699fb80 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f6684abc640 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff669b1f60 0x55ff669af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f667f7fe640 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 0x55ff669af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f667ffff640 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff6699fb80 0x55ff669aee60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f667000fd90 tx=0x7f6670013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8713c0aa2777481 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f665f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f665f7fe640 1 -- 172.21.15.45:0/469408355 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670021040 con 0x55ff6699fb80 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f665f7fe640 1 -- 172.21.15.45:0/469408355 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f66700275f0 con 0x55ff6699fb80 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.896+0000 7f66893b1740 1 -- 172.21.15.45:0/469408355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff66901010 con 0x55ff6699fb80 2024-01-31T05:27:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.897+0000 7f665f7fe640 1 -- 172.21.15.45:0/469408355 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66700277b0 con 0x55ff6699fb80 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.897+0000 7f66893b1740 10 client.25310 fetch_fsmap learned FSMap version 26 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.897+0000 7f66893b1740 1 -- 172.21.15.45:0/469408355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ff669066a0 con 0x55ff6699fb80 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.897+0000 7f665f7fe640 1 -- 172.21.15.45:0/469408355 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6670034020 con 0x55ff6699fb80 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.897+0000 7f665f7fe640 1 -- 172.21.15.45:0/469408355 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f667000e900 con 0x55ff6699fb80 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.897+0000 7f66893b1740 10 client.25310 fetch_fsmap finished waiting for FSMap version 26 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.897+0000 7f66893b1740 10 client.25310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.897+0000 7f66893b1740 20 client.25310 populate_metadata read hostname 'smithi045' 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.897+0000 7f66893b1740 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ff669c1a00 0x55ff669c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.897+0000 7f66893b1740 4 client.25310 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.897+0000 7f66893b1740 1 -- 172.21.15.45:0/469408355 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff66900d30 con 0x55ff669c1a00 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.897+0000 7f667f7fe640 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ff669c1a00 0x55ff669c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.898+0000 7f665f7fe640 10 client.25310 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.898+0000 7f667f7fe640 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ff669c1a00 0x55ff669c3e20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850daacc745c58b server_cookie=68651421e688b6c in_seq=0 out_seq=0 2024-01-31T05:27:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.898+0000 7f665f7fe640 1 -- 172.21.15.45:0/469408355 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff66900d30 con 0x55ff669c1a00 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.898+0000 7f665f7fe640 10 client.25310 handle_command_reply: tid=0 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.898+0000 7f66893b1740 1 client.25310 shutdown 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.899+0000 7f66893b1740 1 -- 172.21.15.45:0/469408355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff6699fb80 msgr2=0x55ff669aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.899+0000 7f66893b1740 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff6699fb80 0x55ff669aee60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f667000fd90 tx=0x7f6670013040 comp rx=0 tx=0).stop 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.899+0000 7f66893b1740 1 -- 172.21.15.45:0/469408355 shutdown_connections 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.899+0000 7f66893b1740 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ff669c1a00 0x55ff669c3e20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.899+0000 7f66893b1740 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 0x55ff669af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.899+0000 7f66893b1740 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff669b1f60 0x55ff669af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.899+0000 7f665f7fe640 0 client.25310 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.899+0000 7f66893b1740 1 --2- 172.21.15.45:0/469408355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff6699fb80 0x55ff669aee60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.899+0000 7f66893b1740 1 -- 172.21.15.45:0/469408355 >> 172.21.15.45:0/469408355 conn(0x55ff6699a980 msgr2=0x55ff6699c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.899+0000 7f66893b1740 1 -- 172.21.15.45:0/469408355 shutdown_connections 2024-01-31T05:27:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.899+0000 7f66893b1740 1 -- 172.21.15.45:0/469408355 wait complete. 2024-01-31T05:27:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.899+0000 7f66893b1740 20 client.25310 trim_cache size 0 max 16384 2024-01-31T05:27:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.913+0000 7f66893b1740 1 Processor -- start 2024-01-31T05:27:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f66893b1740 1 -- start start 2024-01-31T05:27:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f66893b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff6699fb80 0x55ff669af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f66893b1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff669a04a0 0x55ff669ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f66893b1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669b1f60 0x55ff669ac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f66893b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6699a1c0 con 0x55ff6699fb80 2024-01-31T05:27:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f66893b1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6699f9e0 con 0x55ff669b1f60 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f66893b1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff669b1930 con 0x55ff669a04a0 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f667ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff6699fb80 0x55ff669af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f667ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff6699fb80 0x55ff669af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39552/0 (socket says 172.21.15.45:39552) 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f667ffff640 1 -- 172.21.15.45:0/3886096919 learned_addr learned my addr 172.21.15.45:0/3886096919 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f667f7fe640 1 --2- 172.21.15.45:0/3886096919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff669a04a0 0x55ff669ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f6684abc640 1 --2- 172.21.15.45:0/3886096919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669b1f60 0x55ff669ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f667ffff640 1 -- 172.21.15.45:0/3886096919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff669a04a0 msgr2=0x55ff669ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f667ffff640 1 --2- 172.21.15.45:0/3886096919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff669a04a0 0x55ff669ac0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f667ffff640 1 -- 172.21.15.45:0/3886096919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669b1f60 msgr2=0x55ff669ac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f667ffff640 1 --2- 172.21.15.45:0/3886096919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669b1f60 0x55ff669ac620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f667ffff640 1 -- 172.21.15.45:0/3886096919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff668a3030 con 0x55ff6699fb80 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f667f7fe640 1 --2- 172.21.15.45:0/3886096919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff669a04a0 0x55ff669ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.914+0000 7f667ffff640 1 --2- 172.21.15.45:0/3886096919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff6699fb80 0x55ff669af700 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6670016040 tx=0x7f667000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d40ea750cf723b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.915+0000 7f6665ffb640 1 -- 172.21.15.45:0/3886096919 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667000e5b0 con 0x55ff6699fb80 2024-01-31T05:27:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.915+0000 7f6665ffb640 1 -- 172.21.15.45:0/3886096919 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6670034040 con 0x55ff6699fb80 2024-01-31T05:27:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.915+0000 7f6665ffb640 1 -- 172.21.15.45:0/3886096919 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670004680 con 0x55ff6699fb80 2024-01-31T05:27:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.915+0000 7f66893b1740 1 -- 172.21.15.45:0/3886096919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff6699fb80 msgr2=0x55ff669af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.915+0000 7f66893b1740 1 --2- 172.21.15.45:0/3886096919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff6699fb80 0x55ff669af700 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6670016040 tx=0x7f667000eed0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.915+0000 7f66893b1740 1 -- 172.21.15.45:0/3886096919 shutdown_connections 2024-01-31T05:27:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.915+0000 7f66893b1740 1 --2- 172.21.15.45:0/3886096919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669b1f60 0x55ff669ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.915+0000 7f66893b1740 1 --2- 172.21.15.45:0/3886096919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff669a04a0 0x55ff669ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.915+0000 7f66893b1740 1 --2- 172.21.15.45:0/3886096919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff6699fb80 0x55ff669af700 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.915+0000 7f66893b1740 1 -- 172.21.15.45:0/3886096919 >> 172.21.15.45:0/3886096919 conn(0x55ff6699a980 msgr2=0x55ff669aa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f66893b1740 1 -- 172.21.15.45:0/3886096919 shutdown_connections 2024-01-31T05:27:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f66893b1740 1 -- 172.21.15.45:0/3886096919 wait complete. 2024-01-31T05:27:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f66893b1740 1 -- start start 2024-01-31T05:27:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f66893b1740 1 Processor -- start 2024-01-31T05:27:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f66893b1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff6699fb80 0x55ff669b8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f66893b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 0x55ff669b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f66893b1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669b1f60 0x55ff669b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f66893b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff669b7550 con 0x55ff669a04a0 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f66893b1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff669b76c0 con 0x55ff669b1f60 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f66893b1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff669b7880 con 0x55ff6699fb80 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f66893b1740 10 client.0 _ll_register_callbacks cb 0x7f667402cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f667f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 0x55ff669b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f667f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 0x55ff669b8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39558/0 (socket says 172.21.15.45:39558) 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f667ffff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff6699fb80 0x55ff669b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f667f7fe640 1 -- 172.21.15.45:0/3554018700 learned_addr learned my addr 172.21.15.45:0/3554018700 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.916+0000 7f6684abc640 1 --2- 172.21.15.45:0/3554018700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669b1f60 0x55ff669b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f667f7fe640 1 -- 172.21.15.45:0/3554018700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff6699fb80 msgr2=0x55ff669b8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f667f7fe640 1 --2- 172.21.15.45:0/3554018700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff6699fb80 0x55ff669b8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f667f7fe640 1 -- 172.21.15.45:0/3554018700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669b1f60 msgr2=0x55ff669b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f667f7fe640 1 --2- 172.21.15.45:0/3554018700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669b1f60 0x55ff669b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f667f7fe640 1 -- 172.21.15.45:0/3554018700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff668ffe40 con 0x55ff669a04a0 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f667f7fe640 1 --2- 172.21.15.45:0/3554018700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 0x55ff669b8e40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f66740b8070 tx=0x7f6674032600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2c1dc0adfc2361 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f665f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f665f7fe640 1 -- 172.21.15.45:0/3554018700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66740b99f0 con 0x55ff669a04a0 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f665f7fe640 1 -- 172.21.15.45:0/3554018700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f66740bab40 con 0x55ff669a04a0 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f66893b1740 1 -- 172.21.15.45:0/3554018700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66740bf7a0 con 0x55ff669a04a0 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f665f7fe640 1 -- 172.21.15.45:0/3554018700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66740bad00 con 0x55ff669a04a0 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f66893b1740 10 client.15795 fetch_fsmap learned FSMap version 26 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f66893b1740 1 -- 172.21.15.45:0/3554018700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ff669066a0 con 0x55ff669a04a0 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f665f7fe640 1 -- 172.21.15.45:0/3554018700 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66740bf7a0 con 0x55ff669a04a0 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.917+0000 7f665f7fe640 1 -- 172.21.15.45:0/3554018700 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f66740ba6b0 con 0x55ff669a04a0 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.918+0000 7f66893b1740 10 client.15795 fetch_fsmap finished waiting for FSMap version 26 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.918+0000 7f66893b1740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.918+0000 7f66893b1740 20 client.15795 populate_metadata read hostname 'smithi045' 2024-01-31T05:27:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.918+0000 7f66893b1740 1 --2- 172.21.15.45:0/3554018700 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ff66a18f90 0x55ff66a1b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.918+0000 7f66893b1740 4 client.15795 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:27:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.918+0000 7f66893b1740 1 -- 172.21.15.45:0/3554018700 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff66607600 con 0x55ff66a18f90 2024-01-31T05:27:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.918+0000 7f667ffff640 1 --2- 172.21.15.45:0/3554018700 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ff66a18f90 0x55ff66a1b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.918+0000 7f665f7fe640 10 client.15795 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.918+0000 7f667ffff640 1 --2- 172.21.15.45:0/3554018700 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ff66a18f90 0x55ff66a1b3b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33036d8d8045f41 server_cookie=d1eb292184096fff in_seq=0 out_seq=0 2024-01-31T05:27:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f665f7fe640 1 -- 172.21.15.45:0/3554018700 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff66607600 con 0x55ff66a18f90 2024-01-31T05:27:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f665f7fe640 10 client.15795 handle_command_reply: tid=0 2024-01-31T05:27:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f66893b1740 1 client.15795 shutdown 2024-01-31T05:27:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f66893b1740 1 -- 172.21.15.45:0/3554018700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 msgr2=0x55ff669b8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f66893b1740 1 --2- 172.21.15.45:0/3554018700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 0x55ff669b8e40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f66740b8070 tx=0x7f6674032600 comp rx=0 tx=0).stop 2024-01-31T05:27:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f66893b1740 1 -- 172.21.15.45:0/3554018700 shutdown_connections 2024-01-31T05:27:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f66893b1740 1 --2- 172.21.15.45:0/3554018700 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ff66a18f90 0x55ff66a1b3b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f66893b1740 1 --2- 172.21.15.45:0/3554018700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff669a04a0 0x55ff669b8e40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f66893b1740 1 --2- 172.21.15.45:0/3554018700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff669b1f60 0x55ff669b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f66893b1740 1 --2- 172.21.15.45:0/3554018700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff6699fb80 0x55ff669b8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f665f7fe640 0 client.15795 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f66893b1740 1 -- 172.21.15.45:0/3554018700 >> 172.21.15.45:0/3554018700 conn(0x55ff6699a980 msgr2=0x55ff669af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f66893b1740 1 -- 172.21.15.45:0/3554018700 shutdown_connections 2024-01-31T05:27:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f66893b1740 1 -- 172.21.15.45:0/3554018700 wait complete. 2024-01-31T05:27:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.928+0000 7f66893b1740 20 client.15795 trim_cache size 0 max 16384 2024-01-31T05:27:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:27:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:27:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:27:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:27:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.929+0000 7f668651f640 1 -- 172.21.15.45:0/2226943320 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f66680a1570 msgr2=0x7f66680a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.929+0000 7f668651f640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f66680a1570 0x7f66680a3a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6670009530 tx=0x7f6670009260 comp rx=0 tx=0).stop 2024-01-31T05:27:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.929+0000 7f668651f640 1 -- 172.21.15.45:0/2226943320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680106020 msgr2=0x7f6680073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.929+0000 7f668651f640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680106020 0x7f6680073530 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6674037040 tx=0x7f66740042f0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.929+0000 7f668651f640 1 -- 172.21.15.45:0/2226943320 shutdown_connections 2024-01-31T05:27:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.929+0000 7f668651f640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f66680a1570 0x7f66680a3a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.929+0000 7f668651f640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680106020 0x7f6680073530 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.929+0000 7f668651f640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6680073a70 0x7f66801a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.929+0000 7f668651f640 1 --2- 172.21.15.45:0/2226943320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6680100f50 0x7f6680072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.929+0000 7f668651f640 1 -- 172.21.15.45:0/2226943320 >> 172.21.15.45:0/2226943320 conn(0x7f66800fc7b0 msgr2=0x7f66800feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.929+0000 7f668651f640 1 -- 172.21.15.45:0/2226943320 shutdown_connections 2024-01-31T05:27:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:36.929+0000 7f668651f640 1 -- 172.21.15.45:0/2226943320 wait complete. 2024-01-31T05:27:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:36 smithi156 ceph-mon[35720]: pgmap v151: 97 pgs: 97 active+clean; 262 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 215 op/s 2024-01-31T05:27:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:36 smithi156 ceph-mon[35720]: from='mds.? ' entity='mds.b' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.45:0/1441130788"}]': finished 2024-01-31T05:27:36.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:36 smithi156 ceph-mon[35720]: osdmap e91: 12 total, 12 up, 12 in 2024-01-31T05:27:37.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 497479 13.00 MB/sec execute 97 sec latency 7.485 ms 2024-01-31T05:27:38.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 499831 12.95 MB/sec execute 98 sec latency 7.356 ms 2024-01-31T05:27:38.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:38 smithi186 ceph-mon[35659]: pgmap v153: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-01-31T05:27:38.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:38 smithi045 ceph-mon[31585]: pgmap v153: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-01-31T05:27:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:38 smithi156 ceph-mon[35720]: pgmap v153: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-01-31T05:27:39.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 501915 12.95 MB/sec execute 99 sec latency 15.933 ms 2024-01-31T05:27:39.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:39 smithi045 ceph-mon[31585]: pgmap v154: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-01-31T05:27:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:39 smithi156 ceph-mon[35720]: pgmap v154: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-01-31T05:27:40.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:39 smithi186 ceph-mon[35659]: pgmap v154: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-01-31T05:27:40.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 504810 13.00 MB/sec execute 100 sec latency 32.818 ms 2024-01-31T05:27:41.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 506868 12.94 MB/sec execute 101 sec latency 137.762 ms 2024-01-31T05:27:42.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 509035 12.95 MB/sec execute 102 sec latency 21.641 ms 2024-01-31T05:27:42.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:42 smithi186 ceph-mon[35659]: pgmap v155: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-01-31T05:27:42.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:42 smithi045 ceph-mon[31585]: pgmap v155: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-01-31T05:27:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:42 smithi156 ceph-mon[35720]: pgmap v155: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-01-31T05:27:43.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 511981 13.00 MB/sec execute 103 sec latency 23.032 ms 2024-01-31T05:27:44.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 515115 13.04 MB/sec execute 104 sec latency 13.597 ms 2024-01-31T05:27:44.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:44 smithi186 ceph-mon[35659]: pgmap v156: 97 pgs: 97 active+clean; 275 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-01-31T05:27:44.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:44 smithi045 ceph-mon[31585]: pgmap v156: 97 pgs: 97 active+clean; 275 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-01-31T05:27:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:44 smithi156 ceph-mon[35720]: pgmap v156: 97 pgs: 97 active+clean; 275 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-01-31T05:27:45.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 517574 13.00 MB/sec execute 105 sec latency 15.738 ms 2024-01-31T05:27:46.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 519146 12.99 MB/sec execute 106 sec latency 116.791 ms 2024-01-31T05:27:46.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:46 smithi186 ceph-mon[35659]: pgmap v157: 97 pgs: 97 active+clean; 275 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-01-31T05:27:46.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:46 smithi045 ceph-mon[31585]: pgmap v157: 97 pgs: 97 active+clean; 275 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-01-31T05:27:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:46 smithi156 ceph-mon[35720]: pgmap v157: 97 pgs: 97 active+clean; 275 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-01-31T05:27:47.781 INFO:tasks.workunit.client.0.smithi045.stdout: 1 522354 13.03 MB/sec execute 107 sec latency 11.430 ms 2024-01-31T05:27:48.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 524823 13.00 MB/sec execute 108 sec latency 7.938 ms 2024-01-31T05:27:48.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:48 smithi186 ceph-mon[35659]: pgmap v158: 97 pgs: 97 active+clean; 283 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 324 op/s 2024-01-31T05:27:48.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:48 smithi045 ceph-mon[31585]: pgmap v158: 97 pgs: 97 active+clean; 283 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 324 op/s 2024-01-31T05:27:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:48 smithi156 ceph-mon[35720]: pgmap v158: 97 pgs: 97 active+clean; 283 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 324 op/s 2024-01-31T05:27:49.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 527294 13.03 MB/sec execute 109 sec latency 6.093 ms 2024-01-31T05:27:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.023+0000 7fe56233b640 1 -- 172.21.15.45:0/4003552211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c106e00 msgr2=0x7fe55c1071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.023+0000 7fe56233b640 1 --2- 172.21.15.45:0/4003552211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c106e00 0x7fe55c1071e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009900 tx=0x7fe54c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:27:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.023+0000 7fe56233b640 1 -- 172.21.15.45:0/4003552211 shutdown_connections 2024-01-31T05:27:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.023+0000 7fe56233b640 1 --2- 172.21.15.45:0/4003552211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe55c068d10 0x7fe55c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.023+0000 7fe56233b640 1 --2- 172.21.15.45:0/4003552211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe55c068370 0x7fe55c0687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.023+0000 7fe56233b640 1 --2- 172.21.15.45:0/4003552211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c106e00 0x7fe55c1071e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.023+0000 7fe56233b640 1 -- 172.21.15.45:0/4003552211 >> 172.21.15.45:0/4003552211 conn(0x7fe55c076a40 msgr2=0x7fe55c076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.023+0000 7fe56233b640 1 -- 172.21.15.45:0/4003552211 shutdown_connections 2024-01-31T05:27:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.023+0000 7fe56233b640 1 -- 172.21.15.45:0/4003552211 wait complete. 2024-01-31T05:27:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe56233b640 1 Processor -- start 2024-01-31T05:27:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe56233b640 1 -- start start 2024-01-31T05:27:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe56233b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe55c068370 0x7fe55c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe56233b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c068d10 0x7fe55c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe56233b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe55c106e00 0x7fe55c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe56233b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c19c050 con 0x7fe55c068d10 2024-01-31T05:27:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe56233b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c19c1c0 con 0x7fe55c068370 2024-01-31T05:27:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe56233b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c19c380 con 0x7fe55c106e00 2024-01-31T05:27:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c068d10 0x7fe55c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c068d10 0x7fe55c197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50962/0 (socket says 172.21.15.45:50962) 2024-01-31T05:27:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe55b7fe640 1 -- 172.21.15.45:0/2636267023 learned_addr learned my addr 172.21.15.45:0/2636267023 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe5608d8640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe55c106e00 0x7fe55c19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.024+0000 7fe55bfff640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe55c068370 0x7fe55c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.025+0000 7fe55b7fe640 1 -- 172.21.15.45:0/2636267023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe55c106e00 msgr2=0x7fe55c19ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.025+0000 7fe55b7fe640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe55c106e00 0x7fe55c19ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.025+0000 7fe55b7fe640 1 -- 172.21.15.45:0/2636267023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe55c068370 msgr2=0x7fe55c1971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.025+0000 7fe55b7fe640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe55c068370 0x7fe55c1971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.025+0000 7fe55b7fe640 1 -- 172.21.15.45:0/2636267023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c009560 con 0x7fe55c068d10 2024-01-31T05:27:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.025+0000 7fe55b7fe640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c068d10 0x7fe55c197720 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe55000e9f0 tx=0x7fe55000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.025+0000 7fe5597fa640 1 -- 172.21.15.45:0/2636267023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550009720 con 0x7fe55c068d10 2024-01-31T05:27:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.025+0000 7fe5597fa640 1 -- 172.21.15.45:0/2636267023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe55000cd80 con 0x7fe55c068d10 2024-01-31T05:27:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.025+0000 7fe56233b640 1 -- 172.21.15.45:0/2636267023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c19c660 con 0x7fe55c068d10 2024-01-31T05:27:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.025+0000 7fe5597fa640 1 -- 172.21.15.45:0/2636267023 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550010430 con 0x7fe55c068d10 2024-01-31T05:27:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.025+0000 7fe56233b640 1 -- 172.21.15.45:0/2636267023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c1a3ee0 con 0x7fe55c068d10 2024-01-31T05:27:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.027+0000 7fe5597fa640 1 -- 172.21.15.45:0/2636267023 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fe550005020 con 0x7fe55c068d10 2024-01-31T05:27:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5651d5740 1 Processor -- start 2024-01-31T05:27:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5651d5740 1 -- start start 2024-01-31T05:27:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5597fa640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe5440a1570 0x7fe5440a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5651d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f62060 0x556770f62440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5651d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62980 0x556770f6b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5651d5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f6b750 0x556770f6db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5651d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556770ed1040 con 0x556770f62060 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5651d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556770ece730 con 0x556770f62980 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5651d5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556770eca020 con 0x556770f6b750 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5608d8640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe5440a1570 0x7fe5440a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5597fa640 1 -- 172.21.15.45:0/2636267023 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe550014070 con 0x7fe55c068d10 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe55bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f62060 0x556770f62440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe55bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f62060 0x556770f62440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50970/0 (socket says 172.21.15.45:50970) 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe55bfff640 1 -- 172.21.15.45:0/138527468 learned_addr learned my addr 172.21.15.45:0/138527468 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe55b7fe640 1 --2- 172.21.15.45:0/138527468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62980 0x556770f6b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5608d8640 1 --2- 172.21.15.45:0/138527468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f6b750 0x556770f6db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe55bfff640 1 -- 172.21.15.45:0/138527468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f6b750 msgr2=0x556770f6db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe55bfff640 1 --2- 172.21.15.45:0/138527468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f6b750 0x556770f6db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe55bfff640 1 -- 172.21.15.45:0/138527468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62980 msgr2=0x556770f6b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe55bfff640 1 --2- 172.21.15.45:0/138527468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62980 0x556770f6b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe55bfff640 1 -- 172.21.15.45:0/138527468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556770ec1e40 con 0x556770f62060 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5608d8640 1 --2- 172.21.15.45:0/138527468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f6b750 0x556770f6db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.028+0000 7fe5608d8640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe5440a1570 0x7fe5440a3a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe554005fd0 tx=0x7fe554014b50 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.029+0000 7fe55bfff640 1 --2- 172.21.15.45:0/138527468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f62060 0x556770f62440 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe54c000c00 tx=0x7fe54c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cbd7e22867c288 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.029+0000 7fe5427fc640 1 -- 172.21.15.45:0/138527468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0048f0 con 0x556770f62060 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.029+0000 7fe5427fc640 1 -- 172.21.15.45:0/138527468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe54c004a50 con 0x556770f62060 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.029+0000 7fe5427fc640 1 -- 172.21.15.45:0/138527468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c003830 con 0x556770f62060 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.030+0000 7fe5651d5740 1 -- 172.21.15.45:0/138527468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f62060 msgr2=0x556770f62440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.030+0000 7fe5651d5740 1 --2- 172.21.15.45:0/138527468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f62060 0x556770f62440 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe54c000c00 tx=0x7fe54c0046c0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.030+0000 7fe5651d5740 1 -- 172.21.15.45:0/138527468 shutdown_connections 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.030+0000 7fe5651d5740 1 --2- 172.21.15.45:0/138527468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f6b750 0x556770f6db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.030+0000 7fe5651d5740 1 --2- 172.21.15.45:0/138527468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62980 0x556770f6b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.030+0000 7fe5651d5740 1 --2- 172.21.15.45:0/138527468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f62060 0x556770f62440 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.030+0000 7fe5651d5740 1 -- 172.21.15.45:0/138527468 >> 172.21.15.45:0/138527468 conn(0x556770f5c980 msgr2=0x556770f5cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.030+0000 7fe5651d5740 1 -- 172.21.15.45:0/138527468 shutdown_connections 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.030+0000 7fe5651d5740 1 -- 172.21.15.45:0/138527468 wait complete. 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.030+0000 7fe5651d5740 1 -- start start 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.030+0000 7fe5651d5740 1 Processor -- start 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5651d5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 0x556770f68650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5651d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62980 0x556770f68b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5651d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 0x556770f690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5651d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556770e831b0 con 0x556770f6b750 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5651d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556770e82e50 con 0x556770f62980 2024-01-31T05:27:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5651d5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556770e48090 con 0x556770f62060 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5608d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 0x556770f690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5651d5740 10 client.0 _ll_register_callbacks cb 0x556770e6da90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5608d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 0x556770f690d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50986/0 (socket says 172.21.15.45:50986) 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5608d8640 1 -- 172.21.15.45:0/623960527 learned_addr learned my addr 172.21.15.45:0/623960527 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe55bfff640 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 0x556770f68650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe55b7fe640 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62980 0x556770f68b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5608d8640 1 -- 172.21.15.45:0/623960527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 msgr2=0x556770f68650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5608d8640 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 0x556770f68650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5608d8640 1 -- 172.21.15.45:0/623960527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62980 msgr2=0x556770f68b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5608d8640 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62980 0x556770f68b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5608d8640 1 -- 172.21.15.45:0/623960527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c10e480 con 0x556770f6b750 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe55b7fe640 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62980 0x556770f68b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:27:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe55bfff640 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 0x556770f68650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:27:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe5608d8640 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 0x556770f690d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe554010be0 tx=0x7fe554004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb42015107c43565 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.031+0000 7fe53b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:27:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe53b7fe640 1 -- 172.21.15.45:0/623960527 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554003970 con 0x556770f6b750 2024-01-31T05:27:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe53b7fe640 1 -- 172.21.15.45:0/623960527 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe554003ad0 con 0x556770f6b750 2024-01-31T05:27:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe5651d5740 1 -- 172.21.15.45:0/623960527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556770ec3010 con 0x556770f6b750 2024-01-31T05:27:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe53b7fe640 1 -- 172.21.15.45:0/623960527 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe554003cd0 con 0x556770f6b750 2024-01-31T05:27:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe5651d5740 10 client.15807 fetch_fsmap learned FSMap version 26 2024-01-31T05:27:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe5651d5740 1 -- 172.21.15.45:0/623960527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556770ec86a0 con 0x556770f6b750 2024-01-31T05:27:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe53b7fe640 1 -- 172.21.15.45:0/623960527 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556770ec3010 con 0x556770f6b750 2024-01-31T05:27:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe53b7fe640 1 -- 172.21.15.45:0/623960527 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe5540205d0 con 0x556770f6b750 2024-01-31T05:27:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe5651d5740 10 client.15807 fetch_fsmap finished waiting for FSMap version 26 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe5651d5740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe5651d5740 20 client.15807 populate_metadata read hostname 'smithi045' 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe5651d5740 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556770f7f8d0 0x556770f81cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe5651d5740 4 client.15807 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.032+0000 7fe5651d5740 1 -- 172.21.15.45:0/623960527 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556770ec2d30 con 0x556770f7f8d0 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.033+0000 7fe55bfff640 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556770f7f8d0 0x556770f81cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.033+0000 7fe55bfff640 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556770f7f8d0 0x556770f81cf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e85be468b075402f server_cookie=691ed82a2b89ae36 in_seq=0 out_seq=0 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.033+0000 7fe53b7fe640 10 client.15807 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe53b7fe640 1 -- 172.21.15.45:0/623960527 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556770ec2d30 con 0x556770f7f8d0 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe53b7fe640 10 client.15807 handle_command_reply: tid=0 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe5651d5740 1 client.15807 shutdown 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe5651d5740 1 -- 172.21.15.45:0/623960527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 msgr2=0x556770f690d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe5651d5740 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 0x556770f690d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe554010be0 tx=0x7fe554004ab0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe5651d5740 1 -- 172.21.15.45:0/623960527 shutdown_connections 2024-01-31T05:27:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe5651d5740 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556770f7f8d0 0x556770f81cf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe5651d5740 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 0x556770f690d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe53b7fe640 0 client.15807 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe5651d5740 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62980 0x556770f68b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe5651d5740 1 --2- 172.21.15.45:0/623960527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 0x556770f68650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe5651d5740 1 -- 172.21.15.45:0/623960527 >> 172.21.15.45:0/623960527 conn(0x556770f5c980 msgr2=0x556770f5e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe5651d5740 1 -- 172.21.15.45:0/623960527 shutdown_connections 2024-01-31T05:27:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe5651d5740 1 -- 172.21.15.45:0/623960527 wait complete. 2024-01-31T05:27:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.034+0000 7fe5651d5740 20 client.15807 trim_cache size 0 max 16384 2024-01-31T05:27:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.049+0000 7fe5651d5740 1 Processor -- start 2024-01-31T05:27:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.049+0000 7fe5651d5740 1 -- start start 2024-01-31T05:27:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe5651d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62060 0x556770f6ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe5651d5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62980 0x556770f68ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe5651d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 0x556770f67630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe5651d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556770f86ce0 con 0x556770f6b750 2024-01-31T05:27:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe5651d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556770ec2d60 con 0x556770f62060 2024-01-31T05:27:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe5651d5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556770f58080 con 0x556770f62980 2024-01-31T05:27:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe5608d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 0x556770f67630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62980 0x556770f68ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe5608d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 0x556770f67630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50994/0 (socket says 172.21.15.45:50994) 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe55bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62060 0x556770f6ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe5608d8640 1 -- 172.21.15.45:0/3665191734 learned_addr learned my addr 172.21.15.45:0/3665191734 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe55b7fe640 1 -- 172.21.15.45:0/3665191734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62060 msgr2=0x556770f6ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe55b7fe640 1 --2- 172.21.15.45:0/3665191734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62060 0x556770f6ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe55b7fe640 1 -- 172.21.15.45:0/3665191734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 msgr2=0x556770f67630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe55b7fe640 1 --2- 172.21.15.45:0/3665191734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 0x556770f67630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe55b7fe640 1 -- 172.21.15.45:0/3665191734 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556770e65030 con 0x556770f62980 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe55b7fe640 1 --2- 172.21.15.45:0/3665191734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62980 0x556770f68ef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5500ca970 tx=0x7fe550099ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb9821b5136096 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe541ffb640 1 -- 172.21.15.45:0/3665191734 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5500cd050 con 0x556770f62980 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe541ffb640 1 -- 172.21.15.45:0/3665191734 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe55009a020 con 0x556770f62980 2024-01-31T05:27:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.050+0000 7fe541ffb640 1 -- 172.21.15.45:0/3665191734 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55005c420 con 0x556770f62980 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.051+0000 7fe5651d5740 1 -- 172.21.15.45:0/3665191734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62980 msgr2=0x556770f68ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.051+0000 7fe5651d5740 1 --2- 172.21.15.45:0/3665191734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62980 0x556770f68ef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5500ca970 tx=0x7fe550099ab0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.051+0000 7fe5651d5740 1 -- 172.21.15.45:0/3665191734 shutdown_connections 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.051+0000 7fe5651d5740 1 --2- 172.21.15.45:0/3665191734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f6b750 0x556770f67630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.051+0000 7fe5651d5740 1 --2- 172.21.15.45:0/3665191734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62980 0x556770f68ef0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.051+0000 7fe5651d5740 1 --2- 172.21.15.45:0/3665191734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f62060 0x556770f6ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.051+0000 7fe5651d5740 1 -- 172.21.15.45:0/3665191734 >> 172.21.15.45:0/3665191734 conn(0x556770f5c980 msgr2=0x556770f66f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.051+0000 7fe5651d5740 1 -- 172.21.15.45:0/3665191734 shutdown_connections 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.051+0000 7fe5651d5740 1 -- 172.21.15.45:0/3665191734 wait complete. 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.051+0000 7fe5651d5740 1 -- start start 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe5651d5740 1 Processor -- start 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe5651d5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 0x556770f76a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe5651d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f62980 0x556770f74df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe5651d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f6b750 0x556770f75330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe5651d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556770f758c0 con 0x556770f62980 2024-01-31T05:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe5651d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556770f75a30 con 0x556770f6b750 2024-01-31T05:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe5651d5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556770f75bf0 con 0x556770f62060 2024-01-31T05:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe5651d5740 10 client.0 _ll_register_callbacks cb 0x7fe54c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f62980 0x556770f74df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe55bfff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 0x556770f76a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe55bfff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 0x556770f76a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:55530/0 (socket says 172.21.15.45:55530) 2024-01-31T05:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe55bfff640 1 -- 172.21.15.45:0/2165476684 learned_addr learned my addr 172.21.15.45:0/2165476684 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe5608d8640 1 --2- 172.21.15.45:0/2165476684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f6b750 0x556770f75330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe55bfff640 1 -- 172.21.15.45:0/2165476684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f6b750 msgr2=0x556770f75330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe55bfff640 1 --2- 172.21.15.45:0/2165476684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f6b750 0x556770f75330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe55bfff640 1 -- 172.21.15.45:0/2165476684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f62980 msgr2=0x556770f74df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe55bfff640 1 --2- 172.21.15.45:0/2165476684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f62980 0x556770f74df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.052+0000 7fe55bfff640 1 -- 172.21.15.45:0/2165476684 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c1a3ee0 con 0x556770f62060 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe55bfff640 1 --2- 172.21.15.45:0/2165476684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 0x556770f76a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe54c02c6f0 tx=0x7fe54c02c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d53e2c7d74c08000 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe53b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe53b7fe640 1 -- 172.21.15.45:0/2165476684 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c057070 con 0x556770f62060 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe53b7fe640 1 -- 172.21.15.45:0/2165476684 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe54c050040 con 0x556770f62060 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe5651d5740 1 -- 172.21.15.45:0/2165476684 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe54c004a50 con 0x556770f62060 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe53b7fe640 1 -- 172.21.15.45:0/2165476684 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe54c004040 con 0x556770f62060 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe5651d5740 10 client.25363 fetch_fsmap learned FSMap version 26 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe5651d5740 1 -- 172.21.15.45:0/2165476684 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556770e8aa50 con 0x556770f62060 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe53b7fe640 1 -- 172.21.15.45:0/2165476684 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe54c004a50 con 0x556770f62060 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe53b7fe640 1 -- 172.21.15.45:0/2165476684 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe54c05a020 con 0x556770f62060 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe5651d5740 10 client.25363 fetch_fsmap finished waiting for FSMap version 26 2024-01-31T05:27:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe5651d5740 10 client.25363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:27:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe5651d5740 20 client.25363 populate_metadata read hostname 'smithi045' 2024-01-31T05:27:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe5651d5740 1 --2- 172.21.15.45:0/2165476684 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556770fd7080 0x556770fd94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:27:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe5651d5740 4 client.25363 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:27:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe5651d5740 1 -- 172.21.15.45:0/2165476684 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556770bc9600 con 0x556770fd7080 2024-01-31T05:27:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.053+0000 7fe55b7fe640 1 --2- 172.21.15.45:0/2165476684 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556770fd7080 0x556770fd94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:27:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.054+0000 7fe53b7fe640 10 client.25363 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.054+0000 7fe55b7fe640 1 --2- 172.21.15.45:0/2165476684 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556770fd7080 0x556770fd94a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5637d99180b0499 server_cookie=58b74d71f19f1a8d in_seq=0 out_seq=0 2024-01-31T05:27:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.063+0000 7fe53b7fe640 1 -- 172.21.15.45:0/2165476684 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556770bc9600 con 0x556770fd7080 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.063+0000 7fe53b7fe640 10 client.25363 handle_command_reply: tid=0 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.063+0000 7fe5651d5740 1 client.25363 shutdown 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe5651d5740 1 -- 172.21.15.45:0/2165476684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 msgr2=0x556770f76a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe5651d5740 1 --2- 172.21.15.45:0/2165476684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 0x556770f76a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe54c02c6f0 tx=0x7fe54c02c680 comp rx=0 tx=0).stop 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe5651d5740 1 -- 172.21.15.45:0/2165476684 shutdown_connections 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe5651d5740 1 --2- 172.21.15.45:0/2165476684 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556770fd7080 0x556770fd94a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe5651d5740 1 --2- 172.21.15.45:0/2165476684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556770f62980 0x556770f74df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe53b7fe640 0 client.25363 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe5651d5740 1 --2- 172.21.15.45:0/2165476684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556770f6b750 0x556770f75330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe5651d5740 1 --2- 172.21.15.45:0/2165476684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556770f62060 0x556770f76a00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe5651d5740 1 -- 172.21.15.45:0/2165476684 >> 172.21.15.45:0/2165476684 conn(0x556770f5c980 msgr2=0x556770f5e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe5651d5740 1 -- 172.21.15.45:0/2165476684 shutdown_connections 2024-01-31T05:27:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe5651d5740 1 -- 172.21.15.45:0/2165476684 wait complete. 2024-01-31T05:27:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe5651d5740 20 client.25363 trim_cache size 0 max 16384 2024-01-31T05:27:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:27:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:27:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:27:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe56233b640 1 -- 172.21.15.45:0/2636267023 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe5440a1570 msgr2=0x7fe5440a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe56233b640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe5440a1570 0x7fe5440a3a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe554005fd0 tx=0x7fe554014b50 comp rx=0 tx=0).stop 2024-01-31T05:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe56233b640 1 -- 172.21.15.45:0/2636267023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c068d10 msgr2=0x7fe55c197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe56233b640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c068d10 0x7fe55c197720 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe55000e9f0 tx=0x7fe55000edf0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe56233b640 1 -- 172.21.15.45:0/2636267023 shutdown_connections 2024-01-31T05:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe56233b640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe5440a1570 0x7fe5440a3a30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe56233b640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c068d10 0x7fe55c197720 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe56233b640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe55c106e00 0x7fe55c19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe56233b640 1 --2- 172.21.15.45:0/2636267023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe55c068370 0x7fe55c1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe56233b640 1 -- 172.21.15.45:0/2636267023 >> 172.21.15.45:0/2636267023 conn(0x7fe55c076a40 msgr2=0x7fe55c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:27:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.064+0000 7fe56233b640 1 -- 172.21.15.45:0/2636267023 shutdown_connections 2024-01-31T05:27:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:27:50.065+0000 7fe56233b640 1 -- 172.21.15.45:0/2636267023 wait complete. 2024-01-31T05:27:50.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 529847 13.04 MB/sec execute 110 sec latency 13.977 ms 2024-01-31T05:27:50.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:50 smithi186 ceph-mon[35659]: pgmap v159: 97 pgs: 97 active+clean; 283 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 212 op/s 2024-01-31T05:27:50.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:50 smithi045 ceph-mon[31585]: pgmap v159: 97 pgs: 97 active+clean; 283 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 212 op/s 2024-01-31T05:27:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:50 smithi156 ceph-mon[35720]: pgmap v159: 97 pgs: 97 active+clean; 283 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 212 op/s 2024-01-31T05:27:51.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 532117 13.02 MB/sec execute 111 sec latency 137.897 ms 2024-01-31T05:27:51.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:51 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:27:51.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:51 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:27:52.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:51 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:27:52.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 533995 12.99 MB/sec execute 112 sec latency 78.968 ms 2024-01-31T05:27:52.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:52 smithi045 ceph-mon[31585]: pgmap v160: 97 pgs: 97 active+clean; 283 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 212 op/s 2024-01-31T05:27:52.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:52 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:27:52.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:52 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:27:52.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:52 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:27:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:52 smithi156 ceph-mon[35720]: pgmap v160: 97 pgs: 97 active+clean; 283 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 212 op/s 2024-01-31T05:27:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:52 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:27:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:52 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:27:52.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:52 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:27:53.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:52 smithi186 ceph-mon[35659]: pgmap v160: 97 pgs: 97 active+clean; 283 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 212 op/s 2024-01-31T05:27:53.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:52 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:27:53.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:52 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:27:53.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:52 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:27:53.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 536833 13.03 MB/sec execute 113 sec latency 7.811 ms 2024-01-31T05:27:53.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:53 smithi045 ceph-mon[31585]: pgmap v161: 97 pgs: 97 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 322 op/s 2024-01-31T05:27:53.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:53 smithi156 ceph-mon[35720]: pgmap v161: 97 pgs: 97 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 322 op/s 2024-01-31T05:27:54.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:53 smithi186 ceph-mon[35659]: pgmap v161: 97 pgs: 97 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 322 op/s 2024-01-31T05:27:54.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 539792 13.04 MB/sec execute 114 sec latency 7.030 ms 2024-01-31T05:27:55.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 542088 13.02 MB/sec execute 115 sec latency 15.290 ms 2024-01-31T05:27:56.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 543996 13.02 MB/sec execute 116 sec latency 71.904 ms 2024-01-31T05:27:56.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:56 smithi186 ceph-mon[35659]: pgmap v162: 97 pgs: 97 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 218 op/s 2024-01-31T05:27:56.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:56 smithi045 ceph-mon[31585]: pgmap v162: 97 pgs: 97 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 218 op/s 2024-01-31T05:27:56.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:56 smithi156 ceph-mon[35720]: pgmap v162: 97 pgs: 97 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 218 op/s 2024-01-31T05:27:57.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 546512 13.02 MB/sec execute 117 sec latency 24.414 ms 2024-01-31T05:27:58.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 548520 13.01 MB/sec execute 118 sec latency 36.793 ms 2024-01-31T05:27:58.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:27:58 smithi186 ceph-mon[35659]: pgmap v163: 97 pgs: 97 active+clean; 317 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 328 op/s 2024-01-31T05:27:58.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:27:58 smithi045 ceph-mon[31585]: pgmap v163: 97 pgs: 97 active+clean; 317 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 328 op/s 2024-01-31T05:27:58.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:27:58 smithi156 ceph-mon[35720]: pgmap v163: 97 pgs: 97 active+clean; 317 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 328 op/s 2024-01-31T05:27:59.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 551095 13.02 MB/sec execute 119 sec latency 13.525 ms 2024-01-31T05:28:00.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 553105 12.97 MB/sec execute 120 sec latency 21.949 ms 2024-01-31T05:28:00.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:00 smithi186 ceph-mon[35659]: pgmap v164: 97 pgs: 97 active+clean; 317 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 220 op/s 2024-01-31T05:28:00.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:00 smithi045 ceph-mon[31585]: pgmap v164: 97 pgs: 97 active+clean; 317 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 220 op/s 2024-01-31T05:28:00.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:00 smithi156 ceph-mon[35720]: pgmap v164: 97 pgs: 97 active+clean; 317 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 220 op/s 2024-01-31T05:28:01.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 554837 12.96 MB/sec execute 121 sec latency 63.295 ms 2024-01-31T05:28:02.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 557723 13.00 MB/sec execute 122 sec latency 19.130 ms 2024-01-31T05:28:02.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:02 smithi186 ceph-mon[35659]: pgmap v165: 97 pgs: 97 active+clean; 317 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 220 op/s 2024-01-31T05:28:02.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:02 smithi045 ceph-mon[31585]: pgmap v165: 97 pgs: 97 active+clean; 317 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 220 op/s 2024-01-31T05:28:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:02 smithi156 ceph-mon[35720]: pgmap v165: 97 pgs: 97 active+clean; 317 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 220 op/s 2024-01-31T05:28:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.153+0000 7f4cc416b640 1 -- 172.21.15.45:0/3215359590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cbc068d10 msgr2=0x7f4cbc10db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.153+0000 7f4cc416b640 1 --2- 172.21.15.45:0/3215359590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cbc068d10 0x7f4cbc10db80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009900 tx=0x7f4cb802bc30 comp rx=0 tx=0).stop 2024-01-31T05:28:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.153+0000 7f4cc416b640 1 -- 172.21.15.45:0/3215359590 shutdown_connections 2024-01-31T05:28:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.153+0000 7f4cc416b640 1 --2- 172.21.15.45:0/3215359590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cbc068d10 0x7f4cbc10db80 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.153+0000 7f4cc416b640 1 --2- 172.21.15.45:0/3215359590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cbc068370 0x7f4cbc0687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.153+0000 7f4cc416b640 1 --2- 172.21.15.45:0/3215359590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cbc106e00 0x7f4cbc1071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.153+0000 7f4cc416b640 1 -- 172.21.15.45:0/3215359590 >> 172.21.15.45:0/3215359590 conn(0x7f4cbc076a40 msgr2=0x7f4cbc076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.153+0000 7f4cc416b640 1 -- 172.21.15.45:0/3215359590 shutdown_connections 2024-01-31T05:28:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.153+0000 7f4cc416b640 1 -- 172.21.15.45:0/3215359590 wait complete. 2024-01-31T05:28:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.153+0000 7f4cc416b640 1 Processor -- start 2024-01-31T05:28:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.153+0000 7f4cc416b640 1 -- start start 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc416b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cbc068370 0x7f4cbc197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc416b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cbc068d10 0x7f4cbc197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc416b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cbc106e00 0x7f4cbc19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc416b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc19bf80 con 0x7f4cbc068d10 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc416b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc19c0f0 con 0x7f4cbc106e00 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc416b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc19c2b0 con 0x7f4cbc068370 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc1706640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cbc068d10 0x7f4cbc197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc1f07640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cbc068370 0x7f4cbc197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc1706640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cbc068d10 0x7f4cbc197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48256/0 (socket says 172.21.15.45:48256) 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc1706640 1 -- 172.21.15.45:0/4192787132 learned_addr learned my addr 172.21.15.45:0/4192787132 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc2708640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cbc106e00 0x7f4cbc19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc1f07640 1 -- 172.21.15.45:0/4192787132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cbc106e00 msgr2=0x7f4cbc19b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc1f07640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cbc106e00 0x7f4cbc19b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc1f07640 1 -- 172.21.15.45:0/4192787132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cbc068d10 msgr2=0x7f4cbc197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc1f07640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cbc068d10 0x7f4cbc197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc1f07640 1 -- 172.21.15.45:0/4192787132 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb8009560 con 0x7f4cbc068370 2024-01-31T05:28:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.154+0000 7f4cc1f07640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cbc068370 0x7f4cbc197110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009fd0 tx=0x7f4cb400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.155+0000 7f4cb2ffd640 1 -- 172.21.15.45:0/4192787132 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400cea0 con 0x7f4cbc068370 2024-01-31T05:28:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.155+0000 7f4cc416b640 1 -- 172.21.15.45:0/4192787132 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cbc19c590 con 0x7f4cbc068370 2024-01-31T05:28:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.155+0000 7f4cc416b640 1 -- 172.21.15.45:0/4192787132 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cbc1a3e10 con 0x7f4cbc068370 2024-01-31T05:28:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.155+0000 7f4cb2ffd640 1 -- 172.21.15.45:0/4192787132 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb4010430 con 0x7f4cbc068370 2024-01-31T05:28:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.155+0000 7f4cb2ffd640 1 -- 172.21.15.45:0/4192787132 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400f620 con 0x7f4cbc068370 2024-01-31T05:28:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.157+0000 7f4cb2ffd640 1 -- 172.21.15.45:0/4192787132 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f4cb40105a0 con 0x7f4cbc068370 2024-01-31T05:28:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.157+0000 7f4cc700a740 1 Processor -- start 2024-01-31T05:28:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.157+0000 7f4cc700a740 1 -- start start 2024-01-31T05:28:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.157+0000 7f4cc700a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3060 0x55b64afe3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.157+0000 7f4cc700a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3980 0x55b64afec210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.157+0000 7f4cb2ffd640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4ca80a14f0 0x7f4ca80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.157+0000 7f4cc700a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64afeeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.157+0000 7f4cc700a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64af52040 con 0x55b64afec750 2024-01-31T05:28:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.157+0000 7f4cc700a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64af4f730 con 0x55b64afe3060 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.157+0000 7f4cc700a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64af4b020 con 0x55b64afe3980 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc2708640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3060 0x55b64afe3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc1f07640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3980 0x55b64afec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc1706640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4ca80a14f0 0x7f4ca80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc2708640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3060 0x55b64afe3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:40928/0 (socket says 172.21.15.45:40928) 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc2708640 1 -- 172.21.15.45:0/3060816069 learned_addr learned my addr 172.21.15.45:0/3060816069 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc2708640 1 --2- 172.21.15.45:0/3060816069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64afeeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cb2ffd640 1 -- 172.21.15.45:0/4192787132 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4cb4014070 con 0x7f4cbc068370 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc2708640 1 -- 172.21.15.45:0/3060816069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3980 msgr2=0x55b64afec210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc2708640 1 --2- 172.21.15.45:0/3060816069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3980 0x55b64afec210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc2708640 1 -- 172.21.15.45:0/3060816069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 msgr2=0x55b64afeeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc2708640 1 --2- 172.21.15.45:0/3060816069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64afeeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc2708640 1 -- 172.21.15.45:0/3060816069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b64af42e40 con 0x55b64afe3060 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc1f07640 1 --2- 172.21.15.45:0/3060816069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3980 0x55b64afec210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc1706640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4ca80a14f0 0x7f4ca80a39b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc0770b0 tx=0x7f4cac00b480 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4cc2708640 1 --2- 172.21.15.45:0/3060816069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3060 0x55b64afe3440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8004f80 tx=0x7f4cb8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86623f7d1f1bc26 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4ca7fff640 1 -- 172.21.15.45:0/3060816069 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb802c780 con 0x55b64afe3060 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4ca7fff640 1 -- 172.21.15.45:0/3060816069 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb8004030 con 0x55b64afe3060 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.158+0000 7f4ca7fff640 1 -- 172.21.15.45:0/3060816069 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8005380 con 0x55b64afe3060 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.159+0000 7f4cc700a740 1 -- 172.21.15.45:0/3060816069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3060 msgr2=0x55b64afe3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.159+0000 7f4cc700a740 1 --2- 172.21.15.45:0/3060816069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3060 0x55b64afe3440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8004f80 tx=0x7f4cb8009260 comp rx=0 tx=0).stop 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.159+0000 7f4cc700a740 1 -- 172.21.15.45:0/3060816069 shutdown_connections 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.159+0000 7f4cc700a740 1 --2- 172.21.15.45:0/3060816069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64afeeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.159+0000 7f4cc700a740 1 --2- 172.21.15.45:0/3060816069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3980 0x55b64afec210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.159+0000 7f4cc700a740 1 --2- 172.21.15.45:0/3060816069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3060 0x55b64afe3440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.159+0000 7f4cc700a740 1 -- 172.21.15.45:0/3060816069 >> 172.21.15.45:0/3060816069 conn(0x55b64afdd980 msgr2=0x55b64afddd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.159+0000 7f4cc700a740 1 -- 172.21.15.45:0/3060816069 shutdown_connections 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.159+0000 7f4cc700a740 1 -- 172.21.15.45:0/3060816069 wait complete. 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc700a740 1 -- start start 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc700a740 1 Processor -- start 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc700a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64afe9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc700a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3980 0x55b64afe9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc700a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64afea080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc700a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64af041b0 con 0x55b64afec750 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc700a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64af03e50 con 0x55b64afe3980 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc700a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64aec9090 con 0x55b64afe3060 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc700a740 10 client.0 _ll_register_callbacks cb 0x55b64aeeea90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc2708640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64afea080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc1f07640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64afe9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc2708640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64afea080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48274/0 (socket says 172.21.15.45:48274) 2024-01-31T05:28:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc1706640 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3980 0x55b64afe9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc2708640 1 -- 172.21.15.45:0/945809426 learned_addr learned my addr 172.21.15.45:0/945809426 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc1f07640 1 -- 172.21.15.45:0/945809426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3980 msgr2=0x55b64afe9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc1f07640 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3980 0x55b64afe9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc1f07640 1 -- 172.21.15.45:0/945809426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 msgr2=0x55b64afea080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc1f07640 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64afea080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc1f07640 1 -- 172.21.15.45:0/945809426 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cbc10e480 con 0x55b64afe3060 2024-01-31T05:28:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.160+0000 7f4cc2708640 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64afea080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:28:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4cc1f07640 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64afe9600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40ca980 tx=0x7f4cb40ada10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71fb2e6dfb5f72f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4ca57fa640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:28:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4ca57fa640 1 -- 172.21.15.45:0/945809426 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb40cd070 con 0x55b64afe3060 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4ca57fa640 1 -- 172.21.15.45:0/945809426 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4cb40afb70 con 0x55b64afe3060 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4cc700a740 1 -- 172.21.15.45:0/945809426 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b64af44010 con 0x55b64afe3060 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4ca57fa640 1 -- 172.21.15.45:0/945809426 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cb40b8170 con 0x55b64afe3060 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4cc700a740 10 client.25375 fetch_fsmap learned FSMap version 26 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4cc700a740 1 -- 172.21.15.45:0/945809426 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b64af496a0 con 0x55b64afe3060 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4ca57fa640 1 -- 172.21.15.45:0/945809426 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb407c080 con 0x55b64afe3060 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4ca57fa640 1 -- 172.21.15.45:0/945809426 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4cb4010850 con 0x55b64afe3060 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4cc700a740 10 client.25375 fetch_fsmap finished waiting for FSMap version 26 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4cc700a740 10 client.25375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4cc700a740 20 client.25375 populate_metadata read hostname 'smithi045' 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4cc700a740 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b64affc4d0 0x55b64affe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4cc700a740 4 client.25375 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.161+0000 7f4cc700a740 1 -- 172.21.15.45:0/945809426 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b64af43d30 con 0x55b64affc4d0 2024-01-31T05:28:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.162+0000 7f4cc1706640 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b64affc4d0 0x55b64affe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.162+0000 7f4cc1706640 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b64affc4d0 0x55b64affe8f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a2ea3bd6f47b6f9 server_cookie=7c5f5f454153e269 in_seq=0 out_seq=0 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.162+0000 7f4ca57fa640 10 client.25375 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4ca57fa640 1 -- 172.21.15.45:0/945809426 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b64af43d30 con 0x55b64affc4d0 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4ca57fa640 10 client.25375 handle_command_reply: tid=0 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4cc700a740 1 client.25375 shutdown 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4cc700a740 1 -- 172.21.15.45:0/945809426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 msgr2=0x55b64afe9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4cc700a740 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64afe9600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40ca980 tx=0x7f4cb40ada10 comp rx=0 tx=0).stop 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4cc700a740 1 -- 172.21.15.45:0/945809426 shutdown_connections 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4cc700a740 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b64affc4d0 0x55b64affe8f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4cc700a740 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64afea080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4cc700a740 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3980 0x55b64afe9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4cc700a740 1 --2- 172.21.15.45:0/945809426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64afe9600 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4ca57fa640 0 client.25375 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4cc700a740 1 -- 172.21.15.45:0/945809426 >> 172.21.15.45:0/945809426 conn(0x55b64afdd980 msgr2=0x55b64afdf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4cc700a740 1 -- 172.21.15.45:0/945809426 shutdown_connections 2024-01-31T05:28:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4cc700a740 1 -- 172.21.15.45:0/945809426 wait complete. 2024-01-31T05:28:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.163+0000 7f4cc700a740 20 client.25375 trim_cache size 0 max 16384 2024-01-31T05:28:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.176+0000 7f4cc700a740 1 Processor -- start 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.176+0000 7f4cc700a740 1 -- start start 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.176+0000 7f4cc700a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64afe8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.176+0000 7f4cc700a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afe3980 0x55b64afdc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.176+0000 7f4cc700a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afec750 0x55b64afe9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.176+0000 7f4cc700a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64afe8cc0 con 0x55b64afe3980 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.176+0000 7f4cc700a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64afeeec0 con 0x55b64afec750 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.176+0000 7f4cc700a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64afdd7b0 con 0x55b64afe3060 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4cc1706640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afe3980 0x55b64afdc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4cc1706640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afe3980 0x55b64afdc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48288/0 (socket says 172.21.15.45:48288) 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4cc1706640 1 -- 172.21.15.45:0/1672013983 learned_addr learned my addr 172.21.15.45:0/1672013983 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4cc1f07640 1 --2- 172.21.15.45:0/1672013983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64afe8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4cc2708640 1 --2- 172.21.15.45:0/1672013983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afec750 0x55b64afe9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4cc1706640 1 -- 172.21.15.45:0/1672013983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 msgr2=0x55b64afe8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4cc1706640 1 --2- 172.21.15.45:0/1672013983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64afe8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4cc1706640 1 -- 172.21.15.45:0/1672013983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afec750 msgr2=0x55b64afe9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4cc1706640 1 --2- 172.21.15.45:0/1672013983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afec750 0x55b64afe9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4cc1706640 1 -- 172.21.15.45:0/1672013983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b64aee6030 con 0x55b64afe3980 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4cc2708640 1 --2- 172.21.15.45:0/1672013983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afec750 0x55b64afe9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4cc1706640 1 --2- 172.21.15.45:0/1672013983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afe3980 0x55b64afdc8a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00b770 tx=0x7f4cac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4957323951e9014d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4ca77fe640 1 -- 172.21.15.45:0/1672013983 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac026b00 con 0x55b64afe3980 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4ca77fe640 1 -- 172.21.15.45:0/1672013983 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cac025040 con 0x55b64afe3980 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.177+0000 7f4ca77fe640 1 -- 172.21.15.45:0/1672013983 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0054e0 con 0x55b64afe3980 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.178+0000 7f4cc700a740 1 -- 172.21.15.45:0/1672013983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afe3980 msgr2=0x55b64afdc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.178+0000 7f4cc700a740 1 --2- 172.21.15.45:0/1672013983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afe3980 0x55b64afdc8a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00b770 tx=0x7f4cac013000 comp rx=0 tx=0).stop 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.178+0000 7f4cc700a740 1 -- 172.21.15.45:0/1672013983 shutdown_connections 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.178+0000 7f4cc700a740 1 --2- 172.21.15.45:0/1672013983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afe3980 0x55b64afdc8a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.178+0000 7f4cc700a740 1 --2- 172.21.15.45:0/1672013983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afec750 0x55b64afe9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.178+0000 7f4cc700a740 1 --2- 172.21.15.45:0/1672013983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64afe8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.178+0000 7f4cc700a740 1 -- 172.21.15.45:0/1672013983 >> 172.21.15.45:0/1672013983 conn(0x55b64afdd980 msgr2=0x55b64afdc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.178+0000 7f4cc700a740 1 -- 172.21.15.45:0/1672013983 shutdown_connections 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.178+0000 7f4cc700a740 1 -- 172.21.15.45:0/1672013983 wait complete. 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.178+0000 7f4cc700a740 1 -- start start 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.178+0000 7f4cc700a740 1 Processor -- start 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc700a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64aff2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc700a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3980 0x55b64aff2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc700a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64b04b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc700a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64b04bec0 con 0x55b64afec750 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc700a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64b04c030 con 0x55b64afe3980 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc700a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64b04c1f0 con 0x55b64afe3060 2024-01-31T05:28:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc700a740 10 client.0 _ll_register_callbacks cb 0x7f4cb40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:28:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc2708640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64b04b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc2708640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64b04b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48296/0 (socket says 172.21.15.45:48296) 2024-01-31T05:28:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc2708640 1 -- 172.21.15.45:0/3877676611 learned_addr learned my addr 172.21.15.45:0/3877676611 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc1f07640 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64aff2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc1706640 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3980 0x55b64aff2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc2708640 1 -- 172.21.15.45:0/3877676611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 msgr2=0x55b64aff2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc2708640 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64aff2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc2708640 1 -- 172.21.15.45:0/3877676611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3980 msgr2=0x55b64aff2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc2708640 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3980 0x55b64aff2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc2708640 1 -- 172.21.15.45:0/3877676611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cac011560 con 0x55b64afec750 2024-01-31T05:28:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc2708640 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64b04b980 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009900 tx=0x7f4cb802c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7da0324f4b8c98 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4ca57fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4ca57fa640 1 -- 172.21.15.45:0/3877676611 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb80046c0 con 0x55b64afec750 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4ca57fa640 1 -- 172.21.15.45:0/3877676611 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4cb80035f0 con 0x55b64afec750 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.179+0000 7f4cc700a740 1 -- 172.21.15.45:0/3877676611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cb8004030 con 0x55b64afec750 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.180+0000 7f4ca57fa640 1 -- 172.21.15.45:0/3877676611 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cb80037b0 con 0x55b64afec750 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.180+0000 7f4cc700a740 10 client.15831 fetch_fsmap learned FSMap version 26 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.180+0000 7f4cc700a740 1 -- 172.21.15.45:0/3877676611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b64af0ba50 con 0x55b64afec750 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.180+0000 7f4ca57fa640 1 -- 172.21.15.45:0/3877676611 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb8004030 con 0x55b64afec750 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.180+0000 7f4ca57fa640 1 -- 172.21.15.45:0/3877676611 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4cb8005490 con 0x55b64afec750 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.180+0000 7f4cc700a740 10 client.15831 fetch_fsmap finished waiting for FSMap version 26 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.180+0000 7f4cc700a740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.180+0000 7f4cc700a740 20 client.15831 populate_metadata read hostname 'smithi045' 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.180+0000 7f4cc700a740 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b64b0547a0 0x55b64b056b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.180+0000 7f4cc700a740 4 client.15831 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:28:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.180+0000 7f4cc700a740 1 -- 172.21.15.45:0/3877676611 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b64ac4a600 con 0x55b64b0547a0 2024-01-31T05:28:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.180+0000 7f4cc1f07640 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b64b0547a0 0x55b64b056b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.181+0000 7f4cc1f07640 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b64b0547a0 0x55b64b056b90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7a9f12d8da1d07 server_cookie=fb4e23c420e7b837 in_seq=0 out_seq=0 2024-01-31T05:28:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:03.181+0000 7f4ca57fa640 10 client.15831 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:03.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 559481 12.95 MB/sec execute 123 sec latency 94.550 ms 2024-01-31T05:28:04.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 561855 12.95 MB/sec execute 124 sec latency 10.738 ms 2024-01-31T05:28:04.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:04 smithi045 ceph-mon[31585]: pgmap v166: 97 pgs: 97 active+clean; 324 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.6 MiB/s wr, 365 op/s 2024-01-31T05:28:04.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:04 smithi156 ceph-mon[35720]: pgmap v166: 97 pgs: 97 active+clean; 324 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.6 MiB/s wr, 365 op/s 2024-01-31T05:28:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4ca57fa640 1 -- 172.21.15.45:0/3877676611 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b64ac4a600 con 0x55b64b0547a0 2024-01-31T05:28:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4ca57fa640 10 client.15831 handle_command_reply: tid=0 2024-01-31T05:28:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4cc700a740 1 client.15831 shutdown 2024-01-31T05:28:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4cc700a740 1 -- 172.21.15.45:0/3877676611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 msgr2=0x55b64b04b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4cc700a740 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64b04b980 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009900 tx=0x7f4cb802c9b0 comp rx=0 tx=0).stop 2024-01-31T05:28:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4cc700a740 1 -- 172.21.15.45:0/3877676611 shutdown_connections 2024-01-31T05:28:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4cc700a740 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b64b0547a0 0x55b64b056b90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4cc700a740 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b64afec750 0x55b64b04b980 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4cc700a740 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b64afe3980 0x55b64aff2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4cc700a740 1 --2- 172.21.15.45:0/3877676611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b64afe3060 0x55b64aff2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4ca57fa640 0 client.15831 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4cc700a740 1 -- 172.21.15.45:0/3877676611 >> 172.21.15.45:0/3877676611 conn(0x55b64afdd980 msgr2=0x55b64afdca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4cc700a740 1 -- 172.21.15.45:0/3877676611 shutdown_connections 2024-01-31T05:28:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4cc700a740 1 -- 172.21.15.45:0/3877676611 wait complete. 2024-01-31T05:28:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.011+0000 7f4cc700a740 20 client.15831 trim_cache size 0 max 16384 2024-01-31T05:28:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:28:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:28:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:28:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:28:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.012+0000 7f4cc416b640 1 -- 172.21.15.45:0/4192787132 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4ca80a14f0 msgr2=0x7f4ca80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.012+0000 7f4cc416b640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4ca80a14f0 0x7f4ca80a39b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc0770b0 tx=0x7f4cac00b480 comp rx=0 tx=0).stop 2024-01-31T05:28:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.012+0000 7f4cc416b640 1 -- 172.21.15.45:0/4192787132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cbc068370 msgr2=0x7f4cbc197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.012+0000 7f4cc416b640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cbc068370 0x7f4cbc197110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009fd0 tx=0x7f4cb400ef90 comp rx=0 tx=0).stop 2024-01-31T05:28:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.012+0000 7f4cc416b640 1 -- 172.21.15.45:0/4192787132 shutdown_connections 2024-01-31T05:28:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.012+0000 7f4cc416b640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4ca80a14f0 0x7f4ca80a39b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.012+0000 7f4cc416b640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cbc068d10 0x7f4cbc197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.012+0000 7f4cc416b640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cbc106e00 0x7f4cbc19b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.012+0000 7f4cc416b640 1 --2- 172.21.15.45:0/4192787132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cbc068370 0x7f4cbc197110 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.012+0000 7f4cc416b640 1 -- 172.21.15.45:0/4192787132 >> 172.21.15.45:0/4192787132 conn(0x7f4cbc076a40 msgr2=0x7f4cbc0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.012+0000 7f4cc416b640 1 -- 172.21.15.45:0/4192787132 shutdown_connections 2024-01-31T05:28:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:05.012+0000 7f4cc416b640 1 -- 172.21.15.45:0/4192787132 wait complete. 2024-01-31T05:28:05.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:04 smithi186 ceph-mon[35659]: pgmap v166: 97 pgs: 97 active+clean; 324 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.6 MiB/s wr, 365 op/s 2024-01-31T05:28:05.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 564912 12.99 MB/sec execute 125 sec latency 8.620 ms 2024-01-31T05:28:05.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:05 smithi045 ceph-mon[31585]: pgmap v167: 97 pgs: 97 active+clean; 324 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 255 op/s 2024-01-31T05:28:05.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-mon[35720]: pgmap v167: 97 pgs: 97 active+clean; 324 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 255 op/s 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: *** Caught signal (Segmentation fault) ** 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: in thread 7fed06a80700 thread_name:md_log_replay 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fed17640d20] 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 2: /usr/bin/ceph-mds(+0x5040bf) [0x5606e9dcd0bf] 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5606e9dcf2f3] 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 4: (MDLog::_replay_thread()+0x647) [0x5606e9e2b697] 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5606e9e34b31] 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fed176361ca] 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 7: clone() 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 2024-01-31T05:28:05.681+0000 7fed06a80700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: in thread 7fed06a80700 thread_name:md_log_replay 2024-01-31T05:28:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fed17640d20] 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 2: /usr/bin/ceph-mds(+0x5040bf) [0x5606e9dcd0bf] 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5606e9dcf2f3] 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 4: (MDLog::_replay_thread()+0x647) [0x5606e9e2b697] 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5606e9e34b31] 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fed176361ca] 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 7: clone() 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 0> 2024-01-31T05:28:05.681+0000 7fed06a80700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: in thread 7fed06a80700 thread_name:md_log_replay 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:28:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fed17640d20] 2024-01-31T05:28:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 2: /usr/bin/ceph-mds(+0x5040bf) [0x5606e9dcd0bf] 2024-01-31T05:28:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5606e9dcf2f3] 2024-01-31T05:28:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 4: (MDLog::_replay_thread()+0x647) [0x5606e9e2b697] 2024-01-31T05:28:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5606e9e34b31] 2024-01-31T05:28:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fed176361ca] 2024-01-31T05:28:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 7: clone() 2024-01-31T05:28:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:28:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 2024-01-31T05:28:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 0> 2024-01-31T05:28:05.681+0000 7fed06a80700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:28:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: in thread 7fed06a80700 thread_name:md_log_replay 2024-01-31T05:28:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 2024-01-31T05:28:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:28:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fed17640d20] 2024-01-31T05:28:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 2: /usr/bin/ceph-mds(+0x5040bf) [0x5606e9dcd0bf] 2024-01-31T05:28:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5606e9dcf2f3] 2024-01-31T05:28:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 4: (MDLog::_replay_thread()+0x647) [0x5606e9e2b697] 2024-01-31T05:28:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5606e9e34b31] 2024-01-31T05:28:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fed176361ca] 2024-01-31T05:28:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 7: clone() 2024-01-31T05:28:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:28:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70238]: 2024-01-31T05:28:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:05 smithi156 podman[70660]: 2024-01-31 05:28:05.729981001 +0000 UTC m=+0.020249647 container died e62386f272badf26e91fc8224e715afc91bc1c1f157b60f20c521561afddbc3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:28:06.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:05 smithi186 ceph-mon[35659]: pgmap v167: 97 pgs: 97 active+clean; 324 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 255 op/s 2024-01-31T05:28:06.726 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:06 smithi156 podman[70660]: 2024-01-31 05:28:06.33969221 +0000 UTC m=+0.629960856 container remove e62386f272badf26e91fc8224e715afc91bc1c1f157b60f20c521561afddbc3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-01-31T05:28:06.727 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:06 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-01-31T05:28:06.727 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:06 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-01-31T05:28:06.782 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 12.90 MB/sec execute 126 sec latency 608.770 ms 2024-01-31T05:28:07.783 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 12.80 MB/sec execute 127 sec latency 1608.853 ms 2024-01-31T05:28:08.783 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 12.70 MB/sec execute 128 sec latency 2608.967 ms 2024-01-31T05:28:08.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:08 smithi186 ceph-mon[35659]: pgmap v168: 97 pgs: 97 active+clean; 341 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.6 MiB/s wr, 337 op/s 2024-01-31T05:28:08.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:08 smithi045 ceph-mon[31585]: pgmap v168: 97 pgs: 97 active+clean; 341 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.6 MiB/s wr, 337 op/s 2024-01-31T05:28:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:08 smithi156 ceph-mon[35720]: pgmap v168: 97 pgs: 97 active+clean; 341 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.6 MiB/s wr, 337 op/s 2024-01-31T05:28:09.783 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 12.60 MB/sec execute 129 sec latency 3609.013 ms 2024-01-31T05:28:10.783 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 12.50 MB/sec execute 130 sec latency 4609.143 ms 2024-01-31T05:28:10.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:10 smithi186 ceph-mon[35659]: pgmap v169: 97 pgs: 97 active+clean; 341 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-01-31T05:28:10.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:10 smithi045 ceph-mon[31585]: pgmap v169: 97 pgs: 97 active+clean; 341 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-01-31T05:28:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:10 smithi156 ceph-mon[35720]: pgmap v169: 97 pgs: 97 active+clean; 341 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-01-31T05:28:11.783 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 12.41 MB/sec execute 131 sec latency 5609.175 ms 2024-01-31T05:28:12.783 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 12.31 MB/sec execute 132 sec latency 6609.207 ms 2024-01-31T05:28:12.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:12 smithi045 ceph-mon[31585]: pgmap v170: 97 pgs: 97 active+clean; 341 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-01-31T05:28:12.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:12 smithi156 ceph-mon[35720]: pgmap v170: 97 pgs: 97 active+clean; 341 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-01-31T05:28:13.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:12 smithi186 ceph-mon[35659]: pgmap v170: 97 pgs: 97 active+clean; 341 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-01-31T05:28:13.783 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 12.22 MB/sec execute 133 sec latency 7609.301 ms 2024-01-31T05:28:14.783 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 12.13 MB/sec execute 134 sec latency 8609.417 ms 2024-01-31T05:28:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:14 smithi045 ceph-mon[31585]: pgmap v171: 97 pgs: 97 active+clean; 338 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 262 op/s 2024-01-31T05:28:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:14 smithi156 ceph-mon[35720]: pgmap v171: 97 pgs: 97 active+clean; 338 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 262 op/s 2024-01-31T05:28:15.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:14 smithi186 ceph-mon[35659]: pgmap v171: 97 pgs: 97 active+clean; 338 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 262 op/s 2024-01-31T05:28:15.783 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 12.04 MB/sec execute 135 sec latency 9609.533 ms 2024-01-31T05:28:16.783 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 11.95 MB/sec execute 136 sec latency 10609.603 ms 2024-01-31T05:28:16.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:16 smithi045 ceph-mon[31585]: pgmap v172: 97 pgs: 97 active+clean; 338 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 117 op/s 2024-01-31T05:28:16.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:16 smithi156 ceph-mon[35720]: pgmap v172: 97 pgs: 97 active+clean; 338 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 117 op/s 2024-01-31T05:28:16.976 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:16 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 2. 2024-01-31T05:28:16.976 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:16 smithi156 systemd[1]: Stopped Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:28:16.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:16 smithi156 systemd[1]: Starting Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:28:17.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:16 smithi186 ceph-mon[35659]: pgmap v172: 97 pgs: 97 active+clean; 338 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 117 op/s 2024-01-31T05:28:17.232 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:16 smithi156 podman[70758]: 2024-01-31 05:28:16.884239077 +0000 UTC m=+0.017851290 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:28:17.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:17 smithi156 podman[70758]: 2024-01-31 05:28:17.01510727 +0000 UTC m=+0.148719529 container create d474a44b77d08d4dd5c66da9285d20797cac701a3cd4e20f60b14296b13fe043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-01-31T05:28:17.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:17 smithi156 podman[70758]: 2024-01-31 05:28:17.234103989 +0000 UTC m=+0.367716192 container init d474a44b77d08d4dd5c66da9285d20797cac701a3cd4e20f60b14296b13fe043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0) 2024-01-31T05:28:17.499 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:17 smithi156 podman[70758]: 2024-01-31 05:28:17.236917381 +0000 UTC m=+0.370529584 container start d474a44b77d08d4dd5c66da9285d20797cac701a3cd4e20f60b14296b13fe043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, org.label-schema.build-date=20240123, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3) 2024-01-31T05:28:17.499 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:17 smithi156 ceph-mds[70776]: -- 172.21.15.156:0/842954092 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5558da055520 con 0x5558da02b000 2024-01-31T05:28:17.499 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:17 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: starting mds.h at 2024-01-31T05:28:17.499 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:17 smithi156 bash[70758]: d474a44b77d08d4dd5c66da9285d20797cac701a3cd4e20f60b14296b13fe043 2024-01-31T05:28:17.499 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:28:17 smithi156 systemd[1]: Started Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:28:17.783 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 11.86 MB/sec execute 137 sec latency 11609.714 ms 2024-01-31T05:28:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:17 smithi045 ceph-mon[31585]: daemon mds.h restarted 2024-01-31T05:28:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:17 smithi045 ceph-mon[31585]: osdmap e92: 12 total, 12 up, 12 in 2024-01-31T05:28:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:17 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.156:6836/3704899919,v1:172.21.15.156:6837/3704899919] up:boot 2024-01-31T05:28:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:17 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:28:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:17 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:28:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:17 smithi156 ceph-mon[35720]: daemon mds.h restarted 2024-01-31T05:28:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:17 smithi156 ceph-mon[35720]: osdmap e92: 12 total, 12 up, 12 in 2024-01-31T05:28:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:17 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.156:6836/3704899919,v1:172.21.15.156:6837/3704899919] up:boot 2024-01-31T05:28:17.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:17 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:28:17.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:17 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:28:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:17 smithi186 ceph-mon[35659]: daemon mds.h restarted 2024-01-31T05:28:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:17 smithi186 ceph-mon[35659]: osdmap e92: 12 total, 12 up, 12 in 2024-01-31T05:28:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:17 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.156:6836/3704899919,v1:172.21.15.156:6837/3704899919] up:boot 2024-01-31T05:28:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:17 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:28:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:17 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:28:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.108+0000 7f4b70885640 1 -- 172.21.15.45:0/3668078571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c068370 msgr2=0x7f4b6c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.108+0000 7f4b70885640 1 --2- 172.21.15.45:0/3668078571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c068370 0x7f4b6c0687d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009900 tx=0x7f4b5402bc30 comp rx=0 tx=0).stop 2024-01-31T05:28:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.108+0000 7f4b70885640 1 -- 172.21.15.45:0/3668078571 shutdown_connections 2024-01-31T05:28:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.108+0000 7f4b70885640 1 --2- 172.21.15.45:0/3668078571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c068370 0x7f4b6c0687d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.108+0000 7f4b70885640 1 --2- 172.21.15.45:0/3668078571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b6c068d10 0x7f4b6c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.108+0000 7f4b70885640 1 --2- 172.21.15.45:0/3668078571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6c106e00 0x7f4b6c1071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.108+0000 7f4b70885640 1 -- 172.21.15.45:0/3668078571 >> 172.21.15.45:0/3668078571 conn(0x7f4b6c076a40 msgr2=0x7f4b6c076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.108+0000 7f4b70885640 1 -- 172.21.15.45:0/3668078571 shutdown_connections 2024-01-31T05:28:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.109+0000 7f4b70885640 1 -- 172.21.15.45:0/3668078571 wait complete. 2024-01-31T05:28:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.109+0000 7f4b70885640 1 Processor -- start 2024-01-31T05:28:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.109+0000 7f4b70885640 1 -- start start 2024-01-31T05:28:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b70885640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6c068370 0x7f4b6c197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b70885640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b6c068d10 0x7f4b6c1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b70885640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106e00 0x7f4b6c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b70885640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c19c030 con 0x7f4b6c106e00 2024-01-31T05:28:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b70885640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c19c1a0 con 0x7f4b6c068d10 2024-01-31T05:28:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b70885640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c19c360 con 0x7f4b6c068370 2024-01-31T05:28:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b6ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106e00 0x7f4b6c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b6ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106e00 0x7f4b6c19ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58134/0 (socket says 172.21.15.45:58134) 2024-01-31T05:28:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b6ad9d640 1 -- 172.21.15.45:0/2757681899 learned_addr learned my addr 172.21.15.45:0/2757681899 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b6a59c640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6c068370 0x7f4b6c197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b63fff640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b6c068d10 0x7f4b6c1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b6ad9d640 1 -- 172.21.15.45:0/2757681899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6c068370 msgr2=0x7f4b6c197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b6ad9d640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6c068370 0x7f4b6c197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b6ad9d640 1 -- 172.21.15.45:0/2757681899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b6c068d10 msgr2=0x7f4b6c1976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b6ad9d640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b6c068d10 0x7f4b6c1976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b6ad9d640 1 -- 172.21.15.45:0/2757681899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b54009560 con 0x7f4b6c106e00 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b6ad9d640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106e00 0x7f4b6c19ba30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400ea60 tx=0x7f4b6400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b637fe640 1 -- 172.21.15.45:0/2757681899 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400ce40 con 0x7f4b6c106e00 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b637fe640 1 -- 172.21.15.45:0/2757681899 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b64005020 con 0x7f4b6c106e00 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b70885640 1 -- 172.21.15.45:0/2757681899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6c19c640 con 0x7f4b6c106e00 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b637fe640 1 -- 172.21.15.45:0/2757681899 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64010670 con 0x7f4b6c106e00 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.110+0000 7f4b70885640 1 -- 172.21.15.45:0/2757681899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c1a3ec0 con 0x7f4b6c106e00 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.113+0000 7f4b637fe640 1 -- 172.21.15.45:0/2757681899 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f4b64005450 con 0x7f4b6c106e00 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.113+0000 7f4b73645740 1 Processor -- start 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b73645740 1 -- start start 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b637fe640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4b580a14f0 0x7f4b580a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b73645740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b73645740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7df620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b73645740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7e1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b73645740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6a741040 con 0x55ae6a7dfb60 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b73645740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6a73e730 con 0x55ae6a7d24a0 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b73645740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6a73a020 con 0x55ae6a7d1b80 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b63fff640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4b580a14f0 0x7f4b580a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b637fe640 1 -- 172.21.15.45:0/2757681899 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4b64014070 con 0x7f4b6c106e00 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b6ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7e1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b6ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7e1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58146/0 (socket says 172.21.15.45:58146) 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b6ad9d640 1 -- 172.21.15.45:0/568553570 learned_addr learned my addr 172.21.15.45:0/568553570 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b63fff640 1 --2- 172.21.15.45:0/568553570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7df620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b6a59c640 1 --2- 172.21.15.45:0/568553570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b6ad9d640 1 -- 172.21.15.45:0/568553570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 msgr2=0x55ae6a7d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b6ad9d640 1 --2- 172.21.15.45:0/568553570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b6ad9d640 1 -- 172.21.15.45:0/568553570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7d24a0 msgr2=0x55ae6a7df620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b6ad9d640 1 --2- 172.21.15.45:0/568553570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7df620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b6ad9d640 1 -- 172.21.15.45:0/568553570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae6a731e40 con 0x55ae6a7dfb60 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.114+0000 7f4b63fff640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4b580a14f0 0x7f4b580a39b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4b54006010 tx=0x7f4b54004f00 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.115+0000 7f4b6ad9d640 1 --2- 172.21.15.45:0/568553570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7e1f50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4b640a2060 tx=0x7f4b64099e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4bdbc495646822 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.115+0000 7f4b60ff9640 1 -- 172.21.15.45:0/568553570 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6409e120 con 0x55ae6a7dfb60 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.115+0000 7f4b60ff9640 1 -- 172.21.15.45:0/568553570 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b6409e350 con 0x55ae6a7dfb60 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.115+0000 7f4b60ff9640 1 -- 172.21.15.45:0/568553570 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640cadd0 con 0x55ae6a7dfb60 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.115+0000 7f4b73645740 1 -- 172.21.15.45:0/568553570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7dfb60 msgr2=0x55ae6a7e1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.115+0000 7f4b73645740 1 --2- 172.21.15.45:0/568553570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7e1f50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4b640a2060 tx=0x7f4b64099e70 comp rx=0 tx=0).stop 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.115+0000 7f4b73645740 1 -- 172.21.15.45:0/568553570 shutdown_connections 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.115+0000 7f4b73645740 1 --2- 172.21.15.45:0/568553570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7e1f50 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.115+0000 7f4b73645740 1 --2- 172.21.15.45:0/568553570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7df620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.115+0000 7f4b73645740 1 --2- 172.21.15.45:0/568553570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.115+0000 7f4b73645740 1 -- 172.21.15.45:0/568553570 >> 172.21.15.45:0/568553570 conn(0x55ae6a7cc980 msgr2=0x55ae6a7ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b73645740 1 -- 172.21.15.45:0/568553570 shutdown_connections 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b73645740 1 -- 172.21.15.45:0/568553570 wait complete. 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b73645740 1 -- start start 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b73645740 1 Processor -- start 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b73645740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7dca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b73645740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7dcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b73645740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7dd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b73645740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6a6f31b0 con 0x55ae6a7d24a0 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b73645740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6a6f2e50 con 0x55ae6a7dfb60 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b73645740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6a6b8090 con 0x55ae6a7d1b80 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b73645740 10 client.0 _ll_register_callbacks cb 0x55ae6a6dda90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7dcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7dcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58154/0 (socket says 172.21.15.45:58154) 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b63fff640 1 -- 172.21.15.45:0/4104602424 learned_addr learned my addr 172.21.15.45:0/4104602424 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.116+0000 7f4b6ad9d640 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7dd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b6a59c640 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7dca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b63fff640 1 -- 172.21.15.45:0/4104602424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 msgr2=0x55ae6a7dca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b63fff640 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7dca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b63fff640 1 -- 172.21.15.45:0/4104602424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 msgr2=0x55ae6a7dd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b63fff640 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7dd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b63fff640 1 -- 172.21.15.45:0/4104602424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c1a3ec0 con 0x55ae6a7d24a0 2024-01-31T05:28:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b6a59c640 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7dca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:28:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b63fff640 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7dcfd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4b54035000 tx=0x7f4b5402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9503f7ce1af66ff1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:28:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b51ffb640 1 -- 172.21.15.45:0/4104602424 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54048070 con 0x55ae6a7d24a0 2024-01-31T05:28:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b73645740 1 -- 172.21.15.45:0/4104602424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae6a733010 con 0x55ae6a7d24a0 2024-01-31T05:28:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b51ffb640 1 -- 172.21.15.45:0/4104602424 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4b54005000 con 0x55ae6a7d24a0 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b51ffb640 1 -- 172.21.15.45:0/4104602424 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b5402cc20 con 0x55ae6a7d24a0 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b73645740 10 client.15843 fetch_fsmap learned FSMap version 27 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b73645740 1 -- 172.21.15.45:0/4104602424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ae6a7386a0 con 0x55ae6a7d24a0 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.117+0000 7f4b51ffb640 1 -- 172.21.15.45:0/4104602424 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5405b020 con 0x55ae6a7d24a0 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.118+0000 7f4b51ffb640 1 -- 172.21.15.45:0/4104602424 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4b5403c030 con 0x55ae6a7d24a0 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.118+0000 7f4b73645740 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.118+0000 7f4b73645740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.118+0000 7f4b73645740 20 client.15843 populate_metadata read hostname 'smithi045' 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.118+0000 7f4b73645740 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ae6a7ef880 0x55ae6a7f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.118+0000 7f4b73645740 4 client.15843 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.118+0000 7f4b73645740 1 -- 172.21.15.45:0/4104602424 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae6a732d30 con 0x55ae6a7ef880 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.118+0000 7f4b6a59c640 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ae6a7ef880 0x55ae6a7f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.118+0000 7f4b6a59c640 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ae6a7ef880 0x55ae6a7f1ca0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=904957853cfebafd server_cookie=c0d0b1c4874e4692 in_seq=0 out_seq=0 2024-01-31T05:28:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.118+0000 7f4b51ffb640 10 client.15843 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b51ffb640 1 -- 172.21.15.45:0/4104602424 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ae6a732d30 con 0x55ae6a7ef880 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b51ffb640 10 client.15843 handle_command_reply: tid=0 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b73645740 1 client.15843 shutdown 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b73645740 1 -- 172.21.15.45:0/4104602424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 msgr2=0x55ae6a7dcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b73645740 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7dcfd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4b54035000 tx=0x7f4b5402bc30 comp rx=0 tx=0).stop 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b73645740 1 -- 172.21.15.45:0/4104602424 shutdown_connections 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b73645740 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ae6a7ef880 0x55ae6a7f1ca0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b51ffb640 0 client.15843 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b73645740 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7dcfd0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b73645740 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7dd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b73645740 1 --2- 172.21.15.45:0/4104602424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7dca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b73645740 1 -- 172.21.15.45:0/4104602424 >> 172.21.15.45:0/4104602424 conn(0x55ae6a7cc980 msgr2=0x55ae6a7ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b73645740 1 -- 172.21.15.45:0/4104602424 shutdown_connections 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b73645740 1 -- 172.21.15.45:0/4104602424 wait complete. 2024-01-31T05:28:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.119+0000 7f4b73645740 20 client.15843 trim_cache size 0 max 16384 2024-01-31T05:28:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.133+0000 7f4b73645740 1 Processor -- start 2024-01-31T05:28:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.133+0000 7f4b73645740 1 -- start start 2024-01-31T05:28:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.133+0000 7f4b73645740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7df1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.133+0000 7f4b73645740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7dd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.133+0000 7f4b73645740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.133+0000 7f4b73645740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6a7d2d60 con 0x55ae6a7d24a0 2024-01-31T05:28:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.133+0000 7f4b73645740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6a7f6ef0 con 0x55ae6a7dfb60 2024-01-31T05:28:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.133+0000 7f4b73645740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6a7e22d0 con 0x55ae6a7d1b80 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.133+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7dd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.133+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7dd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58166/0 (socket says 172.21.15.45:58166) 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.133+0000 7f4b63fff640 1 -- 172.21.15.45:0/1626452243 learned_addr learned my addr 172.21.15.45:0/1626452243 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.133+0000 7f4b6ad9d640 1 --2- 172.21.15.45:0/1626452243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b6a59c640 1 --2- 172.21.15.45:0/1626452243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7df1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b63fff640 1 -- 172.21.15.45:0/1626452243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 msgr2=0x55ae6a7df1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b63fff640 1 --2- 172.21.15.45:0/1626452243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7df1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b63fff640 1 -- 172.21.15.45:0/1626452243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 msgr2=0x55ae6a7dba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b63fff640 1 --2- 172.21.15.45:0/1626452243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7dba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b63fff640 1 -- 172.21.15.45:0/1626452243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae6a6d5030 con 0x55ae6a7d24a0 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b6a59c640 1 --2- 172.21.15.45:0/1626452243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7df1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b63fff640 1 --2- 172.21.15.45:0/1626452243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7dd330 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4b54003d30 tx=0x7f4b54005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5abdbbe55237ca4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b53fff640 1 -- 172.21.15.45:0/1626452243 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54048070 con 0x55ae6a7d24a0 2024-01-31T05:28:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b53fff640 1 -- 172.21.15.45:0/1626452243 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5405b030 con 0x55ae6a7d24a0 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b53fff640 1 -- 172.21.15.45:0/1626452243 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54034440 con 0x55ae6a7d24a0 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b73645740 1 -- 172.21.15.45:0/1626452243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 msgr2=0x55ae6a7dd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.134+0000 7f4b73645740 1 --2- 172.21.15.45:0/1626452243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7dd330 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4b54003d30 tx=0x7f4b54005b50 comp rx=0 tx=0).stop 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 -- 172.21.15.45:0/1626452243 shutdown_connections 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 --2- 172.21.15.45:0/1626452243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7dd330 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 --2- 172.21.15.45:0/1626452243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 --2- 172.21.15.45:0/1626452243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7df1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 -- 172.21.15.45:0/1626452243 >> 172.21.15.45:0/1626452243 conn(0x55ae6a7cc980 msgr2=0x55ae6a7db360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 -- 172.21.15.45:0/1626452243 shutdown_connections 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 -- 172.21.15.45:0/1626452243 wait complete. 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 -- start start 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 Processor -- start 2024-01-31T05:28:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7e4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7e5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6a7e58a0 con 0x55ae6a7d24a0 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6a7e5a10 con 0x55ae6a7dfb60 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6a7e5bd0 con 0x55ae6a7d1b80 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b73645740 10 client.0 _ll_register_callbacks cb 0x7f4b64096540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7e4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7e4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58168/0 (socket says 172.21.15.45:58168) 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b63fff640 1 -- 172.21.15.45:0/3330309610 learned_addr learned my addr 172.21.15.45:0/3330309610 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b6ad9d640 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7e5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.135+0000 7f4b6a59c640 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b63fff640 1 -- 172.21.15.45:0/3330309610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 msgr2=0x55ae6a7e68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b63fff640 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7e68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b63fff640 1 -- 172.21.15.45:0/3330309610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 msgr2=0x55ae6a7e5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b63fff640 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7e5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b63fff640 1 -- 172.21.15.45:0/3330309610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae6a7386a0 con 0x55ae6a7d24a0 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b63fff640 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7e4e20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009900 tx=0x7f4b540052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09da85f1ac08af server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b51ffb640 1 -- 172.21.15.45:0/3330309610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54048040 con 0x55ae6a7d24a0 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b51ffb640 1 -- 172.21.15.45:0/3330309610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4b5405c080 con 0x55ae6a7d24a0 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b73645740 1 -- 172.21.15.45:0/3330309610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b6409e350 con 0x55ae6a7d24a0 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b51ffb640 1 -- 172.21.15.45:0/3330309610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b54003ea0 con 0x55ae6a7d24a0 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b73645740 10 client.15849 fetch_fsmap learned FSMap version 27 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b73645740 1 -- 172.21.15.45:0/3330309610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ae6a6faa50 con 0x55ae6a7d24a0 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.136+0000 7f4b51ffb640 1 -- 172.21.15.45:0/3330309610 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b6409e350 con 0x55ae6a7d24a0 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.137+0000 7f4b51ffb640 1 -- 172.21.15.45:0/3330309610 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4b5403c070 con 0x55ae6a7d24a0 2024-01-31T05:28:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.137+0000 7f4b73645740 10 client.15849 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:28:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.137+0000 7f4b73645740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:28:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.137+0000 7f4b73645740 20 client.15849 populate_metadata read hostname 'smithi045' 2024-01-31T05:28:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.137+0000 7f4b73645740 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ae6a846ce0 0x55ae6a8490d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.137+0000 7f4b73645740 4 client.15849 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:28:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.137+0000 7f4b73645740 1 -- 172.21.15.45:0/3330309610 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ae6a439600 con 0x55ae6a846ce0 2024-01-31T05:28:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.137+0000 7f4b6a59c640 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ae6a846ce0 0x55ae6a8490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.137+0000 7f4b51ffb640 10 client.15849 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.137+0000 7f4b6a59c640 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ae6a846ce0 0x55ae6a8490d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d0883a8600a8bb server_cookie=a115c9de369ba698 in_seq=0 out_seq=0 2024-01-31T05:28:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b51ffb640 1 -- 172.21.15.45:0/3330309610 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ae6a439600 con 0x55ae6a846ce0 2024-01-31T05:28:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b51ffb640 10 client.15849 handle_command_reply: tid=0 2024-01-31T05:28:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b73645740 1 client.15849 shutdown 2024-01-31T05:28:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b73645740 1 -- 172.21.15.45:0/3330309610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 msgr2=0x55ae6a7e4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b73645740 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7e4e20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009900 tx=0x7f4b540052b0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b73645740 1 -- 172.21.15.45:0/3330309610 shutdown_connections 2024-01-31T05:28:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b73645740 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ae6a846ce0 0x55ae6a8490d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b73645740 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae6a7d24a0 0x55ae6a7e4e20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b73645740 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae6a7dfb60 0x55ae6a7e5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b51ffb640 0 client.15849 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b73645740 1 --2- 172.21.15.45:0/3330309610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6a7d1b80 0x55ae6a7e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b73645740 1 -- 172.21.15.45:0/3330309610 >> 172.21.15.45:0/3330309610 conn(0x55ae6a7cc980 msgr2=0x55ae6a7ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b73645740 1 -- 172.21.15.45:0/3330309610 shutdown_connections 2024-01-31T05:28:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b73645740 1 -- 172.21.15.45:0/3330309610 wait complete. 2024-01-31T05:28:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.148+0000 7f4b73645740 20 client.15849 trim_cache size 0 max 16384 2024-01-31T05:28:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:28:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:28:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:28:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:28:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.149+0000 7f4b70885640 1 -- 172.21.15.45:0/2757681899 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4b580a14f0 msgr2=0x7f4b580a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.149+0000 7f4b70885640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4b580a14f0 0x7f4b580a39b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4b54006010 tx=0x7f4b54004f00 comp rx=0 tx=0).stop 2024-01-31T05:28:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.149+0000 7f4b70885640 1 -- 172.21.15.45:0/2757681899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106e00 msgr2=0x7f4b6c19ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.149+0000 7f4b70885640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106e00 0x7f4b6c19ba30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400ea60 tx=0x7f4b6400ef20 comp rx=0 tx=0).stop 2024-01-31T05:28:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.149+0000 7f4b70885640 1 -- 172.21.15.45:0/2757681899 shutdown_connections 2024-01-31T05:28:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.149+0000 7f4b70885640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4b580a14f0 0x7f4b580a39b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.149+0000 7f4b70885640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b6c106e00 0x7f4b6c19ba30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.149+0000 7f4b70885640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b6c068d10 0x7f4b6c1976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.149+0000 7f4b70885640 1 --2- 172.21.15.45:0/2757681899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6c068370 0x7f4b6c197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.149+0000 7f4b70885640 1 -- 172.21.15.45:0/2757681899 >> 172.21.15.45:0/2757681899 conn(0x7f4b6c076a40 msgr2=0x7f4b6c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.149+0000 7f4b70885640 1 -- 172.21.15.45:0/2757681899 shutdown_connections 2024-01-31T05:28:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:18.149+0000 7f4b70885640 1 -- 172.21.15.45:0/2757681899 wait complete. 2024-01-31T05:28:18.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:18 smithi156 ceph-mon[35720]: pgmap v173: 97 pgs: 97 active+clean; 339 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 150 op/s 2024-01-31T05:28:18.783 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 11.78 MB/sec execute 138 sec latency 12609.850 ms 2024-01-31T05:28:18.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:18 smithi045 ceph-mon[31585]: pgmap v173: 97 pgs: 97 active+clean; 339 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 150 op/s 2024-01-31T05:28:19.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:18 smithi186 ceph-mon[35659]: pgmap v173: 97 pgs: 97 active+clean; 339 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 150 op/s 2024-01-31T05:28:19.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 11.69 MB/sec execute 139 sec latency 13609.941 ms 2024-01-31T05:28:19.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:19 smithi045 ceph-mon[31585]: pgmap v175: 97 pgs: 97 active+clean; 339 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.2 MiB/s wr, 82 op/s 2024-01-31T05:28:19.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:19 smithi156 ceph-mon[35720]: pgmap v175: 97 pgs: 97 active+clean; 339 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.2 MiB/s wr, 82 op/s 2024-01-31T05:28:20.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:19 smithi186 ceph-mon[35659]: pgmap v175: 97 pgs: 97 active+clean; 339 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.2 MiB/s wr, 82 op/s 2024-01-31T05:28:20.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 565388 11.61 MB/sec execute 140 sec latency 14610.029 ms 2024-01-31T05:28:21.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 566353 11.56 MB/sec execute 141 sec latency 15148.050 ms 2024-01-31T05:28:22.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 568566 11.58 MB/sec execute 142 sec latency 87.812 ms 2024-01-31T05:28:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:22 smithi186 ceph-mon[35659]: pgmap v176: 97 pgs: 97 active+clean; 339 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.2 MiB/s wr, 82 op/s 2024-01-31T05:28:22.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:22 smithi045 ceph-mon[31585]: pgmap v176: 97 pgs: 97 active+clean; 339 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.2 MiB/s wr, 82 op/s 2024-01-31T05:28:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:22 smithi156 ceph-mon[35720]: pgmap v176: 97 pgs: 97 active+clean; 339 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.2 MiB/s wr, 82 op/s 2024-01-31T05:28:23.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 570544 11.54 MB/sec execute 143 sec latency 18.383 ms 2024-01-31T05:28:24.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 572598 11.56 MB/sec execute 144 sec latency 17.939 ms 2024-01-31T05:28:24.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:24 smithi186 ceph-mon[35659]: pgmap v177: 97 pgs: 97 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.6 MiB/s wr, 125 op/s 2024-01-31T05:28:24.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:24 smithi045 ceph-mon[31585]: pgmap v177: 97 pgs: 97 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.6 MiB/s wr, 125 op/s 2024-01-31T05:28:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:24 smithi156 ceph-mon[35720]: pgmap v177: 97 pgs: 97 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.6 MiB/s wr, 125 op/s 2024-01-31T05:28:25.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 574669 11.55 MB/sec execute 145 sec latency 135.027 ms 2024-01-31T05:28:26.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 576558 11.54 MB/sec execute 146 sec latency 14.388 ms 2024-01-31T05:28:26.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:26 smithi045 ceph-mon[31585]: pgmap v178: 97 pgs: 97 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.6 MiB/s wr, 125 op/s 2024-01-31T05:28:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:26 smithi156 ceph-mon[35720]: pgmap v178: 97 pgs: 97 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.6 MiB/s wr, 125 op/s 2024-01-31T05:28:27.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:26 smithi186 ceph-mon[35659]: pgmap v178: 97 pgs: 97 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.6 MiB/s wr, 125 op/s 2024-01-31T05:28:27.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 578697 11.55 MB/sec execute 147 sec latency 102.078 ms 2024-01-31T05:28:28.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 580483 11.53 MB/sec execute 148 sec latency 22.870 ms 2024-01-31T05:28:28.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:28 smithi045 ceph-mon[31585]: pgmap v179: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-01-31T05:28:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:28 smithi156 ceph-mon[35720]: pgmap v179: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-01-31T05:28:29.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:28 smithi186 ceph-mon[35659]: pgmap v179: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-01-31T05:28:29.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 582833 11.55 MB/sec execute 149 sec latency 21.263 ms 2024-01-31T05:28:30.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 584391 11.51 MB/sec execute 150 sec latency 70.185 ms 2024-01-31T05:28:30.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:30 smithi045 ceph-mon[31585]: pgmap v180: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 171 op/s 2024-01-31T05:28:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:30 smithi156 ceph-mon[35720]: pgmap v180: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 171 op/s 2024-01-31T05:28:31.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:30 smithi186 ceph-mon[35659]: pgmap v180: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 171 op/s 2024-01-31T05:28:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.237+0000 7fee45c2a640 1 -- 172.21.15.45:0/2825792492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee40100800 msgr2=0x7fee4010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.237+0000 7fee45c2a640 1 --2- 172.21.15.45:0/2825792492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee40100800 0x7fee4010ca20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fee38009ca0 tx=0x7fee3802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.237+0000 7fee45c2a640 1 -- 172.21.15.45:0/2825792492 shutdown_connections 2024-01-31T05:28:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.237+0000 7fee45c2a640 1 --2- 172.21.15.45:0/2825792492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee40100800 0x7fee4010ca20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.237+0000 7fee45c2a640 1 --2- 172.21.15.45:0/2825792492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee400ffe40 0x7fee401002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.237+0000 7fee45c2a640 1 --2- 172.21.15.45:0/2825792492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee40104b90 0x7fee40104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.237+0000 7fee45c2a640 1 -- 172.21.15.45:0/2825792492 >> 172.21.15.45:0/2825792492 conn(0x7fee40076a40 msgr2=0x7fee40076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.237+0000 7fee45c2a640 1 -- 172.21.15.45:0/2825792492 shutdown_connections 2024-01-31T05:28:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.237+0000 7fee45c2a640 1 -- 172.21.15.45:0/2825792492 wait complete. 2024-01-31T05:28:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.238+0000 7fee45c2a640 1 Processor -- start 2024-01-31T05:28:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.238+0000 7fee45c2a640 1 -- start start 2024-01-31T05:28:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.238+0000 7fee45c2a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee400ffe40 0x7fee401971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.238+0000 7fee45c2a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee40100800 0x7fee40197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.238+0000 7fee45c2a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee40104b90 0x7fee4019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.238+0000 7fee45c2a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4019c030 con 0x7fee400ffe40 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.238+0000 7fee45c2a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4019c1a0 con 0x7fee40100800 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.238+0000 7fee45c2a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4019c360 con 0x7fee40104b90 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.238+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee400ffe40 0x7fee401971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee400ffe40 0x7fee401971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52740/0 (socket says 172.21.15.45:52740) 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3f7fe640 1 -- 172.21.15.45:0/1708304654 learned_addr learned my addr 172.21.15.45:0/1708304654 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3ffff640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee40104b90 0x7fee4019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3effd640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee40100800 0x7fee40197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3f7fe640 1 -- 172.21.15.45:0/1708304654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee40104b90 msgr2=0x7fee4019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3f7fe640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee40104b90 0x7fee4019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3f7fe640 1 -- 172.21.15.45:0/1708304654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee40100800 msgr2=0x7fee40197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3f7fe640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee40100800 0x7fee40197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3f7fe640 1 -- 172.21.15.45:0/1708304654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee38009900 con 0x7fee400ffe40 2024-01-31T05:28:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3ffff640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee40104b90 0x7fee4019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:28:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3f7fe640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee400ffe40 0x7fee401971c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fee3400e9f0 tx=0x7fee3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3cff9640 1 -- 172.21.15.45:0/1708304654 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34009700 con 0x7fee400ffe40 2024-01-31T05:28:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3cff9640 1 -- 172.21.15.45:0/1708304654 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee3400cd80 con 0x7fee400ffe40 2024-01-31T05:28:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee45c2a640 1 -- 172.21.15.45:0/1708304654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4019c640 con 0x7fee400ffe40 2024-01-31T05:28:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee3cff9640 1 -- 172.21.15.45:0/1708304654 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34018860 con 0x7fee400ffe40 2024-01-31T05:28:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.239+0000 7fee45c2a640 1 -- 172.21.15.45:0/1708304654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee401a3ec0 con 0x7fee400ffe40 2024-01-31T05:28:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.241+0000 7fee3cff9640 1 -- 172.21.15.45:0/1708304654 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fee340189c0 con 0x7fee400ffe40 2024-01-31T05:28:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee489f2740 1 Processor -- start 2024-01-31T05:28:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee489f2740 1 -- start start 2024-01-31T05:28:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee3cff9640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fee280a14f0 0x7fee280a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee489f2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 0x5644a2cab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee489f2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab980 0x5644a2cb4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee489f2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cb4750 0x5644a2cb6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee489f2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a2c1a040 con 0x5644a2cab060 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee489f2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a2c17730 con 0x5644a2cb4750 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee489f2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a2c13020 con 0x5644a2cab980 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee3ffff640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fee280a14f0 0x7fee280a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 0x5644a2cab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee3effd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab980 0x5644a2cb4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee3cff9640 1 -- 172.21.15.45:0/1708304654 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fee34014070 con 0x7fee400ffe40 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee3ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cb4750 0x5644a2cb6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee3effd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab980 0x5644a2cb4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:55810/0 (socket says 172.21.15.45:55810) 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee3effd640 1 -- 172.21.15.45:0/3848045069 learned_addr learned my addr 172.21.15.45:0/3848045069 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.242+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 0x5644a2cab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52750/0 (socket says 172.21.15.45:52750) 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.243+0000 7fee3effd640 1 -- 172.21.15.45:0/3848045069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cb4750 msgr2=0x5644a2cb6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.243+0000 7fee3effd640 1 --2- 172.21.15.45:0/3848045069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cb4750 0x5644a2cb6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.243+0000 7fee3effd640 1 -- 172.21.15.45:0/3848045069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 msgr2=0x5644a2cab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.243+0000 7fee3effd640 1 --2- 172.21.15.45:0/3848045069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 0x5644a2cab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.243+0000 7fee3effd640 1 -- 172.21.15.45:0/3848045069 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644a2c0ae40 con 0x5644a2cab980 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.243+0000 7fee3ffff640 1 --2- 172.21.15.45:0/3848045069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cb4750 0x5644a2cb6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.243+0000 7fee3f7fe640 1 --2- 172.21.15.45:0/3848045069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 0x5644a2cab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.243+0000 7fee3ffff640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fee280a14f0 0x7fee280a39b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fee38004af0 tx=0x7fee38005ef0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.243+0000 7fee3effd640 1 --2- 172.21.15.45:0/3848045069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab980 0x5644a2cb4210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fee30009c20 tx=0x7fee300117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb82b5c5f6da1df server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.243+0000 7fee25ffb640 1 -- 172.21.15.45:0/3848045069 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30009720 con 0x5644a2cab980 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.243+0000 7fee25ffb640 1 -- 172.21.15.45:0/3848045069 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee30012910 con 0x5644a2cab980 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.243+0000 7fee25ffb640 1 -- 172.21.15.45:0/3848045069 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3000fae0 con 0x5644a2cab980 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.244+0000 7fee489f2740 1 -- 172.21.15.45:0/3848045069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab980 msgr2=0x5644a2cb4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.244+0000 7fee489f2740 1 --2- 172.21.15.45:0/3848045069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab980 0x5644a2cb4210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fee30009c20 tx=0x7fee300117b0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.244+0000 7fee489f2740 1 -- 172.21.15.45:0/3848045069 shutdown_connections 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.244+0000 7fee489f2740 1 --2- 172.21.15.45:0/3848045069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cb4750 0x5644a2cb6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.244+0000 7fee489f2740 1 --2- 172.21.15.45:0/3848045069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab980 0x5644a2cb4210 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.244+0000 7fee489f2740 1 --2- 172.21.15.45:0/3848045069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 0x5644a2cab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.244+0000 7fee489f2740 1 -- 172.21.15.45:0/3848045069 >> 172.21.15.45:0/3848045069 conn(0x5644a2ca5980 msgr2=0x5644a2ca5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.244+0000 7fee489f2740 1 -- 172.21.15.45:0/3848045069 shutdown_connections 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.244+0000 7fee489f2740 1 -- 172.21.15.45:0/3848045069 wait complete. 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.244+0000 7fee489f2740 1 -- start start 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.244+0000 7fee489f2740 1 Processor -- start 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee489f2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab060 0x5644a2cb1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee489f2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cab980 0x5644a2cb1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee489f2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cb4750 0x5644a2cb20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee489f2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a2bcc1b0 con 0x5644a2cb4750 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee489f2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a2bcbe50 con 0x5644a2cab980 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee489f2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a2b91090 con 0x5644a2cab060 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee489f2740 10 client.0 _ll_register_callbacks cb 0x5644a2bb6a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee3ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cb4750 0x5644a2cb20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee3ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cb4750 0x5644a2cb20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52754/0 (socket says 172.21.15.45:52754) 2024-01-31T05:28:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee3ffff640 1 -- 172.21.15.45:0/1870662962 learned_addr learned my addr 172.21.15.45:0/1870662962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee3f7fe640 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab060 0x5644a2cb1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee3effd640 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cab980 0x5644a2cb1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee3ffff640 1 -- 172.21.15.45:0/1870662962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab060 msgr2=0x5644a2cb1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee3ffff640 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab060 0x5644a2cb1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee3ffff640 1 -- 172.21.15.45:0/1870662962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cab980 msgr2=0x5644a2cb1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee3ffff640 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cab980 0x5644a2cb1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee3ffff640 1 -- 172.21.15.45:0/1870662962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee30009880 con 0x5644a2cb4750 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee3ffff640 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cb4750 0x5644a2cb20a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fee38036000 tx=0x7fee3802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d652bcda4aa27e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee1effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee1effd640 1 -- 172.21.15.45:0/1870662962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38003b70 con 0x5644a2cb4750 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee1effd640 1 -- 172.21.15.45:0/1870662962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fee38003cd0 con 0x5644a2cb4750 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.245+0000 7fee489f2740 1 -- 172.21.15.45:0/1870662962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644a2c0c010 con 0x5644a2cb4750 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.246+0000 7fee1effd640 1 -- 172.21.15.45:0/1870662962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee38003e90 con 0x5644a2cb4750 2024-01-31T05:28:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.246+0000 7fee489f2740 10 client.15861 fetch_fsmap learned FSMap version 27 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.246+0000 7fee489f2740 1 -- 172.21.15.45:0/1870662962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5644a2c116a0 con 0x5644a2cb4750 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.246+0000 7fee1effd640 1 -- 172.21.15.45:0/1870662962 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee3805a020 con 0x5644a2cb4750 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.246+0000 7fee1effd640 1 -- 172.21.15.45:0/1870662962 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fee3804d9e0 con 0x5644a2cb4750 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.246+0000 7fee489f2740 10 client.15861 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.246+0000 7fee489f2740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.246+0000 7fee489f2740 20 client.15861 populate_metadata read hostname 'smithi045' 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.246+0000 7fee489f2740 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5644a2cc4490 0x5644a2cc68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.246+0000 7fee489f2740 4 client.15861 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.246+0000 7fee489f2740 1 -- 172.21.15.45:0/1870662962 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644a2c0bd30 con 0x5644a2cc4490 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.247+0000 7fee3f7fe640 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5644a2cc4490 0x5644a2cc68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.247+0000 7fee1effd640 10 client.15861 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.247+0000 7fee3f7fe640 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5644a2cc4490 0x5644a2cc68b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b20baa336f9123f server_cookie=796176a5f5e502bc in_seq=0 out_seq=0 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee1effd640 1 -- 172.21.15.45:0/1870662962 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644a2c0bd30 con 0x5644a2cc4490 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee1effd640 10 client.15861 handle_command_reply: tid=0 2024-01-31T05:28:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee489f2740 1 client.15861 shutdown 2024-01-31T05:28:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee489f2740 1 -- 172.21.15.45:0/1870662962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cb4750 msgr2=0x5644a2cb20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee489f2740 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cb4750 0x5644a2cb20a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fee38036000 tx=0x7fee3802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee489f2740 1 -- 172.21.15.45:0/1870662962 shutdown_connections 2024-01-31T05:28:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee489f2740 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5644a2cc4490 0x5644a2cc68b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee489f2740 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cb4750 0x5644a2cb20a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee489f2740 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cab980 0x5644a2cb1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee489f2740 1 --2- 172.21.15.45:0/1870662962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab060 0x5644a2cb1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee1effd640 0 client.15861 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee489f2740 1 -- 172.21.15.45:0/1870662962 >> 172.21.15.45:0/1870662962 conn(0x5644a2ca5980 msgr2=0x5644a2ca76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee489f2740 1 -- 172.21.15.45:0/1870662962 shutdown_connections 2024-01-31T05:28:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee489f2740 1 -- 172.21.15.45:0/1870662962 wait complete. 2024-01-31T05:28:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.248+0000 7fee489f2740 20 client.15861 trim_cache size 0 max 16384 2024-01-31T05:28:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee489f2740 1 Processor -- start 2024-01-31T05:28:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee489f2740 1 -- start start 2024-01-31T05:28:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee489f2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab060 0x5644a2cb1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee489f2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab980 0x5644a2cbb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee489f2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cb4750 0x5644a2cbb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee489f2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a2cb6ec0 con 0x5644a2cab980 2024-01-31T05:28:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee489f2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a2ca57b0 con 0x5644a2cb4750 2024-01-31T05:28:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee489f2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a2cafb70 con 0x5644a2cab060 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee3effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab980 0x5644a2cbb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee3effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab980 0x5644a2cbb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52758/0 (socket says 172.21.15.45:52758) 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee3effd640 1 -- 172.21.15.45:0/3856244932 learned_addr learned my addr 172.21.15.45:0/3856244932 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee3f7fe640 1 --2- 172.21.15.45:0/3856244932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab060 0x5644a2cb1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee3ffff640 1 --2- 172.21.15.45:0/3856244932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cb4750 0x5644a2cbb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee3effd640 1 -- 172.21.15.45:0/3856244932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab060 msgr2=0x5644a2cb1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee3effd640 1 --2- 172.21.15.45:0/3856244932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab060 0x5644a2cb1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee3effd640 1 -- 172.21.15.45:0/3856244932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cb4750 msgr2=0x5644a2cbb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee3effd640 1 --2- 172.21.15.45:0/3856244932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cb4750 0x5644a2cbb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.262+0000 7fee3effd640 1 -- 172.21.15.45:0/3856244932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644a2bae030 con 0x5644a2cab980 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee3effd640 1 --2- 172.21.15.45:0/3856244932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab980 0x5644a2cbb440 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fee30009c20 tx=0x7fee30009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7784e9e3101c8775 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee257fa640 1 -- 172.21.15.45:0/3856244932 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3000f4c0 con 0x5644a2cab980 2024-01-31T05:28:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee257fa640 1 -- 172.21.15.45:0/3856244932 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee30009d10 con 0x5644a2cab980 2024-01-31T05:28:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee257fa640 1 -- 172.21.15.45:0/3856244932 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee300052d0 con 0x5644a2cab980 2024-01-31T05:28:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee489f2740 1 -- 172.21.15.45:0/3856244932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab980 msgr2=0x5644a2cbb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee489f2740 1 --2- 172.21.15.45:0/3856244932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab980 0x5644a2cbb440 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fee30009c20 tx=0x7fee30009700 comp rx=0 tx=0).stop 2024-01-31T05:28:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee489f2740 1 -- 172.21.15.45:0/3856244932 shutdown_connections 2024-01-31T05:28:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee489f2740 1 --2- 172.21.15.45:0/3856244932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab980 0x5644a2cbb440 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee489f2740 1 --2- 172.21.15.45:0/3856244932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cb4750 0x5644a2cbb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee489f2740 1 --2- 172.21.15.45:0/3856244932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cab060 0x5644a2cb1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee489f2740 1 -- 172.21.15.45:0/3856244932 >> 172.21.15.45:0/3856244932 conn(0x5644a2ca5980 msgr2=0x5644a2ca4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee489f2740 1 -- 172.21.15.45:0/3856244932 shutdown_connections 2024-01-31T05:28:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.263+0000 7fee489f2740 1 -- 172.21.15.45:0/3856244932 wait complete. 2024-01-31T05:28:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee489f2740 1 -- start start 2024-01-31T05:28:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee489f2740 1 Processor -- start 2024-01-31T05:28:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee489f2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 0x5644a2cba6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee489f2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cab980 0x5644a2cbac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee489f2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cb4750 0x5644a2d1a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee489f2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a2d1aee0 con 0x5644a2cab060 2024-01-31T05:28:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee489f2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a2d1b020 con 0x5644a2cab980 2024-01-31T05:28:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee489f2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a2d1b1e0 con 0x5644a2cb4750 2024-01-31T05:28:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee489f2740 10 client.0 _ll_register_callbacks cb 0x7fee30009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:28:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 0x5644a2cba6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee3effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cab980 0x5644a2cbac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee3ffff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cb4750 0x5644a2d1a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 0x5644a2cba6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52766/0 (socket says 172.21.15.45:52766) 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.264+0000 7fee3f7fe640 1 -- 172.21.15.45:0/37767243 learned_addr learned my addr 172.21.15.45:0/37767243 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee3f7fe640 1 -- 172.21.15.45:0/37767243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cb4750 msgr2=0x5644a2d1a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee3f7fe640 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cb4750 0x5644a2d1a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee3f7fe640 1 -- 172.21.15.45:0/37767243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cab980 msgr2=0x5644a2cbac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee3f7fe640 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cab980 0x5644a2cbac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee3f7fe640 1 -- 172.21.15.45:0/37767243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3001e040 con 0x5644a2cab060 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee3effd640 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cab980 0x5644a2cbac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee3ffff640 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cb4750 0x5644a2d1a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee3f7fe640 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 0x5644a2cba6c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fee34099de0 tx=0x7fee3409a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0f870864991e9d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee1effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee1effd640 1 -- 172.21.15.45:0/37767243 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34010ad0 con 0x5644a2cab060 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee1effd640 1 -- 172.21.15.45:0/37767243 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fee340cd040 con 0x5644a2cab060 2024-01-31T05:28:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee489f2740 1 -- 172.21.15.45:0/37767243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee30012910 con 0x5644a2cab060 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee1effd640 1 -- 172.21.15.45:0/37767243 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee340967f0 con 0x5644a2cab060 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee489f2740 10 client.15867 fetch_fsmap learned FSMap version 27 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee489f2740 1 -- 172.21.15.45:0/37767243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5644a2bd3a50 con 0x5644a2cab060 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.265+0000 7fee1effd640 1 -- 172.21.15.45:0/37767243 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee30012910 con 0x5644a2cab060 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.266+0000 7fee1effd640 1 -- 172.21.15.45:0/37767243 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fee340add20 con 0x5644a2cab060 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.266+0000 7fee489f2740 10 client.15867 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.266+0000 7fee489f2740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.266+0000 7fee489f2740 20 client.15867 populate_metadata read hostname 'smithi045' 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.266+0000 7fee489f2740 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5644a2d1c9e0 0x5644a2d1ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.266+0000 7fee489f2740 4 client.15867 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.266+0000 7fee489f2740 1 -- 172.21.15.45:0/37767243 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5644a2912600 con 0x5644a2d1c9e0 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.266+0000 7fee3effd640 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5644a2d1c9e0 0x5644a2d1ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.266+0000 7fee1effd640 10 client.15867 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.266+0000 7fee3effd640 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5644a2d1c9e0 0x5644a2d1ee00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe90a3f299b89480 server_cookie=bd81910e14b12945 in_seq=0 out_seq=0 2024-01-31T05:28:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee1effd640 1 -- 172.21.15.45:0/37767243 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5644a2912600 con 0x5644a2d1c9e0 2024-01-31T05:28:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee1effd640 10 client.15867 handle_command_reply: tid=0 2024-01-31T05:28:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee489f2740 1 client.15867 shutdown 2024-01-31T05:28:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee489f2740 1 -- 172.21.15.45:0/37767243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 msgr2=0x5644a2cba6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee489f2740 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 0x5644a2cba6c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fee34099de0 tx=0x7fee3409a0e0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee489f2740 1 -- 172.21.15.45:0/37767243 shutdown_connections 2024-01-31T05:28:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee489f2740 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5644a2d1c9e0 0x5644a2d1ee00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee489f2740 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5644a2cb4750 0x5644a2d1a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee1effd640 0 client.15867 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee489f2740 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644a2cab980 0x5644a2cbac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee489f2740 1 --2- 172.21.15.45:0/37767243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5644a2cab060 0x5644a2cba6c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee489f2740 1 -- 172.21.15.45:0/37767243 >> 172.21.15.45:0/37767243 conn(0x5644a2ca5980 msgr2=0x5644a2ca4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee489f2740 1 -- 172.21.15.45:0/37767243 shutdown_connections 2024-01-31T05:28:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee489f2740 1 -- 172.21.15.45:0/37767243 wait complete. 2024-01-31T05:28:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.276+0000 7fee489f2740 20 client.15867 trim_cache size 0 max 16384 2024-01-31T05:28:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:28:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:28:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:28:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:28:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.277+0000 7fee45c2a640 1 -- 172.21.15.45:0/1708304654 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fee280a14f0 msgr2=0x7fee280a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.277+0000 7fee45c2a640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fee280a14f0 0x7fee280a39b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fee38004af0 tx=0x7fee38005ef0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.277+0000 7fee45c2a640 1 -- 172.21.15.45:0/1708304654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee400ffe40 msgr2=0x7fee401971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.277+0000 7fee45c2a640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee400ffe40 0x7fee401971c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fee3400e9f0 tx=0x7fee3400eeb0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.277+0000 7fee45c2a640 1 -- 172.21.15.45:0/1708304654 shutdown_connections 2024-01-31T05:28:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.277+0000 7fee45c2a640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fee280a14f0 0x7fee280a39b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.277+0000 7fee45c2a640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee40104b90 0x7fee4019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.277+0000 7fee45c2a640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee40100800 0x7fee40197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.277+0000 7fee45c2a640 1 --2- 172.21.15.45:0/1708304654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee400ffe40 0x7fee401971c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.277+0000 7fee45c2a640 1 -- 172.21.15.45:0/1708304654 >> 172.21.15.45:0/1708304654 conn(0x7fee40076a40 msgr2=0x7fee40068c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.277+0000 7fee45c2a640 1 -- 172.21.15.45:0/1708304654 shutdown_connections 2024-01-31T05:28:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:31.277+0000 7fee45c2a640 1 -- 172.21.15.45:0/1708304654 wait complete. 2024-01-31T05:28:31.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 586609 11.52 MB/sec execute 151 sec latency 13.067 ms 2024-01-31T05:28:32.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 588513 11.50 MB/sec execute 152 sec latency 110.079 ms 2024-01-31T05:28:32.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:32 smithi045 ceph-mon[31585]: pgmap v181: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 171 op/s 2024-01-31T05:28:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:32 smithi156 ceph-mon[35720]: pgmap v181: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 171 op/s 2024-01-31T05:28:33.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:32 smithi186 ceph-mon[35659]: pgmap v181: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 171 op/s 2024-01-31T05:28:33.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 590595 11.50 MB/sec execute 153 sec latency 15.426 ms 2024-01-31T05:28:34.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 593327 11.54 MB/sec execute 154 sec latency 15.350 ms 2024-01-31T05:28:34.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:34 smithi045 ceph-mon[31585]: pgmap v182: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 273 op/s 2024-01-31T05:28:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:34 smithi156 ceph-mon[35720]: pgmap v182: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 273 op/s 2024-01-31T05:28:35.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:34 smithi186 ceph-mon[35659]: pgmap v182: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 273 op/s 2024-01-31T05:28:35.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 595256 11.51 MB/sec execute 155 sec latency 25.245 ms 2024-01-31T05:28:36.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 597269 11.52 MB/sec execute 156 sec latency 90.078 ms 2024-01-31T05:28:36.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:36 smithi045 ceph-mon[31585]: pgmap v183: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-01-31T05:28:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:36 smithi156 ceph-mon[35720]: pgmap v183: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-01-31T05:28:37.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:36 smithi186 ceph-mon[35659]: pgmap v183: 97 pgs: 97 active+clean; 343 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-01-31T05:28:37.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 599269 11.50 MB/sec execute 157 sec latency 46.718 ms 2024-01-31T05:28:38.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 601324 11.51 MB/sec execute 158 sec latency 15.803 ms 2024-01-31T05:28:38.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:38 smithi186 ceph-mon[35659]: pgmap v184: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 293 op/s 2024-01-31T05:28:38.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:38 smithi045 ceph-mon[31585]: pgmap v184: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 293 op/s 2024-01-31T05:28:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:38 smithi156 ceph-mon[35720]: pgmap v184: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 293 op/s 2024-01-31T05:28:39.784 INFO:tasks.workunit.client.0.smithi045.stdout: 1 603931 11.54 MB/sec execute 159 sec latency 14.815 ms 2024-01-31T05:28:40.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 605665 11.51 MB/sec execute 160 sec latency 16.595 ms 2024-01-31T05:28:40.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:40 smithi186 ceph-mon[35659]: pgmap v185: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2024-01-31T05:28:40.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:40 smithi045 ceph-mon[31585]: pgmap v185: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2024-01-31T05:28:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:40 smithi156 ceph-mon[35720]: pgmap v185: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2024-01-31T05:28:41.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 607906 11.52 MB/sec execute 161 sec latency 19.620 ms 2024-01-31T05:28:42.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 609921 11.50 MB/sec execute 162 sec latency 108.849 ms 2024-01-31T05:28:42.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:42 smithi186 ceph-mon[35659]: pgmap v186: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2024-01-31T05:28:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:42 smithi045 ceph-mon[31585]: pgmap v186: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2024-01-31T05:28:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:42 smithi156 ceph-mon[35720]: pgmap v186: 97 pgs: 97 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2024-01-31T05:28:43.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 611897 11.50 MB/sec execute 163 sec latency 16.632 ms 2024-01-31T05:28:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.372+0000 7faa923d9640 1 -- 172.21.15.45:0/3795732828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa8c06a150 msgr2=0x7faa8c105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.372+0000 7faa923d9640 1 --2- 172.21.15.45:0/3795732828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa8c06a150 0x7faa8c105940 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7faa74009930 tx=0x7faa7402bc30 comp rx=0 tx=0).stop 2024-01-31T05:28:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.372+0000 7faa923d9640 1 -- 172.21.15.45:0/3795732828 shutdown_connections 2024-01-31T05:28:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.372+0000 7faa923d9640 1 --2- 172.21.15.45:0/3795732828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa8c06a150 0x7faa8c105940 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.372+0000 7faa923d9640 1 --2- 172.21.15.45:0/3795732828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa8c106020 0x7faa8c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.372+0000 7faa923d9640 1 --2- 172.21.15.45:0/3795732828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c0697a0 0x7faa8c069b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.372+0000 7faa923d9640 1 -- 172.21.15.45:0/3795732828 >> 172.21.15.45:0/3795732828 conn(0x7faa8c0769c0 msgr2=0x7faa8c076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.372+0000 7faa923d9640 1 -- 172.21.15.45:0/3795732828 shutdown_connections 2024-01-31T05:28:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.372+0000 7faa923d9640 1 -- 172.21.15.45:0/3795732828 wait complete. 2024-01-31T05:28:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.372+0000 7faa923d9640 1 Processor -- start 2024-01-31T05:28:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.372+0000 7faa923d9640 1 -- start start 2024-01-31T05:28:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.373+0000 7faa923d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa8c0697a0 0x7faa8c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.373+0000 7faa923d9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c106020 0x7faa8c1976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.373+0000 7faa923d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa8c19ba50 0x7faa8c19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.373+0000 7faa923d9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c19c440 con 0x7faa8c0697a0 2024-01-31T05:28:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.373+0000 7faa923d9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c19c5b0 con 0x7faa8c106020 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.373+0000 7faa923d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c19c720 con 0x7faa8c19ba50 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.373+0000 7faa8bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa8c0697a0 0x7faa8c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.373+0000 7faa90976640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa8c19ba50 0x7faa8c19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.373+0000 7faa8bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa8c0697a0 0x7faa8c197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53350/0 (socket says 172.21.15.45:53350) 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.373+0000 7faa8bfff640 1 -- 172.21.15.45:0/3396139483 learned_addr learned my addr 172.21.15.45:0/3396139483 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.373+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c106020 0x7faa8c1976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.374+0000 7faa90976640 1 -- 172.21.15.45:0/3396139483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c106020 msgr2=0x7faa8c1976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.374+0000 7faa90976640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c106020 0x7faa8c1976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.374+0000 7faa90976640 1 -- 172.21.15.45:0/3396139483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa8c0697a0 msgr2=0x7faa8c197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.374+0000 7faa90976640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa8c0697a0 0x7faa8c197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.374+0000 7faa90976640 1 -- 172.21.15.45:0/3396139483 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa74009560 con 0x7faa8c19ba50 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.374+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c106020 0x7faa8c1976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:28:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.374+0000 7faa90976640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa8c19ba50 0x7faa8c19bf00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faa84005fd0 tx=0x7faa8400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.374+0000 7faa897fa640 1 -- 172.21.15.45:0/3396139483 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8400cd80 con 0x7faa8c19ba50 2024-01-31T05:28:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.374+0000 7faa923d9640 1 -- 172.21.15.45:0/3396139483 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8c0ffed0 con 0x7faa8c19ba50 2024-01-31T05:28:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.374+0000 7faa897fa640 1 -- 172.21.15.45:0/3396139483 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa84005020 con 0x7faa8c19ba50 2024-01-31T05:28:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.374+0000 7faa897fa640 1 -- 172.21.15.45:0/3396139483 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84010670 con 0x7faa8c19ba50 2024-01-31T05:28:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.374+0000 7faa923d9640 1 -- 172.21.15.45:0/3396139483 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8c100350 con 0x7faa8c19ba50 2024-01-31T05:28:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.376+0000 7faa9519c740 1 Processor -- start 2024-01-31T05:28:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa897fa640 1 -- 172.21.15.45:0/3396139483 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7faa84005450 con 0x7faa8c19ba50 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa9519c740 1 -- start start 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa9519c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5060 0x5648cdce5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa9519c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5980 0x5648cdcee210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa9519c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 0x5648cdcf0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa9519c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648cdc54040 con 0x5648cdce5980 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa9519c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648cdc51730 con 0x5648cdcee750 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa9519c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648cdc4d020 con 0x5648cdce5060 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa8bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5980 0x5648cdcee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa8bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5980 0x5648cdcee210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53362/0 (socket says 172.21.15.45:53362) 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa8bfff640 1 -- 172.21.15.45:0/2525968466 learned_addr learned my addr 172.21.15.45:0/2525968466 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/2525968466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5060 0x5648cdce5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/2525968466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 0x5648cdcf0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa897fa640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7faa780a14f0 0x7faa780a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa8bfff640 1 -- 172.21.15.45:0/2525968466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5060 msgr2=0x5648cdce5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa8bfff640 1 --2- 172.21.15.45:0/2525968466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5060 0x5648cdce5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa8bfff640 1 -- 172.21.15.45:0/2525968466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 msgr2=0x5648cdcf0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa8bfff640 1 --2- 172.21.15.45:0/2525968466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 0x5648cdcf0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa8bfff640 1 -- 172.21.15.45:0/2525968466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648cdc44e40 con 0x5648cdce5980 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa90976640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7faa780a14f0 0x7faa780a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.377+0000 7faa897fa640 1 -- 172.21.15.45:0/3396139483 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8483+0+0 (secure 0 0 0) 0x7faa84014070 con 0x7faa8c19ba50 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.378+0000 7faa8bfff640 1 --2- 172.21.15.45:0/2525968466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5980 0x5648cdcee210 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009dc0 tx=0x7faa7c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2667f1564ac356f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.378+0000 7faa827fc640 1 -- 172.21.15.45:0/2525968466 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c005220 con 0x5648cdce5980 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.378+0000 7faa827fc640 1 -- 172.21.15.45:0/2525968466 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa7c004c60 con 0x5648cdce5980 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.378+0000 7faa827fc640 1 -- 172.21.15.45:0/2525968466 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c0197f0 con 0x5648cdce5980 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.379+0000 7faa9519c740 1 -- 172.21.15.45:0/2525968466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5980 msgr2=0x5648cdcee210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.379+0000 7faa90976640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7faa780a14f0 0x7faa780a39b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa840cad90 tx=0x7faa840699e0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.379+0000 7faa9519c740 1 --2- 172.21.15.45:0/2525968466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5980 0x5648cdcee210 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009dc0 tx=0x7faa7c011810 comp rx=0 tx=0).stop 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.379+0000 7faa9519c740 1 -- 172.21.15.45:0/2525968466 shutdown_connections 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.379+0000 7faa9519c740 1 --2- 172.21.15.45:0/2525968466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5980 0x5648cdcee210 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.379+0000 7faa9519c740 1 --2- 172.21.15.45:0/2525968466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 0x5648cdcf0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.379+0000 7faa9519c740 1 --2- 172.21.15.45:0/2525968466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5060 0x5648cdce5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.379+0000 7faa9519c740 1 -- 172.21.15.45:0/2525968466 >> 172.21.15.45:0/2525968466 conn(0x5648cdcdf980 msgr2=0x5648cdcdfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.379+0000 7faa9519c740 1 -- 172.21.15.45:0/2525968466 shutdown_connections 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.379+0000 7faa9519c740 1 -- 172.21.15.45:0/2525968466 wait complete. 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.379+0000 7faa9519c740 1 -- start start 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa9519c740 1 Processor -- start 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa9519c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 0x5648cdceb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa9519c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5980 0x5648cdcebb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa9519c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 0x5648cdcec080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa9519c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648cdc061b0 con 0x5648cdce5060 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa9519c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648cdc05e50 con 0x5648cdcee750 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa9519c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648cdbcb090 con 0x5648cdce5980 2024-01-31T05:28:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa9519c740 10 client.0 _ll_register_callbacks cb 0x5648cdbf0a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 0x5648cdceb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 0x5648cdceb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53376/0 (socket says 172.21.15.45:53376) 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa8b7fe640 1 -- 172.21.15.45:0/2876785991 learned_addr learned my addr 172.21.15.45:0/2876785991 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa8bfff640 1 --2- 172.21.15.45:0/2876785991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5980 0x5648cdcebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa8b7fe640 1 -- 172.21.15.45:0/2876785991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5980 msgr2=0x5648cdcebb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/2876785991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5980 0x5648cdcebb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa8b7fe640 1 -- 172.21.15.45:0/2876785991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 msgr2=0x5648cdcec080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/2876785991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 0x5648cdcec080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa8b7fe640 1 -- 172.21.15.45:0/2876785991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c0099f0 con 0x5648cdce5060 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/2876785991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 0x5648cdceb600 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0692c0 tx=0x7faa74004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1cdf3e3296948a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.380+0000 7faa6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa6b7fe640 1 -- 172.21.15.45:0/2876785991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74004040 con 0x5648cdce5060 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa6b7fe640 1 -- 172.21.15.45:0/2876785991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8483+0+0 (secure 0 0 0) 0x7faa740041a0 con 0x5648cdce5060 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa9519c740 1 -- 172.21.15.45:0/2876785991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648cdc46010 con 0x5648cdce5060 2024-01-31T05:28:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa6b7fe640 1 -- 172.21.15.45:0/2876785991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa74004360 con 0x5648cdce5060 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa9519c740 10 client.15879 fetch_fsmap learned FSMap version 27 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa9519c740 1 -- 172.21.15.45:0/2876785991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648cdc4b6a0 con 0x5648cdce5060 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa6b7fe640 1 -- 172.21.15.45:0/2876785991 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648cdc46010 con 0x5648cdce5060 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa6b7fe640 1 -- 172.21.15.45:0/2876785991 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faa7403c070 con 0x5648cdce5060 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa9519c740 10 client.15879 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa9519c740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa9519c740 20 client.15879 populate_metadata read hostname 'smithi045' 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa9519c740 1 --2- 172.21.15.45:0/2876785991 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5648cdcfe460 0x5648cdd00850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa9519c740 4 client.15879 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.381+0000 7faa9519c740 1 -- 172.21.15.45:0/2876785991 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648cdc45d30 con 0x5648cdcfe460 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.382+0000 7faa8bfff640 1 --2- 172.21.15.45:0/2876785991 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5648cdcfe460 0x5648cdd00850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.382+0000 7faa6b7fe640 10 client.15879 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.382+0000 7faa8bfff640 1 --2- 172.21.15.45:0/2876785991 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5648cdcfe460 0x5648cdd00850 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a816944a24882da server_cookie=23b702a0f93fcd79 in_seq=0 out_seq=0 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.382+0000 7faa6b7fe640 1 -- 172.21.15.45:0/2876785991 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648cdc45d30 con 0x5648cdcfe460 2024-01-31T05:28:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.382+0000 7faa6b7fe640 10 client.15879 handle_command_reply: tid=0 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.382+0000 7faa9519c740 1 client.15879 shutdown 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.383+0000 7faa9519c740 1 -- 172.21.15.45:0/2876785991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 msgr2=0x5648cdceb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.383+0000 7faa9519c740 1 --2- 172.21.15.45:0/2876785991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 0x5648cdceb600 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0692c0 tx=0x7faa74004f30 comp rx=0 tx=0).stop 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.383+0000 7faa9519c740 1 -- 172.21.15.45:0/2876785991 shutdown_connections 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.383+0000 7faa9519c740 1 --2- 172.21.15.45:0/2876785991 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5648cdcfe460 0x5648cdd00850 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.383+0000 7faa9519c740 1 --2- 172.21.15.45:0/2876785991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 0x5648cdcec080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.383+0000 7faa6b7fe640 0 client.15879 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.383+0000 7faa9519c740 1 --2- 172.21.15.45:0/2876785991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5980 0x5648cdcebb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.383+0000 7faa9519c740 1 --2- 172.21.15.45:0/2876785991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 0x5648cdceb600 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.383+0000 7faa9519c740 1 -- 172.21.15.45:0/2876785991 >> 172.21.15.45:0/2876785991 conn(0x5648cdcdf980 msgr2=0x5648cdce16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.383+0000 7faa9519c740 1 -- 172.21.15.45:0/2876785991 shutdown_connections 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.383+0000 7faa9519c740 1 -- 172.21.15.45:0/2876785991 wait complete. 2024-01-31T05:28:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.383+0000 7faa9519c740 20 client.15879 trim_cache size 0 max 16384 2024-01-31T05:28:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa9519c740 1 Processor -- start 2024-01-31T05:28:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa9519c740 1 -- start start 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa9519c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdce5060 0x5648cdcedd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa9519c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5980 0x5648cdceaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa9519c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdcee750 0x5648cdce1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa9519c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648cdc45d30 con 0x5648cdce5980 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa9519c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648cdc46010 con 0x5648cdce5060 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa9519c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648cdcec5d0 con 0x5648cdcee750 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdce5060 0x5648cdcedd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdce5060 0x5648cdcedd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:43230/0 (socket says 172.21.15.45:43230) 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa8b7fe640 1 -- 172.21.15.45:0/749250596 learned_addr learned my addr 172.21.15.45:0/749250596 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa8bfff640 1 --2- 172.21.15.45:0/749250596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5980 0x5648cdceaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/749250596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdcee750 0x5648cdce1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa8b7fe640 1 -- 172.21.15.45:0/749250596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdcee750 msgr2=0x5648cdce1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/749250596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdcee750 0x5648cdce1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa8b7fe640 1 -- 172.21.15.45:0/749250596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5980 msgr2=0x5648cdceaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/749250596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5980 0x5648cdceaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.397+0000 7faa8b7fe640 1 -- 172.21.15.45:0/749250596 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648cdbe8030 con 0x5648cdce5060 2024-01-31T05:28:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.398+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/749250596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdce5060 0x5648cdcedd30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faa74004c30 tx=0x7faa7402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4871f54741758c6e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.398+0000 7faa81ffb640 1 -- 172.21.15.45:0/749250596 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74005d50 con 0x5648cdce5060 2024-01-31T05:28:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.398+0000 7faa81ffb640 1 -- 172.21.15.45:0/749250596 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa74004360 con 0x5648cdce5060 2024-01-31T05:28:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.398+0000 7faa81ffb640 1 -- 172.21.15.45:0/749250596 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa740055a0 con 0x5648cdce5060 2024-01-31T05:28:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.399+0000 7faa9519c740 1 -- 172.21.15.45:0/749250596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdce5060 msgr2=0x5648cdcedd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.399+0000 7faa9519c740 1 --2- 172.21.15.45:0/749250596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdce5060 0x5648cdcedd30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faa74004c30 tx=0x7faa7402bc30 comp rx=0 tx=0).stop 2024-01-31T05:28:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.399+0000 7faa9519c740 1 -- 172.21.15.45:0/749250596 shutdown_connections 2024-01-31T05:28:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.399+0000 7faa9519c740 1 --2- 172.21.15.45:0/749250596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5980 0x5648cdceaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.399+0000 7faa9519c740 1 --2- 172.21.15.45:0/749250596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdcee750 0x5648cdce1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.399+0000 7faa9519c740 1 --2- 172.21.15.45:0/749250596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdce5060 0x5648cdcedd30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.399+0000 7faa9519c740 1 -- 172.21.15.45:0/749250596 >> 172.21.15.45:0/749250596 conn(0x5648cdcdf980 msgr2=0x5648cdce0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.399+0000 7faa9519c740 1 -- 172.21.15.45:0/749250596 shutdown_connections 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.399+0000 7faa9519c740 1 -- 172.21.15.45:0/749250596 wait complete. 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.399+0000 7faa9519c740 1 -- start start 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.399+0000 7faa9519c740 1 Processor -- start 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa9519c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 0x5648cdcf3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa9519c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5980 0x5648cdcf4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa9519c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 0x5648cdcf4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa9519c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648cdd54980 con 0x5648cdce5060 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa9519c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648cdd54ac0 con 0x5648cdcee750 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa9519c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648cdd54c00 con 0x5648cdce5980 2024-01-31T05:28:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa9519c740 10 client.0 _ll_register_callbacks cb 0x7faa7c009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 0x5648cdcf3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 0x5648cdcf3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53398/0 (socket says 172.21.15.45:53398) 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa8b7fe640 1 -- 172.21.15.45:0/2517992197 learned_addr learned my addr 172.21.15.45:0/2517992197 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa8bfff640 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5980 0x5648cdcf4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 0x5648cdcf4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa8b7fe640 1 -- 172.21.15.45:0/2517992197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5980 msgr2=0x5648cdcf4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5980 0x5648cdcf4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa8b7fe640 1 -- 172.21.15.45:0/2517992197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 msgr2=0x5648cdcf4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 0x5648cdcf4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa8b7fe640 1 -- 172.21.15.45:0/2517992197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648cdc4b6a0 con 0x5648cdce5060 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa8b7fe640 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 0x5648cdcf3fc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faa74003800 tx=0x7faa74005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842c21bfca0912f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa6b7fe640 1 -- 172.21.15.45:0/2517992197 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa740044d0 con 0x5648cdce5060 2024-01-31T05:28:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa6b7fe640 1 -- 172.21.15.45:0/2517992197 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8483+0+0 (secure 0 0 0) 0x7faa74000e40 con 0x5648cdce5060 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.400+0000 7faa9519c740 1 -- 172.21.15.45:0/2517992197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa7c004c60 con 0x5648cdce5060 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa6b7fe640 1 -- 172.21.15.45:0/2517992197 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa74004a70 con 0x5648cdce5060 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa9519c740 10 client.15885 fetch_fsmap learned FSMap version 27 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa9519c740 1 -- 172.21.15.45:0/2517992197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648cdc0da50 con 0x5648cdce5060 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa6b7fe640 1 -- 172.21.15.45:0/2517992197 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa7c004c60 con 0x5648cdce5060 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa6b7fe640 1 -- 172.21.15.45:0/2517992197 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faa7405f070 con 0x5648cdce5060 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa9519c740 10 client.15885 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa9519c740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa9519c740 20 client.15885 populate_metadata read hostname 'smithi045' 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa9519c740 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5648cdd55ed0 0x5648cdd582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa9519c740 4 client.15885 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa9519c740 1 -- 172.21.15.45:0/2517992197 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5648cd94c600 con 0x5648cdd55ed0 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa8bfff640 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5648cdd55ed0 0x5648cdd582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa6b7fe640 10 client.15885 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.401+0000 7faa8bfff640 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5648cdd55ed0 0x5648cdd582c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9948dfd8199ea536 server_cookie=f8b095243f609b48 in_seq=0 out_seq=0 2024-01-31T05:28:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa6b7fe640 1 -- 172.21.15.45:0/2517992197 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5648cd94c600 con 0x5648cdd55ed0 2024-01-31T05:28:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa6b7fe640 10 client.15885 handle_command_reply: tid=0 2024-01-31T05:28:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa9519c740 1 client.15885 shutdown 2024-01-31T05:28:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa9519c740 1 -- 172.21.15.45:0/2517992197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 msgr2=0x5648cdcf3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa9519c740 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 0x5648cdcf3fc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faa74003800 tx=0x7faa74005180 comp rx=0 tx=0).stop 2024-01-31T05:28:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa9519c740 1 -- 172.21.15.45:0/2517992197 shutdown_connections 2024-01-31T05:28:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa9519c740 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5648cdd55ed0 0x5648cdd582c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa9519c740 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648cdcee750 0x5648cdcf4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa9519c740 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648cdce5980 0x5648cdcf4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa6b7fe640 0 client.15885 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa9519c740 1 --2- 172.21.15.45:0/2517992197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648cdce5060 0x5648cdcf3fc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa9519c740 1 -- 172.21.15.45:0/2517992197 >> 172.21.15.45:0/2517992197 conn(0x5648cdcdf980 msgr2=0x5648cdcebe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa9519c740 1 -- 172.21.15.45:0/2517992197 shutdown_connections 2024-01-31T05:28:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa9519c740 1 -- 172.21.15.45:0/2517992197 wait complete. 2024-01-31T05:28:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.411+0000 7faa9519c740 20 client.15885 trim_cache size 0 max 16384 2024-01-31T05:28:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:28:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:28:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:28:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:28:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.412+0000 7faa923d9640 1 -- 172.21.15.45:0/3396139483 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7faa780a14f0 msgr2=0x7faa780a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.412+0000 7faa923d9640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7faa780a14f0 0x7faa780a39b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa840cad90 tx=0x7faa840699e0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.412+0000 7faa923d9640 1 -- 172.21.15.45:0/3396139483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa8c19ba50 msgr2=0x7faa8c19bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.412+0000 7faa923d9640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa8c19ba50 0x7faa8c19bf00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faa84005fd0 tx=0x7faa8400ee60 comp rx=0 tx=0).stop 2024-01-31T05:28:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.412+0000 7faa923d9640 1 -- 172.21.15.45:0/3396139483 shutdown_connections 2024-01-31T05:28:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.412+0000 7faa923d9640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7faa780a14f0 0x7faa780a39b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.412+0000 7faa923d9640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa8c19ba50 0x7faa8c19bf00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.412+0000 7faa923d9640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c106020 0x7faa8c1976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.412+0000 7faa923d9640 1 --2- 172.21.15.45:0/3396139483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa8c0697a0 0x7faa8c197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.412+0000 7faa923d9640 1 -- 172.21.15.45:0/3396139483 >> 172.21.15.45:0/3396139483 conn(0x7faa8c0769c0 msgr2=0x7faa8c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.412+0000 7faa923d9640 1 -- 172.21.15.45:0/3396139483 shutdown_connections 2024-01-31T05:28:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:44.412+0000 7faa923d9640 1 -- 172.21.15.45:0/3396139483 wait complete. 2024-01-31T05:28:44.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 614581 11.54 MB/sec execute 164 sec latency 12.834 ms 2024-01-31T05:28:44.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:44 smithi186 ceph-mon[35659]: pgmap v187: 97 pgs: 97 active+clean; 358 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 275 op/s 2024-01-31T05:28:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:44 smithi045 ceph-mon[31585]: pgmap v187: 97 pgs: 97 active+clean; 358 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 275 op/s 2024-01-31T05:28:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:44 smithi156 ceph-mon[35720]: pgmap v187: 97 pgs: 97 active+clean; 358 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 275 op/s 2024-01-31T05:28:45.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 616579 11.51 MB/sec execute 165 sec latency 20.865 ms 2024-01-31T05:28:46.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 618764 11.52 MB/sec execute 166 sec latency 16.959 ms 2024-01-31T05:28:46.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:46 smithi186 ceph-mon[35659]: pgmap v188: 97 pgs: 97 active+clean; 358 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 173 op/s 2024-01-31T05:28:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:46 smithi045 ceph-mon[31585]: pgmap v188: 97 pgs: 97 active+clean; 358 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 173 op/s 2024-01-31T05:28:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:46 smithi156 ceph-mon[35720]: pgmap v188: 97 pgs: 97 active+clean; 358 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 173 op/s 2024-01-31T05:28:47.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 621180 11.53 MB/sec execute 167 sec latency 88.396 ms 2024-01-31T05:28:48.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 622980 11.53 MB/sec execute 168 sec latency 18.902 ms 2024-01-31T05:28:48.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:48 smithi186 ceph-mon[35659]: pgmap v189: 97 pgs: 97 active+clean; 367 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 254 op/s 2024-01-31T05:28:48.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:48 smithi045 ceph-mon[31585]: pgmap v189: 97 pgs: 97 active+clean; 367 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 254 op/s 2024-01-31T05:28:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:48 smithi156 ceph-mon[35720]: pgmap v189: 97 pgs: 97 active+clean; 367 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 254 op/s 2024-01-31T05:28:49.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 625327 11.54 MB/sec execute 169 sec latency 16.399 ms 2024-01-31T05:28:50.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 627168 11.51 MB/sec execute 170 sec latency 41.945 ms 2024-01-31T05:28:50.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:50 smithi186 ceph-mon[35659]: pgmap v190: 97 pgs: 97 active+clean; 367 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-01-31T05:28:50.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:50 smithi045 ceph-mon[31585]: pgmap v190: 97 pgs: 97 active+clean; 367 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-01-31T05:28:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:50 smithi156 ceph-mon[35720]: pgmap v190: 97 pgs: 97 active+clean; 367 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-01-31T05:28:51.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 629191 11.52 MB/sec execute 171 sec latency 13.892 ms 2024-01-31T05:28:52.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 631048 11.49 MB/sec execute 172 sec latency 103.011 ms 2024-01-31T05:28:52.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:52 smithi186 ceph-mon[35659]: pgmap v191: 97 pgs: 97 active+clean; 367 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-01-31T05:28:52.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:52 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:28:52.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:52 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:28:52.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:52 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:28:52.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:52 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:28:52.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:52 smithi045 ceph-mon[31585]: pgmap v191: 97 pgs: 97 active+clean; 367 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-01-31T05:28:52.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:52 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:28:52.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:52 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:28:52.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:52 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:28:52.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:52 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:28:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:52 smithi156 ceph-mon[35720]: pgmap v191: 97 pgs: 97 active+clean; 367 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-01-31T05:28:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:52 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:28:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:52 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:28:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:52 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:28:52.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:52 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:28:53.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 632956 11.50 MB/sec execute 173 sec latency 42.295 ms 2024-01-31T05:28:54.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 635527 11.51 MB/sec execute 174 sec latency 14.180 ms 2024-01-31T05:28:54.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:54 smithi186 ceph-mon[35659]: pgmap v192: 97 pgs: 97 active+clean; 375 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-01-31T05:28:54.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:54 smithi045 ceph-mon[31585]: pgmap v192: 97 pgs: 97 active+clean; 375 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-01-31T05:28:54.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:54 smithi156 ceph-mon[35720]: pgmap v192: 97 pgs: 97 active+clean; 375 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-01-31T05:28:55.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 637267 11.50 MB/sec execute 175 sec latency 16.861 ms 2024-01-31T05:28:56.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 639617 11.52 MB/sec execute 176 sec latency 15.288 ms 2024-01-31T05:28:56.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:56 smithi186 ceph-mon[35659]: pgmap v193: 97 pgs: 97 active+clean; 375 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-01-31T05:28:56.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:56 smithi045 ceph-mon[31585]: pgmap v193: 97 pgs: 97 active+clean; 375 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-01-31T05:28:56.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:56 smithi156 ceph-mon[35720]: pgmap v193: 97 pgs: 97 active+clean; 375 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-01-31T05:28:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.510+0000 7f3485be1640 1 -- 172.21.15.45:0/2278902237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3480100f30 msgr2=0x7f3480101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.510+0000 7f3485be1640 1 --2- 172.21.15.45:0/2278902237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3480100f30 0x7f3480101310 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3470009930 tx=0x7f347002bc30 comp rx=0 tx=0).stop 2024-01-31T05:28:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.510+0000 7f3485be1640 1 -- 172.21.15.45:0/2278902237 shutdown_connections 2024-01-31T05:28:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.510+0000 7f3485be1640 1 --2- 172.21.15.45:0/2278902237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3480106000 0x7f348010dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.510+0000 7f3485be1640 1 --2- 172.21.15.45:0/2278902237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34801018e0 0x7f3480105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.510+0000 7f3485be1640 1 --2- 172.21.15.45:0/2278902237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3480100f30 0x7f3480101310 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.510+0000 7f3485be1640 1 -- 172.21.15.45:0/2278902237 >> 172.21.15.45:0/2278902237 conn(0x7f34800fc7d0 msgr2=0x7f34800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.510+0000 7f3485be1640 1 -- 172.21.15.45:0/2278902237 shutdown_connections 2024-01-31T05:28:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.510+0000 7f3485be1640 1 -- 172.21.15.45:0/2278902237 wait complete. 2024-01-31T05:28:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.510+0000 7f3485be1640 1 Processor -- start 2024-01-31T05:28:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f3485be1640 1 -- start start 2024-01-31T05:28:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f3485be1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3480100f30 0x7f3480072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f3485be1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3480106000 0x7f3480073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f3485be1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3480073a70 0x7f34801a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f3485be1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34801a5960 con 0x7f3480073a70 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f3485be1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34801a5ad0 con 0x7f3480106000 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f3485be1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34801a5c90 con 0x7f3480100f30 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f347ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3480073a70 0x7f34801a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f347ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3480073a70 0x7f34801a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59684/0 (socket says 172.21.15.45:59684) 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f347ffff640 1 -- 172.21.15.45:0/1086057731 learned_addr learned my addr 172.21.15.45:0/1086057731 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f347f7fe640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3480100f30 0x7f3480072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f347effd640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3480106000 0x7f3480073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f347ffff640 1 -- 172.21.15.45:0/1086057731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3480100f30 msgr2=0x7f3480072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f347ffff640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3480100f30 0x7f3480072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f347ffff640 1 -- 172.21.15.45:0/1086057731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3480106000 msgr2=0x7f3480073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f347ffff640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3480106000 0x7f3480073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f347ffff640 1 -- 172.21.15.45:0/1086057731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3470009560 con 0x7f3480073a70 2024-01-31T05:28:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.511+0000 7f347ffff640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3480073a70 0x7f34801a53f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f347800eb10 tx=0x7f347800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.512+0000 7f347cff9640 1 -- 172.21.15.45:0/1086057731 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478004b20 con 0x7f3480073a70 2024-01-31T05:28:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.512+0000 7f347cff9640 1 -- 172.21.15.45:0/1086057731 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3478004c80 con 0x7f3480073a70 2024-01-31T05:28:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.512+0000 7f347cff9640 1 -- 172.21.15.45:0/1086057731 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478010690 con 0x7f3480073a70 2024-01-31T05:28:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.512+0000 7f3485be1640 1 -- 172.21.15.45:0/1086057731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34801a5f70 con 0x7f3480073a70 2024-01-31T05:28:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.512+0000 7f3485be1640 1 -- 172.21.15.45:0/1086057731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34801a6400 con 0x7f3480073a70 2024-01-31T05:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.514+0000 7f347cff9640 1 -- 172.21.15.45:0/1086057731 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f3478005020 con 0x7f3480073a70 2024-01-31T05:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f34889aa740 1 Processor -- start 2024-01-31T05:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f34889aa740 1 -- start start 2024-01-31T05:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f347cff9640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f34680a14f0 0x7f34680a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f34889aa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 0x55e8dce91440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f34889aa740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91980 0x55e8dce9a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f34889aa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce9cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f34889aa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dce00040 con 0x55e8dce91060 2024-01-31T05:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f34889aa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dcdfd730 con 0x55e8dce9a750 2024-01-31T05:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f34889aa740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dcdf9020 con 0x55e8dce91980 2024-01-31T05:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f347cff9640 1 -- 172.21.15.45:0/1086057731 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3478014070 con 0x7f3480073a70 2024-01-31T05:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f347f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 0x55e8dce91440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f347effd640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f34680a14f0 0x7f34680a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f347f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 0x55e8dce91440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59688/0 (socket says 172.21.15.45:59688) 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f347f7fe640 1 -- 172.21.15.45:0/3068315595 learned_addr learned my addr 172.21.15.45:0/3068315595 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f347ffff640 1 --2- 172.21.15.45:0/3068315595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce9cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f347effd640 1 --2- 172.21.15.45:0/3068315595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91980 0x55e8dce9a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f347f7fe640 1 -- 172.21.15.45:0/3068315595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91980 msgr2=0x55e8dce9a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f347f7fe640 1 --2- 172.21.15.45:0/3068315595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91980 0x55e8dce9a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f347f7fe640 1 -- 172.21.15.45:0/3068315595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 msgr2=0x55e8dce9cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.515+0000 7f347f7fe640 1 --2- 172.21.15.45:0/3068315595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce9cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.516+0000 7f347f7fe640 1 -- 172.21.15.45:0/3068315595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8dcdf0e40 con 0x55e8dce91060 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.516+0000 7f347effd640 1 --2- 172.21.15.45:0/3068315595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91980 0x55e8dce9a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.516+0000 7f347ffff640 1 --2- 172.21.15.45:0/3068315595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce9cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.516+0000 7f347f7fe640 1 --2- 172.21.15.45:0/3068315595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 0x55e8dce91440 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3470036040 tx=0x7f3470005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30b2e957a14bc8f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.516+0000 7f347effd640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f34680a14f0 0x7f34680a39b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55e8dce90b10 tx=0x7f347400b5b0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.516+0000 7f3465ffb640 1 -- 172.21.15.45:0/3068315595 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34700046c0 con 0x55e8dce91060 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.516+0000 7f3465ffb640 1 -- 172.21.15.45:0/3068315595 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3470004820 con 0x55e8dce91060 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.516+0000 7f3465ffb640 1 -- 172.21.15.45:0/3068315595 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470005000 con 0x55e8dce91060 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 -- 172.21.15.45:0/3068315595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 msgr2=0x55e8dce91440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 --2- 172.21.15.45:0/3068315595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 0x55e8dce91440 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3470036040 tx=0x7f3470005e00 comp rx=0 tx=0).stop 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 -- 172.21.15.45:0/3068315595 shutdown_connections 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 --2- 172.21.15.45:0/3068315595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce9cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 --2- 172.21.15.45:0/3068315595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91980 0x55e8dce9a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 --2- 172.21.15.45:0/3068315595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 0x55e8dce91440 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 -- 172.21.15.45:0/3068315595 >> 172.21.15.45:0/3068315595 conn(0x55e8dce8b980 msgr2=0x55e8dce8bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 -- 172.21.15.45:0/3068315595 shutdown_connections 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 -- 172.21.15.45:0/3068315595 wait complete. 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 -- start start 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 Processor -- start 2024-01-31T05:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce91060 0x55e8dce976b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 0x55e8dce97bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce9a750 0x55e8dce98130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dcdb21b0 con 0x55e8dce91980 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dcdb1e50 con 0x55e8dce91060 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.517+0000 7f34889aa740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dcd77090 con 0x55e8dce9a750 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f34889aa740 10 client.0 _ll_register_callbacks cb 0x55e8dcd9ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 0x55e8dce97bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347ffff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce9a750 0x55e8dce98130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce91060 0x55e8dce976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 0x55e8dce97bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59700/0 (socket says 172.21.15.45:59700) 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347effd640 1 -- 172.21.15.45:0/3101742787 learned_addr learned my addr 172.21.15.45:0/3101742787 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347effd640 1 -- 172.21.15.45:0/3101742787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce9a750 msgr2=0x55e8dce98130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347effd640 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce9a750 0x55e8dce98130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347effd640 1 -- 172.21.15.45:0/3101742787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce91060 msgr2=0x55e8dce976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347effd640 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce91060 0x55e8dce976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347effd640 1 -- 172.21.15.45:0/3101742787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348010e3e0 con 0x55e8dce91980 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347ffff640 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce9a750 0x55e8dce98130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347f7fe640 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce91060 0x55e8dce976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f347effd640 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 0x55e8dce97bf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3474010760 tx=0x7f3474004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bbbde4987222c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f345effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f345effd640 1 -- 172.21.15.45:0/3101742787 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474011a70 con 0x55e8dce91980 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f345effd640 1 -- 172.21.15.45:0/3101742787 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3474011bd0 con 0x55e8dce91980 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f34889aa740 1 -- 172.21.15.45:0/3101742787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8dcdf2010 con 0x55e8dce91980 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f345effd640 1 -- 172.21.15.45:0/3101742787 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3474011d90 con 0x55e8dce91980 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f34889aa740 10 client.15897 fetch_fsmap learned FSMap version 27 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.518+0000 7f34889aa740 1 -- 172.21.15.45:0/3101742787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e8dcdf76a0 con 0x55e8dce91980 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.519+0000 7f345effd640 1 -- 172.21.15.45:0/3101742787 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e8dcdf2010 con 0x55e8dce91980 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.519+0000 7f345effd640 1 -- 172.21.15.45:0/3101742787 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f347402a410 con 0x55e8dce91980 2024-01-31T05:28:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.519+0000 7f34889aa740 10 client.15897 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.519+0000 7f34889aa740 10 client.15897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.519+0000 7f34889aa740 20 client.15897 populate_metadata read hostname 'smithi045' 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.519+0000 7f34889aa740 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e8dceaa4c0 0x55e8dceac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.519+0000 7f34889aa740 4 client.15897 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.519+0000 7f34889aa740 1 -- 172.21.15.45:0/3101742787 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8dcdf1d30 con 0x55e8dceaa4c0 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.519+0000 7f347f7fe640 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e8dceaa4c0 0x55e8dceac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f347f7fe640 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e8dceaa4c0 0x55e8dceac900 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4397051651430649 server_cookie=4ed5f4c04618989c in_seq=0 out_seq=0 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f345effd640 10 client.15897 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f345effd640 1 -- 172.21.15.45:0/3101742787 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e8dcdf1d30 con 0x55e8dceaa4c0 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f345effd640 10 client.15897 handle_command_reply: tid=0 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f34889aa740 1 client.15897 shutdown 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f34889aa740 1 -- 172.21.15.45:0/3101742787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 msgr2=0x55e8dce97bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f34889aa740 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 0x55e8dce97bf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3474010760 tx=0x7f3474004730 comp rx=0 tx=0).stop 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f34889aa740 1 -- 172.21.15.45:0/3101742787 shutdown_connections 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f34889aa740 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e8dceaa4c0 0x55e8dceac900 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f34889aa740 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 0x55e8dce97bf0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f34889aa740 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce9a750 0x55e8dce98130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f34889aa740 1 --2- 172.21.15.45:0/3101742787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce91060 0x55e8dce976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f34889aa740 1 -- 172.21.15.45:0/3101742787 >> 172.21.15.45:0/3101742787 conn(0x55e8dce8b980 msgr2=0x55e8dce8d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.520+0000 7f345effd640 0 client.15897 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.521+0000 7f34889aa740 1 -- 172.21.15.45:0/3101742787 shutdown_connections 2024-01-31T05:28:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.521+0000 7f34889aa740 1 -- 172.21.15.45:0/3101742787 wait complete. 2024-01-31T05:28:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.521+0000 7f34889aa740 20 client.15897 trim_cache size 0 max 16384 2024-01-31T05:28:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f34889aa740 1 Processor -- start 2024-01-31T05:28:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f34889aa740 1 -- start start 2024-01-31T05:28:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f34889aa740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91060 0x55e8dce99de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f34889aa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 0x55e8dce97f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f34889aa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce94970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f34889aa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dcdf2010 con 0x55e8dce91980 2024-01-31T05:28:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f34889aa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dce8c780 con 0x55e8dce9a750 2024-01-31T05:28:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f34889aa740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dce8c980 con 0x55e8dce91060 2024-01-31T05:28:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f347effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 0x55e8dce97f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f347effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 0x55e8dce97f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59710/0 (socket says 172.21.15.45:59710) 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f347effd640 1 -- 172.21.15.45:0/418352833 learned_addr learned my addr 172.21.15.45:0/418352833 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f347f7fe640 1 --2- 172.21.15.45:0/418352833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91060 0x55e8dce99de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.535+0000 7f347ffff640 1 --2- 172.21.15.45:0/418352833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce94970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.536+0000 7f347effd640 1 -- 172.21.15.45:0/418352833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91060 msgr2=0x55e8dce99de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.536+0000 7f347effd640 1 --2- 172.21.15.45:0/418352833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91060 0x55e8dce99de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.536+0000 7f347effd640 1 -- 172.21.15.45:0/418352833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 msgr2=0x55e8dce94970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.536+0000 7f347effd640 1 --2- 172.21.15.45:0/418352833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce94970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.536+0000 7f347effd640 1 -- 172.21.15.45:0/418352833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8dcd94030 con 0x55e8dce91980 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.536+0000 7f347effd640 1 --2- 172.21.15.45:0/418352833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 0x55e8dce97f50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3474004700 tx=0x7f347400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb69bb7947aa6de server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.536+0000 7f34657fa640 1 -- 172.21.15.45:0/418352833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474039070 con 0x55e8dce91980 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.536+0000 7f34657fa640 1 -- 172.21.15.45:0/418352833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f347400b390 con 0x55e8dce91980 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.536+0000 7f34657fa640 1 -- 172.21.15.45:0/418352833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474033460 con 0x55e8dce91980 2024-01-31T05:28:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.537+0000 7f34889aa740 1 -- 172.21.15.45:0/418352833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 msgr2=0x55e8dce97f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.537+0000 7f34889aa740 1 --2- 172.21.15.45:0/418352833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 0x55e8dce97f50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3474004700 tx=0x7f347400bb10 comp rx=0 tx=0).stop 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.537+0000 7f34889aa740 1 -- 172.21.15.45:0/418352833 shutdown_connections 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.537+0000 7f34889aa740 1 --2- 172.21.15.45:0/418352833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91980 0x55e8dce97f50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.537+0000 7f34889aa740 1 --2- 172.21.15.45:0/418352833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce94970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.537+0000 7f34889aa740 1 --2- 172.21.15.45:0/418352833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91060 0x55e8dce99de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.537+0000 7f34889aa740 1 -- 172.21.15.45:0/418352833 >> 172.21.15.45:0/418352833 conn(0x55e8dce8b980 msgr2=0x55e8dce95fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.537+0000 7f34889aa740 1 -- 172.21.15.45:0/418352833 shutdown_connections 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.537+0000 7f34889aa740 1 -- 172.21.15.45:0/418352833 wait complete. 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.537+0000 7f34889aa740 1 -- start start 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.537+0000 7f34889aa740 1 Processor -- start 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f34889aa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 0x55e8dcea1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f34889aa740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91980 0x55e8dce9fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f34889aa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce9ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f34889aa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dcea0510 con 0x55e8dce91060 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f34889aa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dcea0680 con 0x55e8dce9a750 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f34889aa740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dcea0840 con 0x55e8dce91980 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f34889aa740 10 client.0 _ll_register_callbacks cb 0x7f3470009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f347f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 0x55e8dcea1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f347f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 0x55e8dcea1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59720/0 (socket says 172.21.15.45:59720) 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f347f7fe640 1 -- 172.21.15.45:0/3991122814 learned_addr learned my addr 172.21.15.45:0/3991122814 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f347effd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91980 0x55e8dce9fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f347ffff640 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce9ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f347f7fe640 1 -- 172.21.15.45:0/3991122814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91980 msgr2=0x55e8dce9fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f347f7fe640 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91980 0x55e8dce9fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f347f7fe640 1 -- 172.21.15.45:0/3991122814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 msgr2=0x55e8dce9ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f347f7fe640 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce9ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f347f7fe640 1 -- 172.21.15.45:0/3991122814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8dcdf76a0 con 0x55e8dce91060 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f347ffff640 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce9ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:28:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f347f7fe640 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 0x55e8dcea1670 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f34700042d0 tx=0x7f3470004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe279e553d7d5c0d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f345effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f345effd640 1 -- 172.21.15.45:0/3991122814 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470050070 con 0x55e8dce91060 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f345effd640 1 -- 172.21.15.45:0/3991122814 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3470004370 con 0x55e8dce91060 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f34889aa740 1 -- 172.21.15.45:0/3991122814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8dcaf8600 con 0x55e8dce91060 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.538+0000 7f345effd640 1 -- 172.21.15.45:0/3991122814 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3470052b00 con 0x55e8dce91060 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.539+0000 7f34889aa740 10 client.15903 fetch_fsmap learned FSMap version 27 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.539+0000 7f34889aa740 1 -- 172.21.15.45:0/3991122814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e8dcdb9a50 con 0x55e8dce91060 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.539+0000 7f345effd640 1 -- 172.21.15.45:0/3991122814 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e8dcaf8600 con 0x55e8dce91060 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.539+0000 7f345effd640 1 -- 172.21.15.45:0/3991122814 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f347003c050 con 0x55e8dce91060 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.539+0000 7f34889aa740 10 client.15903 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.539+0000 7f34889aa740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.539+0000 7f34889aa740 20 client.15903 populate_metadata read hostname 'smithi045' 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.539+0000 7f34889aa740 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e8dcf01d00 0x55e8dcf04120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.539+0000 7f34889aa740 4 client.15903 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.539+0000 7f34889aa740 1 -- 172.21.15.45:0/3991122814 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e8dcb4bc50 con 0x55e8dcf01d00 2024-01-31T05:28:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.539+0000 7f347effd640 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e8dcf01d00 0x55e8dcf04120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:28:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.540+0000 7f345effd640 10 client.15903 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.540+0000 7f347effd640 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e8dcf01d00 0x55e8dcf04120 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af38e41cc34fd81 server_cookie=8ccc5ae6c5989618 in_seq=0 out_seq=0 2024-01-31T05:28:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f345effd640 1 -- 172.21.15.45:0/3991122814 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e8dcb4bc50 con 0x55e8dcf01d00 2024-01-31T05:28:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f345effd640 10 client.15903 handle_command_reply: tid=0 2024-01-31T05:28:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f34889aa740 1 client.15903 shutdown 2024-01-31T05:28:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f34889aa740 1 -- 172.21.15.45:0/3991122814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 msgr2=0x55e8dcea1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f34889aa740 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 0x55e8dcea1670 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f34700042d0 tx=0x7f3470004260 comp rx=0 tx=0).stop 2024-01-31T05:28:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f34889aa740 1 -- 172.21.15.45:0/3991122814 shutdown_connections 2024-01-31T05:28:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f34889aa740 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e8dcf01d00 0x55e8dcf04120 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f34889aa740 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8dce9a750 0x55e8dce9ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f34889aa740 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dce91980 0x55e8dce9fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f345effd640 0 client.15903 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:28:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f34889aa740 1 --2- 172.21.15.45:0/3991122814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8dce91060 0x55e8dcea1670 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f34889aa740 1 -- 172.21.15.45:0/3991122814 >> 172.21.15.45:0/3991122814 conn(0x55e8dce8b980 msgr2=0x55e8dce8a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f34889aa740 1 -- 172.21.15.45:0/3991122814 shutdown_connections 2024-01-31T05:28:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f34889aa740 1 -- 172.21.15.45:0/3991122814 wait complete. 2024-01-31T05:28:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.549+0000 7f34889aa740 20 client.15903 trim_cache size 0 max 16384 2024-01-31T05:28:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:28:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:28:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:28:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:28:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.550+0000 7f3485be1640 1 -- 172.21.15.45:0/1086057731 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f34680a14f0 msgr2=0x7f34680a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.550+0000 7f3485be1640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f34680a14f0 0x7f34680a39b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55e8dce90b10 tx=0x7f347400b5b0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.550+0000 7f3485be1640 1 -- 172.21.15.45:0/1086057731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3480073a70 msgr2=0x7f34801a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:28:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.550+0000 7f3485be1640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3480073a70 0x7f34801a53f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f347800eb10 tx=0x7f347800c2d0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.550+0000 7f3485be1640 1 -- 172.21.15.45:0/1086057731 shutdown_connections 2024-01-31T05:28:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.550+0000 7f3485be1640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f34680a14f0 0x7f34680a39b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.550+0000 7f3485be1640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3480073a70 0x7f34801a53f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.550+0000 7f3485be1640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3480106000 0x7f3480073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.550+0000 7f3485be1640 1 --2- 172.21.15.45:0/1086057731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3480100f30 0x7f3480072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:28:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.550+0000 7f3485be1640 1 -- 172.21.15.45:0/1086057731 >> 172.21.15.45:0/1086057731 conn(0x7f34800fc7d0 msgr2=0x7f34800feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:28:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.550+0000 7f3485be1640 1 -- 172.21.15.45:0/1086057731 shutdown_connections 2024-01-31T05:28:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:28:57.550+0000 7f3485be1640 1 -- 172.21.15.45:0/1086057731 wait complete. 2024-01-31T05:28:57.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 641654 11.49 MB/sec execute 177 sec latency 24.951 ms 2024-01-31T05:28:58.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 643280 11.49 MB/sec execute 178 sec latency 75.085 ms 2024-01-31T05:28:58.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:28:58 smithi186 ceph-mon[35659]: pgmap v194: 97 pgs: 97 active+clean; 383 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2024-01-31T05:28:58.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:28:58 smithi045 ceph-mon[31585]: pgmap v194: 97 pgs: 97 active+clean; 383 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2024-01-31T05:28:58.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:28:58 smithi156 ceph-mon[35720]: pgmap v194: 97 pgs: 97 active+clean; 383 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2024-01-31T05:28:59.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 645566 11.49 MB/sec execute 179 sec latency 18.060 ms 2024-01-31T05:29:00.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 647357 11.48 MB/sec execute 180 sec latency 13.370 ms 2024-01-31T05:29:00.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:00 smithi186 ceph-mon[35659]: pgmap v195: 97 pgs: 97 active+clean; 383 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-01-31T05:29:00.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:00 smithi045 ceph-mon[31585]: pgmap v195: 97 pgs: 97 active+clean; 383 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-01-31T05:29:00.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:00 smithi156 ceph-mon[35720]: pgmap v195: 97 pgs: 97 active+clean; 383 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-01-31T05:29:01.785 INFO:tasks.workunit.client.0.smithi045.stdout: 1 649116 11.47 MB/sec execute 181 sec latency 23.875 ms 2024-01-31T05:29:02.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 650949 11.46 MB/sec execute 182 sec latency 20.909 ms 2024-01-31T05:29:02.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:02 smithi186 ceph-mon[35659]: pgmap v196: 97 pgs: 97 active+clean; 383 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-01-31T05:29:02.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:02 smithi045 ceph-mon[31585]: pgmap v196: 97 pgs: 97 active+clean; 383 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-01-31T05:29:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:02 smithi156 ceph-mon[35720]: pgmap v196: 97 pgs: 97 active+clean; 383 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-01-31T05:29:03.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 653101 11.46 MB/sec execute 183 sec latency 130.188 ms 2024-01-31T05:29:04.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 654933 11.46 MB/sec execute 184 sec latency 15.370 ms 2024-01-31T05:29:04.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:04 smithi186 ceph-mon[35659]: pgmap v197: 97 pgs: 97 active+clean; 361 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2024-01-31T05:29:04.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:04 smithi045 ceph-mon[31585]: pgmap v197: 97 pgs: 97 active+clean; 361 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2024-01-31T05:29:04.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:04 smithi156 ceph-mon[35720]: pgmap v197: 97 pgs: 97 active+clean; 361 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2024-01-31T05:29:05.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 657350 11.47 MB/sec execute 185 sec latency 13.295 ms 2024-01-31T05:29:06.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 659426 11.45 MB/sec execute 186 sec latency 24.177 ms 2024-01-31T05:29:06.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:06 smithi186 ceph-mon[35659]: pgmap v198: 97 pgs: 97 active+clean; 361 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 211 op/s 2024-01-31T05:29:06.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:06 smithi045 ceph-mon[31585]: pgmap v198: 97 pgs: 97 active+clean; 361 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 211 op/s 2024-01-31T05:29:06.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:06 smithi156 ceph-mon[35720]: pgmap v198: 97 pgs: 97 active+clean; 361 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 211 op/s 2024-01-31T05:29:07.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 661315 11.46 MB/sec execute 187 sec latency 62.127 ms 2024-01-31T05:29:08.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 663430 11.46 MB/sec execute 188 sec latency 58.700 ms 2024-01-31T05:29:08.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:08 smithi045 ceph-mon[31585]: pgmap v199: 97 pgs: 97 active+clean; 369 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 314 op/s 2024-01-31T05:29:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:08 smithi156 ceph-mon[35720]: pgmap v199: 97 pgs: 97 active+clean; 369 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 314 op/s 2024-01-31T05:29:09.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:08 smithi186 ceph-mon[35659]: pgmap v199: 97 pgs: 97 active+clean; 369 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 314 op/s 2024-01-31T05:29:09.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 664806 11.43 MB/sec execute 189 sec latency 208.097 ms 2024-01-31T05:29:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.661+0000 7f15c762a640 1 -- 172.21.15.45:0/637687661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c0104670 msgr2=0x7f15c010c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.661+0000 7f15c762a640 1 --2- 172.21.15.45:0/637687661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c0104670 0x7f15c010c240 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009cd0 tx=0x7f15b802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.661+0000 7f15c762a640 1 -- 172.21.15.45:0/637687661 shutdown_connections 2024-01-31T05:29:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.661+0000 7f15c762a640 1 --2- 172.21.15.45:0/637687661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c0104670 0x7f15c010c240 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.661+0000 7f15c762a640 1 --2- 172.21.15.45:0/637687661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c00fffa0 0x7f15c0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.661+0000 7f15c762a640 1 --2- 172.21.15.45:0/637687661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c00ff5f0 0x7f15c00ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.661+0000 7f15c762a640 1 -- 172.21.15.45:0/637687661 >> 172.21.15.45:0/637687661 conn(0x7f15c0076110 msgr2=0x7f15c00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.661+0000 7f15c762a640 1 -- 172.21.15.45:0/637687661 shutdown_connections 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.661+0000 7f15c762a640 1 -- 172.21.15.45:0/637687661 wait complete. 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c762a640 1 Processor -- start 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c762a640 1 -- start start 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c762a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c00ff5f0 0x7f15c00fb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c762a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c0104670 0x7f15c00fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c762a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c00fbfe0 0x7f15c01a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c762a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c01a5940 con 0x7f15c00fbfe0 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c762a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c01a5ab0 con 0x7f15c0104670 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c762a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c01a5c70 con 0x7f15c00ff5f0 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c5bc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c00fbfe0 0x7f15c01a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c53c6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c00ff5f0 0x7f15c00fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c5bc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c00fbfe0 0x7f15c01a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51374/0 (socket says 172.21.15.45:51374) 2024-01-31T05:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c5bc7640 1 -- 172.21.15.45:0/2085695421 learned_addr learned my addr 172.21.15.45:0/2085695421 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.662+0000 7f15c4bc5640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c0104670 0x7f15c00fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.663+0000 7f15c5bc7640 1 -- 172.21.15.45:0/2085695421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c00ff5f0 msgr2=0x7f15c00fb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.663+0000 7f15c5bc7640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c00ff5f0 0x7f15c00fb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.663+0000 7f15c5bc7640 1 -- 172.21.15.45:0/2085695421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c0104670 msgr2=0x7f15c00fbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.663+0000 7f15c5bc7640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c0104670 0x7f15c00fbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.663+0000 7f15c5bc7640 1 -- 172.21.15.45:0/2085695421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b8009900 con 0x7f15c00fbfe0 2024-01-31T05:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.663+0000 7f15c5bc7640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c00fbfe0 0x7f15c01a53d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f15b8005e30 tx=0x7f15b80325c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.663+0000 7f15b67fc640 1 -- 172.21.15.45:0/2085695421 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8032890 con 0x7f15c00fbfe0 2024-01-31T05:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.663+0000 7f15b67fc640 1 -- 172.21.15.45:0/2085695421 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15b8003e30 con 0x7f15c00fbfe0 2024-01-31T05:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.663+0000 7f15c762a640 1 -- 172.21.15.45:0/2085695421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c01a5ef0 con 0x7f15c00fbfe0 2024-01-31T05:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.663+0000 7f15b67fc640 1 -- 172.21.15.45:0/2085695421 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8004690 con 0x7f15c00fbfe0 2024-01-31T05:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.663+0000 7f15c762a640 1 -- 172.21.15.45:0/2085695421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c01a6370 con 0x7f15c00fbfe0 2024-01-31T05:29:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.665+0000 7f15b67fc640 1 -- 172.21.15.45:0/2085695421 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f15b803b040 con 0x7f15c00fbfe0 2024-01-31T05:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15ca3ee740 1 Processor -- start 2024-01-31T05:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15ca3ee740 1 -- start start 2024-01-31T05:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15b67fc640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f15ac0a14f0 0x7f15ac0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15ca3ee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d2580440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15ca3ee740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 0x5645d2589210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15ca3ee740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d258bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15ca3ee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d24ef040 con 0x5645d2580060 2024-01-31T05:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15ca3ee740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d24ec730 con 0x5645d2589750 2024-01-31T05:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15ca3ee740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d24e8020 con 0x5645d2580980 2024-01-31T05:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15b67fc640 1 -- 172.21.15.45:0/2085695421 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f15b80ef780 con 0x7f15c00fbfe0 2024-01-31T05:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15c4bc5640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f15ac0a14f0 0x7f15ac0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15c53c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d2580440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15c53c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d2580440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51382/0 (socket says 172.21.15.45:51382) 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15c53c6640 1 -- 172.21.15.45:0/2183846904 learned_addr learned my addr 172.21.15.45:0/2183846904 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15c4bc5640 1 --2- 172.21.15.45:0/2183846904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 0x5645d2589210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15c53c6640 1 -- 172.21.15.45:0/2183846904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 msgr2=0x5645d2589210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15c53c6640 1 --2- 172.21.15.45:0/2183846904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 0x5645d2589210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15c5bc7640 1 --2- 172.21.15.45:0/2183846904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d258bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.666+0000 7f15c53c6640 1 -- 172.21.15.45:0/2183846904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 msgr2=0x5645d258bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.667+0000 7f15c53c6640 1 --2- 172.21.15.45:0/2183846904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d258bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.667+0000 7f15c53c6640 1 -- 172.21.15.45:0/2183846904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645d24dfe40 con 0x5645d2580060 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.667+0000 7f15c4bc5640 1 --2- 172.21.15.45:0/2183846904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 0x5645d2589210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.667+0000 7f15c5bc7640 1 --2- 172.21.15.45:0/2183846904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d258bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.667+0000 7f15c4bc5640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f15ac0a14f0 0x7f15ac0a39b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f15c00fcfa0 tx=0x7f15b000e820 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.667+0000 7f15c53c6640 1 --2- 172.21.15.45:0/2183846904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d2580440 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009cf0 tx=0x7f15bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ea6a887447f44c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.667+0000 7f15ab7fe640 1 -- 172.21.15.45:0/2183846904 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc009ea0 con 0x5645d2580060 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.667+0000 7f15ab7fe640 1 -- 172.21.15.45:0/2183846904 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15bc004220 con 0x5645d2580060 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.667+0000 7f15ab7fe640 1 -- 172.21.15.45:0/2183846904 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0187d0 con 0x5645d2580060 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.668+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2183846904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 msgr2=0x5645d2580440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.668+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/2183846904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d2580440 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009cf0 tx=0x7f15bc011810 comp rx=0 tx=0).stop 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.668+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2183846904 shutdown_connections 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.668+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/2183846904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d258bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.668+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/2183846904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 0x5645d2589210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.668+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/2183846904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d2580440 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.668+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2183846904 >> 172.21.15.45:0/2183846904 conn(0x5645d257a980 msgr2=0x5645d257ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.668+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2183846904 shutdown_connections 2024-01-31T05:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.668+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2183846904 wait complete. 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.668+0000 7f15ca3ee740 1 -- start start 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.668+0000 7f15ca3ee740 1 Processor -- start 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15ca3ee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d25866b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15ca3ee740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 0x5645d2586bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15ca3ee740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d2587130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15ca3ee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d24a11b0 con 0x5645d2580060 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15ca3ee740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d24a0e50 con 0x5645d2589750 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15ca3ee740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d2466090 con 0x5645d2580980 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15ca3ee740 10 client.0 _ll_register_callbacks cb 0x5645d248ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c53c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d25866b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c4bc5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 0x5645d2586bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c5bc7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d2587130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c53c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d25866b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51398/0 (socket says 172.21.15.45:51398) 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c4bc5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 0x5645d2586bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:57742/0 (socket says 172.21.15.45:57742) 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c53c6640 1 -- 172.21.15.45:0/2025230176 learned_addr learned my addr 172.21.15.45:0/2025230176 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c5bc7640 1 -- 172.21.15.45:0/2025230176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 msgr2=0x5645d2586bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c5bc7640 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 0x5645d2586bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c5bc7640 1 -- 172.21.15.45:0/2025230176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 msgr2=0x5645d25866b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c5bc7640 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d25866b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c5bc7640 1 -- 172.21.15.45:0/2025230176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc009950 con 0x5645d2589750 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c53c6640 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d25866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.669+0000 7f15c4bc5640 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 0x5645d2586bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15c5bc7640 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d2587130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f15b80ecf50 tx=0x7f15b8004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e928ffcbe0c31645 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15a8ff9640 1 -- 172.21.15.45:0/2025230176 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b80bc600 con 0x5645d2589750 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15a8ff9640 1 -- 172.21.15.45:0/2025230176 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f15b80c0750 con 0x5645d2589750 2024-01-31T05:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2025230176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645d24e1010 con 0x5645d2589750 2024-01-31T05:29:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15a8ff9640 1 -- 172.21.15.45:0/2025230176 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15b80c4990 con 0x5645d2589750 2024-01-31T05:29:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15ca3ee740 10 client.25436 fetch_fsmap learned FSMap version 27 2024-01-31T05:29:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2025230176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5645d24e66a0 con 0x5645d2589750 2024-01-31T05:29:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15a8ff9640 1 -- 172.21.15.45:0/2025230176 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5645d24e1010 con 0x5645d2589750 2024-01-31T05:29:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15a8ff9640 1 -- 172.21.15.45:0/2025230176 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f15b80bc860 con 0x5645d2589750 2024-01-31T05:29:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15ca3ee740 10 client.25436 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:29:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15ca3ee740 10 client.25436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:29:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15ca3ee740 20 client.25436 populate_metadata read hostname 'smithi045' 2024-01-31T05:29:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5645d25992b0 0x5645d259b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15ca3ee740 4 client.25436 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:29:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.670+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2025230176 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645d24e0d30 con 0x5645d25992b0 2024-01-31T05:29:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.671+0000 7f15c53c6640 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5645d25992b0 0x5645d259b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.671+0000 7f15a8ff9640 10 client.25436 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.671+0000 7f15c53c6640 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5645d25992b0 0x5645d259b6d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa340adfd4d8f86 server_cookie=df796d53bc0e88c in_seq=0 out_seq=0 2024-01-31T05:29:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15a8ff9640 1 -- 172.21.15.45:0/2025230176 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5645d24e0d30 con 0x5645d25992b0 2024-01-31T05:29:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15a8ff9640 10 client.25436 handle_command_reply: tid=0 2024-01-31T05:29:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15ca3ee740 1 client.25436 shutdown 2024-01-31T05:29:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2025230176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 msgr2=0x5645d2587130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d2587130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f15b80ecf50 tx=0x7f15b8004180 comp rx=0 tx=0).stop 2024-01-31T05:29:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2025230176 shutdown_connections 2024-01-31T05:29:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5645d25992b0 0x5645d259b6d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d2587130 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15a8ff9640 0 client.25436 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580980 0x5645d2586bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/2025230176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d25866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2025230176 >> 172.21.15.45:0/2025230176 conn(0x5645d257a980 msgr2=0x5645d257c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2025230176 shutdown_connections 2024-01-31T05:29:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15ca3ee740 1 -- 172.21.15.45:0/2025230176 wait complete. 2024-01-31T05:29:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.672+0000 7f15ca3ee740 20 client.25436 trim_cache size 0 max 16384 2024-01-31T05:29:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.686+0000 7f15ca3ee740 1 Processor -- start 2024-01-31T05:29:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.686+0000 7f15ca3ee740 1 -- start start 2024-01-31T05:29:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15ca3ee740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580060 0x5645d2588de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15ca3ee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580980 0x5645d2586f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15ca3ee740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d2583970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15ca3ee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d25a06a0 con 0x5645d2580980 2024-01-31T05:29:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15ca3ee740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d24e0d30 con 0x5645d2589750 2024-01-31T05:29:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15ca3ee740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d2596000 con 0x5645d2580060 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15c4bc5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580980 0x5645d2586f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15c53c6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580060 0x5645d2588de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15c4bc5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580980 0x5645d2586f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51400/0 (socket says 172.21.15.45:51400) 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15c4bc5640 1 -- 172.21.15.45:0/3572572874 learned_addr learned my addr 172.21.15.45:0/3572572874 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15c5bc7640 1 --2- 172.21.15.45:0/3572572874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d2583970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15c4bc5640 1 -- 172.21.15.45:0/3572572874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580060 msgr2=0x5645d2588de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15c4bc5640 1 --2- 172.21.15.45:0/3572572874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580060 0x5645d2588de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15c4bc5640 1 -- 172.21.15.45:0/3572572874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 msgr2=0x5645d2583970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15c4bc5640 1 --2- 172.21.15.45:0/3572572874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d2583970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15c4bc5640 1 -- 172.21.15.45:0/3572572874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645d2483030 con 0x5645d2580980 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15c53c6640 1 --2- 172.21.15.45:0/3572572874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580060 0x5645d2588de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15c4bc5640 1 --2- 172.21.15.45:0/3572572874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580980 0x5645d2586f50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f15b0005850 tx=0x7f15b0010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ede7aaec52d311 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15aaffd640 1 -- 172.21.15.45:0/3572572874 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0013030 con 0x5645d2580980 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.687+0000 7f15aaffd640 1 -- 172.21.15.45:0/3572572874 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15b00054e0 con 0x5645d2580980 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.688+0000 7f15aaffd640 1 -- 172.21.15.45:0/3572572874 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0024040 con 0x5645d2580980 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.688+0000 7f15ca3ee740 1 -- 172.21.15.45:0/3572572874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580980 msgr2=0x5645d2586f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.688+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/3572572874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580980 0x5645d2586f50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f15b0005850 tx=0x7f15b0010040 comp rx=0 tx=0).stop 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.688+0000 7f15ca3ee740 1 -- 172.21.15.45:0/3572572874 shutdown_connections 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.688+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/3572572874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580980 0x5645d2586f50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.688+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/3572572874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2589750 0x5645d2583970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.688+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/3572572874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2580060 0x5645d2588de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.688+0000 7f15ca3ee740 1 -- 172.21.15.45:0/3572572874 >> 172.21.15.45:0/3572572874 conn(0x5645d257a980 msgr2=0x5645d2584fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.688+0000 7f15ca3ee740 1 -- 172.21.15.45:0/3572572874 shutdown_connections 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.688+0000 7f15ca3ee740 1 -- 172.21.15.45:0/3572572874 wait complete. 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15ca3ee740 1 -- start start 2024-01-31T05:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15ca3ee740 1 Processor -- start 2024-01-31T05:29:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15ca3ee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d2590780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15ca3ee740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2580980 0x5645d258eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15ca3ee740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2589750 0x5645d258f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15ca3ee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d258f730 con 0x5645d2580060 2024-01-31T05:29:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15ca3ee740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d258f8a0 con 0x5645d2580980 2024-01-31T05:29:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15ca3ee740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d25ef7c0 con 0x5645d2589750 2024-01-31T05:29:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15c53c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d2590780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15ca3ee740 10 client.0 _ll_register_callbacks cb 0x7f15bc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:29:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15c53c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d2590780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51410/0 (socket says 172.21.15.45:51410) 2024-01-31T05:29:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15c53c6640 1 -- 172.21.15.45:0/504161720 learned_addr learned my addr 172.21.15.45:0/504161720 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15c5bc7640 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2589750 0x5645d258f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15c4bc5640 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2580980 0x5645d258eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15c53c6640 1 -- 172.21.15.45:0/504161720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2589750 msgr2=0x5645d258f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15c53c6640 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2589750 0x5645d258f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15c53c6640 1 -- 172.21.15.45:0/504161720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2580980 msgr2=0x5645d258eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15c53c6640 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2580980 0x5645d258eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.689+0000 7f15c53c6640 1 -- 172.21.15.45:0/504161720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc01d040 con 0x5645d2580060 2024-01-31T05:29:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15c53c6640 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d2590780 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f15bc003cd0 tx=0x7f15bc009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333f416b2ec361cb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15a8ff9640 1 -- 172.21.15.45:0/504161720 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc029040 con 0x5645d2580060 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15a8ff9640 1 -- 172.21.15.45:0/504161720 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f15bc004010 con 0x5645d2580060 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15ca3ee740 1 -- 172.21.15.45:0/504161720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645d2581580 con 0x5645d2580060 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15a8ff9640 1 -- 172.21.15.45:0/504161720 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15bc003900 con 0x5645d2580060 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15ca3ee740 10 client.15921 fetch_fsmap learned FSMap version 27 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15ca3ee740 1 -- 172.21.15.45:0/504161720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f15b0016040 con 0x5645d2580060 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15a8ff9640 1 -- 172.21.15.45:0/504161720 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15bc03a020 con 0x5645d2580060 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15a8ff9640 1 -- 172.21.15.45:0/504161720 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f15bc019770 con 0x5645d2580060 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15ca3ee740 10 client.15921 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15ca3ee740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15ca3ee740 20 client.15921 populate_metadata read hostname 'smithi045' 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5645d25f0e20 0x5645d25f3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15ca3ee740 4 client.15921 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.690+0000 7f15ca3ee740 1 -- 172.21.15.45:0/504161720 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f15bc004220 con 0x5645d25f0e20 2024-01-31T05:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.691+0000 7f15c4bc5640 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5645d25f0e20 0x5645d25f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.691+0000 7f15c4bc5640 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5645d25f0e20 0x5645d25f3240 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8524bf01ca5f4f0c server_cookie=dd6288e8ea8e41ed in_seq=0 out_seq=0 2024-01-31T05:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.691+0000 7f15a8ff9640 10 client.15921 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15a8ff9640 1 -- 172.21.15.45:0/504161720 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f15bc004220 con 0x5645d25f0e20 2024-01-31T05:29:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15a8ff9640 10 client.15921 handle_command_reply: tid=0 2024-01-31T05:29:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15ca3ee740 1 client.15921 shutdown 2024-01-31T05:29:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15ca3ee740 1 -- 172.21.15.45:0/504161720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 msgr2=0x5645d2590780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d2590780 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f15bc003cd0 tx=0x7f15bc009330 comp rx=0 tx=0).stop 2024-01-31T05:29:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15ca3ee740 1 -- 172.21.15.45:0/504161720 shutdown_connections 2024-01-31T05:29:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5645d25f0e20 0x5645d25f3240 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645d2589750 0x5645d258f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5645d2580980 0x5645d258eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15a8ff9640 0 client.15921 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15ca3ee740 1 --2- 172.21.15.45:0/504161720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645d2580060 0x5645d2590780 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15ca3ee740 1 -- 172.21.15.45:0/504161720 >> 172.21.15.45:0/504161720 conn(0x5645d257a980 msgr2=0x5645d2585410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15ca3ee740 1 -- 172.21.15.45:0/504161720 shutdown_connections 2024-01-31T05:29:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15ca3ee740 1 -- 172.21.15.45:0/504161720 wait complete. 2024-01-31T05:29:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.701+0000 7f15ca3ee740 20 client.15921 trim_cache size 0 max 16384 2024-01-31T05:29:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:29:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:29:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:29:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:29:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.702+0000 7f15c762a640 1 -- 172.21.15.45:0/2085695421 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f15ac0a14f0 msgr2=0x7f15ac0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.702+0000 7f15c762a640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f15ac0a14f0 0x7f15ac0a39b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f15c00fcfa0 tx=0x7f15b000e820 comp rx=0 tx=0).stop 2024-01-31T05:29:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.702+0000 7f15c762a640 1 -- 172.21.15.45:0/2085695421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c00fbfe0 msgr2=0x7f15c01a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.702+0000 7f15c762a640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c00fbfe0 0x7f15c01a53d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f15b8005e30 tx=0x7f15b80325c0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.702+0000 7f15c762a640 1 -- 172.21.15.45:0/2085695421 shutdown_connections 2024-01-31T05:29:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.702+0000 7f15c762a640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f15ac0a14f0 0x7f15ac0a39b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.702+0000 7f15c762a640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c00fbfe0 0x7f15c01a53d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.702+0000 7f15c762a640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c0104670 0x7f15c00fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.702+0000 7f15c762a640 1 --2- 172.21.15.45:0/2085695421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c00ff5f0 0x7f15c00fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.702+0000 7f15c762a640 1 -- 172.21.15.45:0/2085695421 >> 172.21.15.45:0/2085695421 conn(0x7f15c0076110 msgr2=0x7f15c00fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.702+0000 7f15c762a640 1 -- 172.21.15.45:0/2085695421 shutdown_connections 2024-01-31T05:29:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:10.702+0000 7f15c762a640 1 -- 172.21.15.45:0/2085695421 wait complete. 2024-01-31T05:29:10.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 666971 11.44 MB/sec execute 190 sec latency 108.538 ms 2024-01-31T05:29:10.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:10 smithi045 ceph-mon[31585]: pgmap v200: 97 pgs: 97 active+clean; 369 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-01-31T05:29:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:10 smithi156 ceph-mon[35720]: pgmap v200: 97 pgs: 97 active+clean; 369 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-01-31T05:29:11.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:10 smithi186 ceph-mon[35659]: pgmap v200: 97 pgs: 97 active+clean; 369 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-01-31T05:29:11.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 668758 11.42 MB/sec execute 191 sec latency 26.952 ms 2024-01-31T05:29:11.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:11 smithi045 ceph-mon[31585]: osdmap e93: 12 total, 12 up, 12 in 2024-01-31T05:29:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:11 smithi156 ceph-mon[35720]: osdmap e93: 12 total, 12 up, 12 in 2024-01-31T05:29:12.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:11 smithi186 ceph-mon[35659]: osdmap e93: 12 total, 12 up, 12 in 2024-01-31T05:29:12.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 671162 11.44 MB/sec execute 192 sec latency 97.671 ms 2024-01-31T05:29:12.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:12 smithi045 ceph-mon[31585]: pgmap v202: 97 pgs: 97 active+clean; 369 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 277 op/s 2024-01-31T05:29:12.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:12 smithi156 ceph-mon[35720]: pgmap v202: 97 pgs: 97 active+clean; 369 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 277 op/s 2024-01-31T05:29:13.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:12 smithi186 ceph-mon[35659]: pgmap v202: 97 pgs: 97 active+clean; 369 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 277 op/s 2024-01-31T05:29:13.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 672549 11.42 MB/sec execute 193 sec latency 56.069 ms 2024-01-31T05:29:13.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:13 smithi156 ceph-mon[35720]: pgmap v203: 97 pgs: 97 active+clean; 380 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.3 MiB/s wr, 251 op/s 2024-01-31T05:29:14.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:13 smithi186 ceph-mon[35659]: pgmap v203: 97 pgs: 97 active+clean; 380 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.3 MiB/s wr, 251 op/s 2024-01-31T05:29:14.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:13 smithi045 ceph-mon[31585]: pgmap v203: 97 pgs: 97 active+clean; 380 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.3 MiB/s wr, 251 op/s 2024-01-31T05:29:14.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 674975 11.43 MB/sec execute 194 sec latency 58.061 ms 2024-01-31T05:29:15.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 676681 11.40 MB/sec execute 195 sec latency 16.387 ms 2024-01-31T05:29:16.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 678727 11.42 MB/sec execute 196 sec latency 61.032 ms 2024-01-31T05:29:16.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:16 smithi045 ceph-mon[31585]: pgmap v204: 97 pgs: 97 active+clean; 380 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.3 MiB/s wr, 251 op/s 2024-01-31T05:29:16.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:16 smithi156 ceph-mon[35720]: pgmap v204: 97 pgs: 97 active+clean; 380 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.3 MiB/s wr, 251 op/s 2024-01-31T05:29:17.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:16 smithi186 ceph-mon[35659]: pgmap v204: 97 pgs: 97 active+clean; 380 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.3 MiB/s wr, 251 op/s 2024-01-31T05:29:17.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 680777 11.40 MB/sec execute 197 sec latency 18.311 ms 2024-01-31T05:29:18.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 682412 11.40 MB/sec execute 198 sec latency 117.533 ms 2024-01-31T05:29:18.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:18 smithi186 ceph-mon[35659]: pgmap v205: 97 pgs: 97 active+clean; 388 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 244 op/s 2024-01-31T05:29:18.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:18 smithi186 ceph-mon[35659]: osdmap e94: 12 total, 12 up, 12 in 2024-01-31T05:29:18.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:18 smithi045 ceph-mon[31585]: pgmap v205: 97 pgs: 97 active+clean; 388 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 244 op/s 2024-01-31T05:29:18.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:18 smithi045 ceph-mon[31585]: osdmap e94: 12 total, 12 up, 12 in 2024-01-31T05:29:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:18 smithi156 ceph-mon[35720]: pgmap v205: 97 pgs: 97 active+clean; 388 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 244 op/s 2024-01-31T05:29:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:18 smithi156 ceph-mon[35720]: osdmap e94: 12 total, 12 up, 12 in 2024-01-31T05:29:19.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 684473 11.38 MB/sec execute 199 sec latency 18.123 ms 2024-01-31T05:29:20.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 686286 11.38 MB/sec execute 200 sec latency 19.312 ms 2024-01-31T05:29:20.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:20 smithi186 ceph-mon[35659]: pgmap v207: 97 pgs: 97 active+clean; 388 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 272 op/s 2024-01-31T05:29:20.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:20 smithi045 ceph-mon[31585]: pgmap v207: 97 pgs: 97 active+clean; 388 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 272 op/s 2024-01-31T05:29:20.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:20 smithi156 ceph-mon[35720]: pgmap v207: 97 pgs: 97 active+clean; 388 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 272 op/s 2024-01-31T05:29:21.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 688936 11.41 MB/sec execute 201 sec latency 15.291 ms 2024-01-31T05:29:22.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 690521 11.39 MB/sec execute 202 sec latency 19.340 ms 2024-01-31T05:29:22.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:22 smithi186 ceph-mon[35659]: pgmap v208: 97 pgs: 97 active+clean; 388 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 244 op/s 2024-01-31T05:29:22.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:22 smithi045 ceph-mon[31585]: pgmap v208: 97 pgs: 97 active+clean; 388 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 244 op/s 2024-01-31T05:29:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:22 smithi156 ceph-mon[35720]: pgmap v208: 97 pgs: 97 active+clean; 388 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 244 op/s 2024-01-31T05:29:23.786 INFO:tasks.workunit.client.0.smithi045.stdout: 1 692711 11.40 MB/sec execute 203 sec latency 121.789 ms 2024-01-31T05:29:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.793+0000 7fd33c867640 1 -- 172.21.15.45:0/1820278671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3380ffb90 msgr2=0x7fd33810db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.793+0000 7fd33c867640 1 --2- 172.21.15.45:0/1820278671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3380ffb90 0x7fd33810db60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd330009cd0 tx=0x7fd33002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.793+0000 7fd33c867640 1 -- 172.21.15.45:0/1820278671 shutdown_connections 2024-01-31T05:29:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.793+0000 7fd33c867640 1 --2- 172.21.15.45:0/1820278671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3380ffb90 0x7fd33810db60 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.793+0000 7fd33c867640 1 --2- 172.21.15.45:0/1820278671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3380ff1d0 0x7fd3380ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.793+0000 7fd33c867640 1 --2- 172.21.15.45:0/1820278671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd338105cb0 0x7fd338106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.793+0000 7fd33c867640 1 -- 172.21.15.45:0/1820278671 >> 172.21.15.45:0/1820278671 conn(0x7fd338076110 msgr2=0x7fd3380fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.793+0000 7fd33c867640 1 -- 172.21.15.45:0/1820278671 shutdown_connections 2024-01-31T05:29:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.793+0000 7fd33c867640 1 -- 172.21.15.45:0/1820278671 wait complete. 2024-01-31T05:29:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd33c867640 1 Processor -- start 2024-01-31T05:29:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd33c867640 1 -- start start 2024-01-31T05:29:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd33c867640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3380ff1d0 0x7fd3381971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd33c867640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3380ffb90 0x7fd338197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd33c867640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd338105cb0 0x7fd33819baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd33c867640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33819c0a0 con 0x7fd338105cb0 2024-01-31T05:29:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd33c867640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33819c210 con 0x7fd3380ffb90 2024-01-31T05:29:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd33c867640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33819c3d0 con 0x7fd3380ff1d0 2024-01-31T05:29:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd336d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd338105cb0 0x7fd33819baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd336d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd338105cb0 0x7fd33819baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58896/0 (socket says 172.21.15.45:58896) 2024-01-31T05:29:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd336d9d640 1 -- 172.21.15.45:0/2377752687 learned_addr learned my addr 172.21.15.45:0/2377752687 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd33659c640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3380ff1d0 0x7fd3381971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.794+0000 7fd32ffff640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3380ffb90 0x7fd338197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.795+0000 7fd336d9d640 1 -- 172.21.15.45:0/2377752687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3380ff1d0 msgr2=0x7fd3381971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.795+0000 7fd336d9d640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3380ff1d0 0x7fd3381971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.795+0000 7fd336d9d640 1 -- 172.21.15.45:0/2377752687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3380ffb90 msgr2=0x7fd338197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.795+0000 7fd336d9d640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3380ffb90 0x7fd338197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.795+0000 7fd336d9d640 1 -- 172.21.15.45:0/2377752687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320009490 con 0x7fd338105cb0 2024-01-31T05:29:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.795+0000 7fd336d9d640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd338105cb0 0x7fd33819baa0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd330009ca0 tx=0x7fd330032550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.795+0000 7fd32f7fe640 1 -- 172.21.15.45:0/2377752687 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd330032690 con 0x7fd338105cb0 2024-01-31T05:29:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.795+0000 7fd32f7fe640 1 -- 172.21.15.45:0/2377752687 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd330003970 con 0x7fd338105cb0 2024-01-31T05:29:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.795+0000 7fd32f7fe640 1 -- 172.21.15.45:0/2377752687 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3300045e0 con 0x7fd338105cb0 2024-01-31T05:29:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.795+0000 7fd33c867640 1 -- 172.21.15.45:0/2377752687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd330009900 con 0x7fd338105cb0 2024-01-31T05:29:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.795+0000 7fd33c867640 1 -- 172.21.15.45:0/2377752687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3381a3d50 con 0x7fd338105cb0 2024-01-31T05:29:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.797+0000 7fd32f7fe640 1 -- 172.21.15.45:0/2377752687 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fd33003b040 con 0x7fd338105cb0 2024-01-31T05:29:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.797+0000 7fd33f621740 1 Processor -- start 2024-01-31T05:29:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd33f621740 1 -- start start 2024-01-31T05:29:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd33f621740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e7e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd33f621740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e87210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd33f621740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 0x55cd29e89b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd33f621740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd29ded040 con 0x55cd29e7e060 2024-01-31T05:29:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd33f621740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd29dea730 con 0x55cd29e7e980 2024-01-31T05:29:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd33f621740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd29de6020 con 0x55cd29e87750 2024-01-31T05:29:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd32ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e7e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd32ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e7e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58906/0 (socket says 172.21.15.45:58906) 2024-01-31T05:29:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd32ffff640 1 -- 172.21.15.45:0/2350533924 learned_addr learned my addr 172.21.15.45:0/2350533924 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd32f7fe640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd3240a14f0 0x7fd3240a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd32ffff640 1 --2- 172.21.15.45:0/2350533924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 0x55cd29e89b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd33659c640 1 --2- 172.21.15.45:0/2350533924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e87210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd336d9d640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd3240a14f0 0x7fd3240a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd32ffff640 1 -- 172.21.15.45:0/2350533924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 msgr2=0x55cd29e89b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd32f7fe640 1 -- 172.21.15.45:0/2377752687 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd3300ef790 con 0x7fd338105cb0 2024-01-31T05:29:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd32ffff640 1 --2- 172.21.15.45:0/2350533924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 0x55cd29e89b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd32ffff640 1 -- 172.21.15.45:0/2350533924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 msgr2=0x55cd29e87210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd32ffff640 1 --2- 172.21.15.45:0/2350533924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e87210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd32ffff640 1 -- 172.21.15.45:0/2350533924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd29ddde40 con 0x55cd29e7e060 2024-01-31T05:29:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.798+0000 7fd33659c640 1 --2- 172.21.15.45:0/2350533924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e87210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:29:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.799+0000 7fd32ffff640 1 --2- 172.21.15.45:0/2350533924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e7e440 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd320005bc0 tx=0x7fd3200099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142ae227c6202de7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.799+0000 7fd336d9d640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd3240a14f0 0x7fd3240a39b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3300bc630 tx=0x7fd3300c0040 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.799+0000 7fd32cff9640 1 -- 172.21.15.45:0/2350533924 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320013120 con 0x55cd29e7e060 2024-01-31T05:29:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.799+0000 7fd32cff9640 1 -- 172.21.15.45:0/2350533924 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd320004510 con 0x55cd29e7e060 2024-01-31T05:29:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.799+0000 7fd32cff9640 1 -- 172.21.15.45:0/2350533924 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32000fe80 con 0x55cd29e7e060 2024-01-31T05:29:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 -- 172.21.15.45:0/2350533924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 msgr2=0x55cd29e7e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 --2- 172.21.15.45:0/2350533924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e7e440 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd320005bc0 tx=0x7fd3200099e0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 -- 172.21.15.45:0/2350533924 shutdown_connections 2024-01-31T05:29:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 --2- 172.21.15.45:0/2350533924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 0x55cd29e89b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 --2- 172.21.15.45:0/2350533924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e87210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 --2- 172.21.15.45:0/2350533924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e7e440 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 -- 172.21.15.45:0/2350533924 >> 172.21.15.45:0/2350533924 conn(0x55cd29e78980 msgr2=0x55cd29e78d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 -- 172.21.15.45:0/2350533924 shutdown_connections 2024-01-31T05:29:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 -- 172.21.15.45:0/2350533924 wait complete. 2024-01-31T05:29:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 -- start start 2024-01-31T05:29:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 Processor -- start 2024-01-31T05:29:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e060 0x55cd29e84640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e7e980 0x55cd29e84b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e87750 0x55cd29e850c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd29d9f1b0 con 0x55cd29e87750 2024-01-31T05:29:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd29d9ee50 con 0x55cd29e7e060 2024-01-31T05:29:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd29d64090 con 0x55cd29e7e980 2024-01-31T05:29:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.800+0000 7fd33f621740 10 client.0 _ll_register_callbacks cb 0x55cd29d89a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:29:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd32ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e060 0x55cd29e84640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd32ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e060 0x55cd29e84640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:59146/0 (socket says 172.21.15.45:59146) 2024-01-31T05:29:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd32ffff640 1 -- 172.21.15.45:0/1148396980 learned_addr learned my addr 172.21.15.45:0/1148396980 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd33659c640 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e7e980 0x55cd29e84b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd32ffff640 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e87750 0x55cd29e850c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd32ffff640 1 -- 172.21.15.45:0/1148396980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e7e980 msgr2=0x55cd29e84b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd32ffff640 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e7e980 0x55cd29e84b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd32ffff640 1 -- 172.21.15.45:0/1148396980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e87750 msgr2=0x55cd29e850c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd32ffff640 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e87750 0x55cd29e850c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd32ffff640 1 -- 172.21.15.45:0/1148396980 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32001b040 con 0x55cd29e7e060 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd33659c640 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e7e980 0x55cd29e84b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd32ffff640 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e060 0x55cd29e84640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd32000fd00 tx=0x7fd32000fa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27754e07eed75216 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd31dffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd31dffb640 1 -- 172.21.15.45:0/1148396980 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320020430 con 0x55cd29e7e060 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd31dffb640 1 -- 172.21.15.45:0/1148396980 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd320020590 con 0x55cd29e7e060 2024-01-31T05:29:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd33f621740 1 -- 172.21.15.45:0/1148396980 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd29ddf010 con 0x55cd29e7e060 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd31dffb640 1 -- 172.21.15.45:0/1148396980 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd320020750 con 0x55cd29e7e060 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd33f621740 10 client.25454 fetch_fsmap learned FSMap version 27 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.801+0000 7fd33f621740 1 -- 172.21.15.45:0/1148396980 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cd29de46a0 con 0x55cd29e7e060 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.802+0000 7fd31dffb640 1 -- 172.21.15.45:0/1148396980 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd29ddf010 con 0x55cd29e7e060 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.802+0000 7fd31dffb640 1 -- 172.21.15.45:0/1148396980 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd320010880 con 0x55cd29e7e060 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.802+0000 7fd33f621740 10 client.25454 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.802+0000 7fd33f621740 10 client.25454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.802+0000 7fd33f621740 20 client.25454 populate_metadata read hostname 'smithi045' 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.802+0000 7fd33f621740 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55cd29e97360 0x55cd29e99750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.802+0000 7fd33f621740 4 client.25454 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.802+0000 7fd33f621740 1 -- 172.21.15.45:0/1148396980 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd29dded30 con 0x55cd29e97360 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.802+0000 7fd33659c640 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55cd29e97360 0x55cd29e99750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.802+0000 7fd33659c640 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55cd29e97360 0x55cd29e99750 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=485084c19b36b535 server_cookie=e5ef8e3e0d0c95ea in_seq=0 out_seq=0 2024-01-31T05:29:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.802+0000 7fd31dffb640 10 client.25454 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd31dffb640 1 -- 172.21.15.45:0/1148396980 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd29dded30 con 0x55cd29e97360 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd31dffb640 10 client.25454 handle_command_reply: tid=0 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd33f621740 1 client.25454 shutdown 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd33f621740 1 -- 172.21.15.45:0/1148396980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e060 msgr2=0x55cd29e84640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd33f621740 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e060 0x55cd29e84640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd32000fd00 tx=0x7fd32000fa20 comp rx=0 tx=0).stop 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd33f621740 1 -- 172.21.15.45:0/1148396980 shutdown_connections 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd33f621740 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55cd29e97360 0x55cd29e99750 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd33f621740 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e87750 0x55cd29e850c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd33f621740 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e7e980 0x55cd29e84b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd31dffb640 0 client.25454 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd33f621740 1 --2- 172.21.15.45:0/1148396980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e060 0x55cd29e84640 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd33f621740 1 -- 172.21.15.45:0/1148396980 >> 172.21.15.45:0/1148396980 conn(0x55cd29e78980 msgr2=0x55cd29e7a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd33f621740 1 -- 172.21.15.45:0/1148396980 shutdown_connections 2024-01-31T05:29:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd33f621740 1 -- 172.21.15.45:0/1148396980 wait complete. 2024-01-31T05:29:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.803+0000 7fd33f621740 20 client.25454 trim_cache size 0 max 16384 2024-01-31T05:29:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd33f621740 1 Processor -- start 2024-01-31T05:29:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd33f621740 1 -- start start 2024-01-31T05:29:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd33f621740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e86d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd33f621740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e7f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd33f621740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 0x55cd29e818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd33f621740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd29dded30 con 0x55cd29e7e060 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd33f621740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd29ddf010 con 0x55cd29e7e980 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd33f621740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd29e72f20 con 0x55cd29e87750 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd32ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e86d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd32ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e86d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58932/0 (socket says 172.21.15.45:58932) 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd32ffff640 1 -- 172.21.15.45:0/3798994332 learned_addr learned my addr 172.21.15.45:0/3798994332 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd33659c640 1 --2- 172.21.15.45:0/3798994332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e7f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd32ffff640 1 --2- 172.21.15.45:0/3798994332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 0x55cd29e818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd32ffff640 1 -- 172.21.15.45:0/3798994332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 msgr2=0x55cd29e818c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd32ffff640 1 --2- 172.21.15.45:0/3798994332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 0x55cd29e818c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd32ffff640 1 -- 172.21.15.45:0/3798994332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 msgr2=0x55cd29e7f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd32ffff640 1 --2- 172.21.15.45:0/3798994332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e7f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd32ffff640 1 -- 172.21.15.45:0/3798994332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd29d81030 con 0x55cd29e7e060 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd32ffff640 1 --2- 172.21.15.45:0/3798994332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e86d70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd3200092d0 tx=0x7fd320005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f2670870cc7b96 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.818+0000 7fd31ffff640 1 -- 172.21.15.45:0/3798994332 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3200134f0 con 0x55cd29e7e060 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.819+0000 7fd31ffff640 1 -- 172.21.15.45:0/3798994332 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd320024070 con 0x55cd29e7e060 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.819+0000 7fd31ffff640 1 -- 172.21.15.45:0/3798994332 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320005470 con 0x55cd29e7e060 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.819+0000 7fd33f621740 1 -- 172.21.15.45:0/3798994332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 msgr2=0x55cd29e86d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.819+0000 7fd33f621740 1 --2- 172.21.15.45:0/3798994332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e86d70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd3200092d0 tx=0x7fd320005b90 comp rx=0 tx=0).stop 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.819+0000 7fd33f621740 1 -- 172.21.15.45:0/3798994332 shutdown_connections 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.819+0000 7fd33f621740 1 --2- 172.21.15.45:0/3798994332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 0x55cd29e818c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.819+0000 7fd33f621740 1 --2- 172.21.15.45:0/3798994332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e7f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.819+0000 7fd33f621740 1 --2- 172.21.15.45:0/3798994332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e86d70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.819+0000 7fd33f621740 1 -- 172.21.15.45:0/3798994332 >> 172.21.15.45:0/3798994332 conn(0x55cd29e78980 msgr2=0x55cd29e7eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.819+0000 7fd33f621740 1 -- 172.21.15.45:0/3798994332 shutdown_connections 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.819+0000 7fd33f621740 1 -- 172.21.15.45:0/3798994332 wait complete. 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33f621740 1 -- start start 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33f621740 1 Processor -- start 2024-01-31T05:29:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33f621740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e8e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33f621740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e8e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33f621740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 0x55cd29e8c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33f621740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd29e8d050 con 0x55cd29e7e060 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33f621740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd29e8d1c0 con 0x55cd29e7e980 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33f621740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd29e8d380 con 0x55cd29e87750 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33f621740 10 client.0 _ll_register_callbacks cb 0x7fd320009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd32ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e8e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd32ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e8e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58948/0 (socket says 172.21.15.45:58948) 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd32ffff640 1 -- 172.21.15.45:0/3531061943 learned_addr learned my addr 172.21.15.45:0/3531061943 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33659c640 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e8e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd32ffff640 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 0x55cd29e8c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33659c640 1 -- 172.21.15.45:0/3531061943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 msgr2=0x55cd29e8c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33659c640 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 0x55cd29e8c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33659c640 1 -- 172.21.15.45:0/3531061943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 msgr2=0x55cd29e8e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33659c640 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e8e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd33659c640 1 -- 172.21.15.45:0/3531061943 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd29de46a0 con 0x55cd29e7e980 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.820+0000 7fd32ffff640 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e8e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd33659c640 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e8e900 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55cd29e9a000 tx=0x7fd328012c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=174cf0549b390f7a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd31dffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd31dffb640 1 -- 172.21.15.45:0/3531061943 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328017040 con 0x55cd29e7e980 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd33f621740 1 -- 172.21.15.45:0/3531061943 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd320004510 con 0x55cd29e7e980 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd31dffb640 1 -- 172.21.15.45:0/3531061943 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd328021710 con 0x55cd29e7e980 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd31dffb640 1 -- 172.21.15.45:0/3531061943 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3280218d0 con 0x55cd29e7e980 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd33f621740 10 client.25460 fetch_fsmap learned FSMap version 27 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd33f621740 1 -- 172.21.15.45:0/3531061943 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cd29da6a50 con 0x55cd29e7e980 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd31dffb640 1 -- 172.21.15.45:0/3531061943 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd320004510 con 0x55cd29e7e980 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd31dffb640 1 -- 172.21.15.45:0/3531061943 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd328003be0 con 0x55cd29e7e980 2024-01-31T05:29:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd33f621740 10 client.25460 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:29:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd33f621740 10 client.25460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:29:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd33f621740 20 client.25460 populate_metadata read hostname 'smithi045' 2024-01-31T05:29:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd33f621740 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55cd29eeec70 0x55cd29ef1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd33f621740 4 client.25460 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:29:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.821+0000 7fd33f621740 1 -- 172.21.15.45:0/3531061943 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cd29ae5600 con 0x55cd29eeec70 2024-01-31T05:29:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.822+0000 7fd32ffff640 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55cd29eeec70 0x55cd29ef1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.822+0000 7fd31dffb640 10 client.25460 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.822+0000 7fd32ffff640 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55cd29eeec70 0x55cd29ef1090 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d02478b8d88b server_cookie=a2c30940da212de in_seq=0 out_seq=0 2024-01-31T05:29:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd31dffb640 1 -- 172.21.15.45:0/3531061943 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cd29ae5600 con 0x55cd29eeec70 2024-01-31T05:29:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd31dffb640 10 client.25460 handle_command_reply: tid=0 2024-01-31T05:29:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd33f621740 1 client.25460 shutdown 2024-01-31T05:29:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd33f621740 1 -- 172.21.15.45:0/3531061943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 msgr2=0x55cd29e8e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd33f621740 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e8e900 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55cd29e9a000 tx=0x7fd328012c40 comp rx=0 tx=0).stop 2024-01-31T05:29:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd33f621740 1 -- 172.21.15.45:0/3531061943 shutdown_connections 2024-01-31T05:29:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd33f621740 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55cd29eeec70 0x55cd29ef1090 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd33f621740 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd29e87750 0x55cd29e8c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd33f621740 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd29e7e980 0x55cd29e8e900 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd33f621740 1 --2- 172.21.15.45:0/3531061943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd29e7e060 0x55cd29e8e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd31dffb640 0 client.25460 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd33f621740 1 -- 172.21.15.45:0/3531061943 >> 172.21.15.45:0/3531061943 conn(0x55cd29e78980 msgr2=0x55cd29e7f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd33f621740 1 -- 172.21.15.45:0/3531061943 shutdown_connections 2024-01-31T05:29:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd33f621740 1 -- 172.21.15.45:0/3531061943 wait complete. 2024-01-31T05:29:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.832+0000 7fd33f621740 20 client.25460 trim_cache size 0 max 16384 2024-01-31T05:29:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:29:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:29:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:29:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:29:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.833+0000 7fd33c867640 1 -- 172.21.15.45:0/2377752687 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd3240a14f0 msgr2=0x7fd3240a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.833+0000 7fd33c867640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd3240a14f0 0x7fd3240a39b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3300bc630 tx=0x7fd3300c0040 comp rx=0 tx=0).stop 2024-01-31T05:29:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.833+0000 7fd33c867640 1 -- 172.21.15.45:0/2377752687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd338105cb0 msgr2=0x7fd33819baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.833+0000 7fd33c867640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd338105cb0 0x7fd33819baa0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd330009ca0 tx=0x7fd330032550 comp rx=0 tx=0).stop 2024-01-31T05:29:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.833+0000 7fd33c867640 1 -- 172.21.15.45:0/2377752687 shutdown_connections 2024-01-31T05:29:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.833+0000 7fd33c867640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd3240a14f0 0x7fd3240a39b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.833+0000 7fd33c867640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd338105cb0 0x7fd33819baa0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.833+0000 7fd33c867640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3380ffb90 0x7fd338197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.833+0000 7fd33c867640 1 --2- 172.21.15.45:0/2377752687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3380ff1d0 0x7fd3381971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.833+0000 7fd33c867640 1 -- 172.21.15.45:0/2377752687 >> 172.21.15.45:0/2377752687 conn(0x7fd338076110 msgr2=0x7fd338103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.833+0000 7fd33c867640 1 -- 172.21.15.45:0/2377752687 shutdown_connections 2024-01-31T05:29:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:23.833+0000 7fd33c867640 1 -- 172.21.15.45:0/2377752687 wait complete. 2024-01-31T05:29:24.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 694362 11.37 MB/sec execute 204 sec latency 22.818 ms 2024-01-31T05:29:24.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:24 smithi186 ceph-mon[35659]: pgmap v209: 97 pgs: 97 active+clean; 396 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 228 op/s 2024-01-31T05:29:24.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:24 smithi045 ceph-mon[31585]: pgmap v209: 97 pgs: 97 active+clean; 396 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 228 op/s 2024-01-31T05:29:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:24 smithi156 ceph-mon[35720]: pgmap v209: 97 pgs: 97 active+clean; 396 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 228 op/s 2024-01-31T05:29:25.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 696634 11.38 MB/sec execute 205 sec latency 12.860 ms 2024-01-31T05:29:26.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 697955 11.35 MB/sec execute 206 sec latency 20.179 ms 2024-01-31T05:29:26.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:26 smithi186 ceph-mon[35659]: pgmap v210: 97 pgs: 97 active+clean; 396 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 228 op/s 2024-01-31T05:29:26.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:26 smithi045 ceph-mon[31585]: pgmap v210: 97 pgs: 97 active+clean; 396 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 228 op/s 2024-01-31T05:29:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:26 smithi156 ceph-mon[35720]: pgmap v210: 97 pgs: 97 active+clean; 396 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 228 op/s 2024-01-31T05:29:27.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 700178 11.36 MB/sec execute 207 sec latency 15.121 ms 2024-01-31T05:29:28.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 702342 11.36 MB/sec execute 208 sec latency 109.830 ms 2024-01-31T05:29:28.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:28 smithi186 ceph-mon[35659]: pgmap v211: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 231 op/s 2024-01-31T05:29:28.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:28 smithi045 ceph-mon[31585]: pgmap v211: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 231 op/s 2024-01-31T05:29:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:28 smithi156 ceph-mon[35720]: pgmap v211: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 231 op/s 2024-01-31T05:29:29.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 704171 11.35 MB/sec execute 209 sec latency 19.840 ms 2024-01-31T05:29:30.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 706784 11.38 MB/sec execute 210 sec latency 15.992 ms 2024-01-31T05:29:30.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:30 smithi186 ceph-mon[35659]: pgmap v212: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-01-31T05:29:30.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:30 smithi045 ceph-mon[31585]: pgmap v212: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-01-31T05:29:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:30 smithi156 ceph-mon[35720]: pgmap v212: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-01-31T05:29:31.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 708126 11.35 MB/sec execute 211 sec latency 214.536 ms 2024-01-31T05:29:32.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 710429 11.36 MB/sec execute 212 sec latency 246.641 ms 2024-01-31T05:29:32.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:32 smithi186 ceph-mon[35659]: pgmap v213: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-01-31T05:29:32.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:32 smithi045 ceph-mon[31585]: pgmap v213: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-01-31T05:29:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:32 smithi156 ceph-mon[35720]: pgmap v213: 97 pgs: 97 active+clean; 403 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-01-31T05:29:33.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 711928 11.34 MB/sec execute 213 sec latency 72.373 ms 2024-01-31T05:29:34.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 714221 11.35 MB/sec execute 214 sec latency 22.266 ms 2024-01-31T05:29:34.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:34 smithi045 ceph-mon[31585]: pgmap v214: 97 pgs: 97 active+clean; 414 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 288 op/s 2024-01-31T05:29:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:34 smithi156 ceph-mon[35720]: pgmap v214: 97 pgs: 97 active+clean; 414 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 288 op/s 2024-01-31T05:29:35.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:34 smithi186 ceph-mon[35659]: pgmap v214: 97 pgs: 97 active+clean; 414 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 288 op/s 2024-01-31T05:29:35.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 716373 11.33 MB/sec execute 215 sec latency 15.938 ms 2024-01-31T05:29:36.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 718197 11.34 MB/sec execute 216 sec latency 63.617 ms 2024-01-31T05:29:36.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:36 smithi045 ceph-mon[31585]: pgmap v215: 97 pgs: 97 active+clean; 414 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-01-31T05:29:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.922+0000 7f78aeec9640 1 -- 172.21.15.45:0/1420280491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a8106000 msgr2=0x7f78a810dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.922+0000 7f78aeec9640 1 --2- 172.21.15.45:0/1420280491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a8106000 0x7f78a810dbd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009cd0 tx=0x7f78a002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.922+0000 7f78aeec9640 1 -- 172.21.15.45:0/1420280491 shutdown_connections 2024-01-31T05:29:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.922+0000 7f78aeec9640 1 --2- 172.21.15.45:0/1420280491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a8106000 0x7f78a810dbd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.922+0000 7f78aeec9640 1 --2- 172.21.15.45:0/1420280491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78a81018e0 0x7f78a8105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.922+0000 7f78aeec9640 1 --2- 172.21.15.45:0/1420280491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78a8100f30 0x7f78a8101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.922+0000 7f78aeec9640 1 -- 172.21.15.45:0/1420280491 >> 172.21.15.45:0/1420280491 conn(0x7f78a80fc7d0 msgr2=0x7f78a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.922+0000 7f78aeec9640 1 -- 172.21.15.45:0/1420280491 shutdown_connections 2024-01-31T05:29:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.922+0000 7f78aeec9640 1 -- 172.21.15.45:0/1420280491 wait complete. 2024-01-31T05:29:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78aeec9640 1 Processor -- start 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78aeec9640 1 -- start start 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78aeec9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78a8100f30 0x7f78a8197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78aeec9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78a8106000 0x7f78a81976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78aeec9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a819ba40 0x7f78a819bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78aeec9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a819c430 con 0x7f78a819ba40 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78aeec9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a819c5a0 con 0x7f78a8100f30 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78aeec9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a819c710 con 0x7f78a8106000 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78ad466640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a819ba40 0x7f78a819bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78ad466640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a819ba40 0x7f78a819bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54408/0 (socket says 172.21.15.45:54408) 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f789ffff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78a8106000 0x7f78a81976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78ad466640 1 -- 172.21.15.45:0/2692449866 learned_addr learned my addr 172.21.15.45:0/2692449866 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78acc65640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78a8100f30 0x7f78a8197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78ad466640 1 -- 172.21.15.45:0/2692449866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78a8106000 msgr2=0x7f78a81976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78ad466640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78a8106000 0x7f78a81976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78ad466640 1 -- 172.21.15.45:0/2692449866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78a8100f30 msgr2=0x7f78a8197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78ad466640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78a8100f30 0x7f78a8197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.923+0000 7f78ad466640 1 -- 172.21.15.45:0/2692449866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a4009490 con 0x7f78a819ba40 2024-01-31T05:29:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.924+0000 7f78ad466640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a819ba40 0x7f78a819bef0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009ca0 tx=0x7f78a00325e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.924+0000 7f789dffb640 1 -- 172.21.15.45:0/2692449866 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0032890 con 0x7f78a819ba40 2024-01-31T05:29:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.924+0000 7f78aeec9640 1 -- 172.21.15.45:0/2692449866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a0009900 con 0x7f78a819ba40 2024-01-31T05:29:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.924+0000 7f789dffb640 1 -- 172.21.15.45:0/2692449866 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a0003e30 con 0x7f78a819ba40 2024-01-31T05:29:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.924+0000 7f789dffb640 1 -- 172.21.15.45:0/2692449866 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0004690 con 0x7f78a819ba40 2024-01-31T05:29:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.924+0000 7f78aeec9640 1 -- 172.21.15.45:0/2692449866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a81a3fe0 con 0x7f78a819ba40 2024-01-31T05:29:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.926+0000 7f789dffb640 1 -- 172.21.15.45:0/2692449866 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f78a003b040 con 0x7f78a819ba40 2024-01-31T05:29:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f789dffb640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f78900a14f0 0x7f78900a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78b1d6b740 1 Processor -- start 2024-01-31T05:29:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f789dffb640 1 -- 172.21.15.45:0/2692449866 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f78a00ef890 con 0x7f78a819ba40 2024-01-31T05:29:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78b1d6b740 1 -- start start 2024-01-31T05:29:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78acc65640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f78900a14f0 0x7f78900a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78b1d6b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e7c060 0x557f77e7c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78b1d6b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 0x557f77e85210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78b1d6b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e85750 0x557f77e87b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78b1d6b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77deb040 con 0x557f77e7c980 2024-01-31T05:29:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78b1d6b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77de8730 con 0x557f77e85750 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78b1d6b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77de4020 con 0x557f77e7c060 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f789ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 0x557f77e85210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f789ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 0x557f77e85210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54424/0 (socket says 172.21.15.45:54424) 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f789ffff640 1 -- 172.21.15.45:0/3709502852 learned_addr learned my addr 172.21.15.45:0/3709502852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78acc65640 1 --2- 172.21.15.45:0/3709502852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e7c060 0x557f77e7c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78ad466640 1 --2- 172.21.15.45:0/3709502852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e85750 0x557f77e87b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78acc65640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f78900a14f0 0x7f78900a39b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009db0 tx=0x7f78a4011810 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f789ffff640 1 -- 172.21.15.45:0/3709502852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e7c060 msgr2=0x557f77e7c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f789ffff640 1 --2- 172.21.15.45:0/3709502852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e7c060 0x557f77e7c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f789ffff640 1 -- 172.21.15.45:0/3709502852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e85750 msgr2=0x557f77e87b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f789ffff640 1 --2- 172.21.15.45:0/3709502852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e85750 0x557f77e87b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f789ffff640 1 -- 172.21.15.45:0/3709502852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f77ddbe40 con 0x557f77e7c980 2024-01-31T05:29:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.927+0000 7f78ad466640 1 --2- 172.21.15.45:0/3709502852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e85750 0x557f77e87b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.928+0000 7f789ffff640 1 --2- 172.21.15.45:0/3709502852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 0x557f77e85210 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f789800bbf0 tx=0x7f78980107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c2b3fb8c657703 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.928+0000 7f788effd640 1 -- 172.21.15.45:0/3709502852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898005020 con 0x557f77e7c980 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.928+0000 7f788effd640 1 -- 172.21.15.45:0/3709502852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7898015070 con 0x557f77e7c980 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.928+0000 7f788effd640 1 -- 172.21.15.45:0/3709502852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898011440 con 0x557f77e7c980 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.928+0000 7f78b1d6b740 1 -- 172.21.15.45:0/3709502852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 msgr2=0x557f77e85210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.928+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/3709502852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 0x557f77e85210 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f789800bbf0 tx=0x7f78980107a0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.928+0000 7f78b1d6b740 1 -- 172.21.15.45:0/3709502852 shutdown_connections 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.928+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/3709502852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 0x557f77e85210 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.928+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/3709502852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e85750 0x557f77e87b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.928+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/3709502852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e7c060 0x557f77e7c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.928+0000 7f78b1d6b740 1 -- 172.21.15.45:0/3709502852 >> 172.21.15.45:0/3709502852 conn(0x557f77e76980 msgr2=0x557f77e76d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78b1d6b740 1 -- 172.21.15.45:0/3709502852 shutdown_connections 2024-01-31T05:29:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78b1d6b740 1 -- 172.21.15.45:0/3709502852 wait complete. 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78b1d6b740 1 -- start start 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78b1d6b740 1 Processor -- start 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78b1d6b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 0x557f77e82720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78b1d6b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c980 0x557f77e82c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78b1d6b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 0x557f77e831a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78b1d6b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77d9d1b0 con 0x557f77e7c060 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78b1d6b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77d9ce50 con 0x557f77e7c980 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78b1d6b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77d62090 con 0x557f77e85750 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78b1d6b740 10 client.0 _ll_register_callbacks cb 0x557f77d87a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78acc65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 0x557f77e82720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78acc65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 0x557f77e82720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54436/0 (socket says 172.21.15.45:54436) 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78acc65640 1 -- 172.21.15.45:0/2914556469 learned_addr learned my addr 172.21.15.45:0/2914556469 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f789ffff640 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c980 0x557f77e82c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.929+0000 7f78ad466640 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 0x557f77e831a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f78acc65640 1 -- 172.21.15.45:0/2914556469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 msgr2=0x557f77e831a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f78acc65640 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 0x557f77e831a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f78acc65640 1 -- 172.21.15.45:0/2914556469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c980 msgr2=0x557f77e82c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f78acc65640 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c980 0x557f77e82c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f78acc65640 1 -- 172.21.15.45:0/2914556469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789800b820 con 0x557f77e7c060 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f789ffff640 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c980 0x557f77e82c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f78acc65640 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 0x557f77e82720 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f78a400fb30 tx=0x7f78a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421dce21dd1a0499 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f7887fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f7887fff640 1 -- 172.21.15.45:0/2914556469 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a401ea50 con 0x557f77e7c060 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f7887fff640 1 -- 172.21.15.45:0/2914556469 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f78a401ebb0 con 0x557f77e7c060 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2914556469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f77ddd010 con 0x557f77e7c060 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f7887fff640 1 -- 172.21.15.45:0/2914556469 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78a401ed70 con 0x557f77e7c060 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f78b1d6b740 10 client.15951 fetch_fsmap learned FSMap version 27 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2914556469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557f77de26a0 con 0x557f77e7c060 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.930+0000 7f7887fff640 1 -- 172.21.15.45:0/2914556469 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a4033080 con 0x557f77e7c060 2024-01-31T05:29:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.931+0000 7f7887fff640 1 -- 172.21.15.45:0/2914556469 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f78a402d030 con 0x557f77e7c060 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.931+0000 7f78b1d6b740 10 client.15951 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.931+0000 7f78b1d6b740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.931+0000 7f78b1d6b740 20 client.15951 populate_metadata read hostname 'smithi045' 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.931+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f77e99a30 0x557f77e9be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.931+0000 7f78b1d6b740 4 client.15951 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.931+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2914556469 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f77ddcd30 con 0x557f77e99a30 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.931+0000 7f789ffff640 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f77e99a30 0x557f77e9be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.931+0000 7f789ffff640 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f77e99a30 0x557f77e9be50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ba5ffd12ff929d server_cookie=502c037db283d55 in_seq=0 out_seq=0 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.931+0000 7f7887fff640 10 client.15951 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f7887fff640 1 -- 172.21.15.45:0/2914556469 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f77ddcd30 con 0x557f77e99a30 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f7887fff640 10 client.15951 handle_command_reply: tid=0 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f78b1d6b740 1 client.15951 shutdown 2024-01-31T05:29:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2914556469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 msgr2=0x557f77e82720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 0x557f77e82720 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f78a400fb30 tx=0x7f78a4004730 comp rx=0 tx=0).stop 2024-01-31T05:29:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2914556469 shutdown_connections 2024-01-31T05:29:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f77e99a30 0x557f77e9be50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 0x557f77e831a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c980 0x557f77e82c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f7887fff640 0 client.15951 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/2914556469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 0x557f77e82720 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2914556469 >> 172.21.15.45:0/2914556469 conn(0x557f77e76980 msgr2=0x557f77e786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2914556469 shutdown_connections 2024-01-31T05:29:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2914556469 wait complete. 2024-01-31T05:29:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.932+0000 7f78b1d6b740 20 client.15951 trim_cache size 0 max 16384 2024-01-31T05:29:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78b1d6b740 1 Processor -- start 2024-01-31T05:29:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78b1d6b740 1 -- start start 2024-01-31T05:29:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78b1d6b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 0x557f77e84e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78b1d6b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c980 0x557f77e82fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78b1d6b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 0x557f77e7f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78b1d6b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77e83890 con 0x557f77e7c060 2024-01-31T05:29:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78b1d6b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77ddcd60 con 0x557f77e7c980 2024-01-31T05:29:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78b1d6b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77e72240 con 0x557f77e85750 2024-01-31T05:29:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78acc65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 0x557f77e84e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78acc65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 0x557f77e84e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54450/0 (socket says 172.21.15.45:54450) 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78acc65640 1 -- 172.21.15.45:0/1604383986 learned_addr learned my addr 172.21.15.45:0/1604383986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78ad466640 1 --2- 172.21.15.45:0/1604383986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 0x557f77e7f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f789ffff640 1 --2- 172.21.15.45:0/1604383986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c980 0x557f77e82fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78acc65640 1 -- 172.21.15.45:0/1604383986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 msgr2=0x557f77e7f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78acc65640 1 --2- 172.21.15.45:0/1604383986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 0x557f77e7f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78acc65640 1 -- 172.21.15.45:0/1604383986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c980 msgr2=0x557f77e82fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78acc65640 1 --2- 172.21.15.45:0/1604383986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c980 0x557f77e82fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.946+0000 7f78acc65640 1 -- 172.21.15.45:0/1604383986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f77d7f030 con 0x557f77e7c060 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.947+0000 7f78acc65640 1 --2- 172.21.15.45:0/1604383986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 0x557f77e84e50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f78a400f600 tx=0x7f78a4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce225635280d6e40 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.947+0000 7f788e7fc640 1 -- 172.21.15.45:0/1604383986 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a400f940 con 0x557f77e7c060 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.947+0000 7f788e7fc640 1 -- 172.21.15.45:0/1604383986 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f77ddd010 con 0x557f77e7c060 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.947+0000 7f788e7fc640 1 -- 172.21.15.45:0/1604383986 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a402fb80 con 0x557f77e7c060 2024-01-31T05:29:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 -- 172.21.15.45:0/1604383986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 msgr2=0x557f77e84e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/1604383986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 0x557f77e84e50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f78a400f600 tx=0x7f78a4009f60 comp rx=0 tx=0).stop 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 -- 172.21.15.45:0/1604383986 shutdown_connections 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/1604383986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 0x557f77e7f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/1604383986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c980 0x557f77e82fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/1604383986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c060 0x557f77e84e50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 -- 172.21.15.45:0/1604383986 >> 172.21.15.45:0/1604383986 conn(0x557f77e76980 msgr2=0x557f77e7da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 -- 172.21.15.45:0/1604383986 shutdown_connections 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 -- 172.21.15.45:0/1604383986 wait complete. 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 -- start start 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 Processor -- start 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c060 0x557f77e8f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 0x557f77e8fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 0x557f77eeff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77e90490 con 0x557f77e7c980 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77ef04c0 con 0x557f77e7c060 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77ef0600 con 0x557f77e85750 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f78b1d6b740 10 client.0 _ll_register_callbacks cb 0x7f789800b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.948+0000 7f789ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 0x557f77e8fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f789ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 0x557f77e8fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54456/0 (socket says 172.21.15.45:54456) 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f789ffff640 1 -- 172.21.15.45:0/2100315927 learned_addr learned my addr 172.21.15.45:0/2100315927 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f78ad466640 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 0x557f77eeff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f78acc65640 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c060 0x557f77e8f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f789ffff640 1 -- 172.21.15.45:0/2100315927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 msgr2=0x557f77eeff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f789ffff640 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 0x557f77eeff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f789ffff640 1 -- 172.21.15.45:0/2100315927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c060 msgr2=0x557f77e8f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f789ffff640 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c060 0x557f77e8f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f789ffff640 1 -- 172.21.15.45:0/2100315927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f77de26a0 con 0x557f77e7c980 2024-01-31T05:29:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f78acc65640 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c060 0x557f77e8f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:29:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f789ffff640 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 0x557f77e8fbe0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7898010ca0 tx=0x7f7898005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3543bf45109e541 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f7887fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:29:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f7887fff640 1 -- 172.21.15.45:0/2100315927 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898021040 con 0x557f77e7c980 2024-01-31T05:29:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f7887fff640 1 -- 172.21.15.45:0/2100315927 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7898017070 con 0x557f77e7c980 2024-01-31T05:29:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2100315927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7898015070 con 0x557f77e7c980 2024-01-31T05:29:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f7887fff640 1 -- 172.21.15.45:0/2100315927 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7898004880 con 0x557f77e7c980 2024-01-31T05:29:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f78b1d6b740 10 client.15957 fetch_fsmap learned FSMap version 27 2024-01-31T05:29:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2100315927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557f77da4a50 con 0x557f77e7c980 2024-01-31T05:29:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.949+0000 7f7887fff640 1 -- 172.21.15.45:0/2100315927 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7898015070 con 0x557f77e7c980 2024-01-31T05:29:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.950+0000 7f7887fff640 1 -- 172.21.15.45:0/2100315927 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7898031c30 con 0x557f77e7c980 2024-01-31T05:29:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.950+0000 7f78b1d6b740 10 client.15957 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:29:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.950+0000 7f78b1d6b740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:29:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.950+0000 7f78b1d6b740 20 client.15957 populate_metadata read hostname 'smithi045' 2024-01-31T05:29:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.950+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f77ef1db0 0x557f77ef41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.950+0000 7f78b1d6b740 4 client.15957 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:29:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.950+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2100315927 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557f77ae3600 con 0x557f77ef1db0 2024-01-31T05:29:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.950+0000 7f78acc65640 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f77ef1db0 0x557f77ef41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.950+0000 7f7887fff640 10 client.15957 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.950+0000 7f78acc65640 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f77ef1db0 0x557f77ef41d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c419b451d6d8c5b server_cookie=44bd0548985bbfc in_seq=0 out_seq=0 2024-01-31T05:29:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f7887fff640 1 -- 172.21.15.45:0/2100315927 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557f77ae3600 con 0x557f77ef1db0 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f7887fff640 10 client.15957 handle_command_reply: tid=0 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f78b1d6b740 1 client.15957 shutdown 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2100315927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 msgr2=0x557f77e8fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 0x557f77e8fbe0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7898010ca0 tx=0x7f7898005c60 comp rx=0 tx=0).stop 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2100315927 shutdown_connections 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557f77ef1db0 0x557f77ef41d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f77e7c980 0x557f77e8fbe0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f77e85750 0x557f77eeff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f7887fff640 0 client.15957 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f78b1d6b740 1 --2- 172.21.15.45:0/2100315927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f77e7c060 0x557f77e8f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2100315927 >> 172.21.15.45:0/2100315927 conn(0x557f77e76980 msgr2=0x557f77e75190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2100315927 shutdown_connections 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f78b1d6b740 1 -- 172.21.15.45:0/2100315927 wait complete. 2024-01-31T05:29:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.961+0000 7f78b1d6b740 20 client.15957 trim_cache size 0 max 16384 2024-01-31T05:29:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:29:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:29:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:29:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:29:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.962+0000 7f78aeec9640 1 -- 172.21.15.45:0/2692449866 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f78900a14f0 msgr2=0x7f78900a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.962+0000 7f78aeec9640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f78900a14f0 0x7f78900a39b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009db0 tx=0x7f78a4011810 comp rx=0 tx=0).stop 2024-01-31T05:29:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.962+0000 7f78aeec9640 1 -- 172.21.15.45:0/2692449866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a819ba40 msgr2=0x7f78a819bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.962+0000 7f78aeec9640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a819ba40 0x7f78a819bef0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009ca0 tx=0x7f78a00325e0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.962+0000 7f78aeec9640 1 -- 172.21.15.45:0/2692449866 shutdown_connections 2024-01-31T05:29:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.962+0000 7f78aeec9640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f78900a14f0 0x7f78900a39b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.962+0000 7f78aeec9640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a819ba40 0x7f78a819bef0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.962+0000 7f78aeec9640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78a8106000 0x7f78a81976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.962+0000 7f78aeec9640 1 --2- 172.21.15.45:0/2692449866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78a8100f30 0x7f78a8197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.962+0000 7f78aeec9640 1 -- 172.21.15.45:0/2692449866 >> 172.21.15.45:0/2692449866 conn(0x7f78a80fc7d0 msgr2=0x7f78a80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.962+0000 7f78aeec9640 1 -- 172.21.15.45:0/2692449866 shutdown_connections 2024-01-31T05:29:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:36.962+0000 7f78aeec9640 1 -- 172.21.15.45:0/2692449866 wait complete. 2024-01-31T05:29:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:36 smithi156 ceph-mon[35720]: pgmap v215: 97 pgs: 97 active+clean; 414 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-01-31T05:29:37.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:36 smithi186 ceph-mon[35659]: pgmap v215: 97 pgs: 97 active+clean; 414 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-01-31T05:29:37.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 720846 11.36 MB/sec execute 217 sec latency 32.700 ms 2024-01-31T05:29:38.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 722447 11.34 MB/sec execute 218 sec latency 26.029 ms 2024-01-31T05:29:38.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:38 smithi045 ceph-mon[31585]: pgmap v216: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 290 op/s 2024-01-31T05:29:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:38 smithi156 ceph-mon[35720]: pgmap v216: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 290 op/s 2024-01-31T05:29:39.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:38 smithi186 ceph-mon[35659]: pgmap v216: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 290 op/s 2024-01-31T05:29:39.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 724664 11.35 MB/sec execute 219 sec latency 69.124 ms 2024-01-31T05:29:40.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 726773 11.33 MB/sec execute 220 sec latency 21.107 ms 2024-01-31T05:29:40.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:40 smithi045 ceph-mon[31585]: pgmap v217: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-01-31T05:29:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:40 smithi156 ceph-mon[35720]: pgmap v217: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-01-31T05:29:41.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:40 smithi186 ceph-mon[35659]: pgmap v217: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-01-31T05:29:41.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 728566 11.34 MB/sec execute 221 sec latency 17.035 ms 2024-01-31T05:29:42.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 730444 11.32 MB/sec execute 222 sec latency 85.306 ms 2024-01-31T05:29:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:42 smithi045 ceph-mon[31585]: pgmap v218: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-01-31T05:29:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:42 smithi156 ceph-mon[35720]: pgmap v218: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-01-31T05:29:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:42 smithi186 ceph-mon[35659]: pgmap v218: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-01-31T05:29:43.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 732382 11.32 MB/sec execute 223 sec latency 15.916 ms 2024-01-31T05:29:44.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 734492 11.32 MB/sec execute 224 sec latency 46.609 ms 2024-01-31T05:29:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:44 smithi045 ceph-mon[31585]: pgmap v219: 97 pgs: 97 active+clean; 396 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.7 MiB/s wr, 290 op/s 2024-01-31T05:29:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:44 smithi156 ceph-mon[35720]: pgmap v219: 97 pgs: 97 active+clean; 396 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.7 MiB/s wr, 290 op/s 2024-01-31T05:29:45.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:44 smithi186 ceph-mon[35659]: pgmap v219: 97 pgs: 97 active+clean; 396 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.7 MiB/s wr, 290 op/s 2024-01-31T05:29:45.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 736447 11.33 MB/sec execute 225 sec latency 16.550 ms 2024-01-31T05:29:46.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 738900 11.34 MB/sec execute 226 sec latency 14.302 ms 2024-01-31T05:29:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:46 smithi045 ceph-mon[31585]: pgmap v220: 97 pgs: 97 active+clean; 396 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 194 op/s 2024-01-31T05:29:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:46 smithi156 ceph-mon[35720]: pgmap v220: 97 pgs: 97 active+clean; 396 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 194 op/s 2024-01-31T05:29:47.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:46 smithi186 ceph-mon[35659]: pgmap v220: 97 pgs: 97 active+clean; 396 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 194 op/s 2024-01-31T05:29:47.787 INFO:tasks.workunit.client.0.smithi045.stdout: 1 741027 11.32 MB/sec execute 227 sec latency 26.419 ms 2024-01-31T05:29:48.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 742955 11.32 MB/sec execute 228 sec latency 22.333 ms 2024-01-31T05:29:48.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:48 smithi045 ceph-mon[31585]: pgmap v221: 97 pgs: 97 active+clean; 404 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.4 MiB/s wr, 292 op/s 2024-01-31T05:29:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:48 smithi156 ceph-mon[35720]: pgmap v221: 97 pgs: 97 active+clean; 404 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.4 MiB/s wr, 292 op/s 2024-01-31T05:29:49.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:48 smithi186 ceph-mon[35659]: pgmap v221: 97 pgs: 97 active+clean; 404 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.4 MiB/s wr, 292 op/s 2024-01-31T05:29:49.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 745150 11.33 MB/sec execute 229 sec latency 82.479 ms 2024-01-31T05:29:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.052+0000 7f695c1cd640 1 -- 172.21.15.45:0/4264121809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6954100f10 msgr2=0x7f69541012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.052+0000 7f695c1cd640 1 --2- 172.21.15.45:0/4264121809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6954100f10 0x7f69541012f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6950009930 tx=0x7f695002bc30 comp rx=0 tx=0).stop 2024-01-31T05:29:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.052+0000 7f695c1cd640 1 -- 172.21.15.45:0/4264121809 shutdown_connections 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.052+0000 7f695c1cd640 1 --2- 172.21.15.45:0/4264121809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6954105fe0 0x7f695410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.052+0000 7f695c1cd640 1 --2- 172.21.15.45:0/4264121809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69541018c0 0x7f6954105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.052+0000 7f695c1cd640 1 --2- 172.21.15.45:0/4264121809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6954100f10 0x7f69541012f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.052+0000 7f695c1cd640 1 -- 172.21.15.45:0/4264121809 >> 172.21.15.45:0/4264121809 conn(0x7f69540fc7b0 msgr2=0x7f69540febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.052+0000 7f695c1cd640 1 -- 172.21.15.45:0/4264121809 shutdown_connections 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.052+0000 7f695c1cd640 1 -- 172.21.15.45:0/4264121809 wait complete. 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f695c1cd640 1 Processor -- start 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f695c1cd640 1 -- start start 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f695c1cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6954100f10 0x7f69541971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f695c1cd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6954105fe0 0x7f6954197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f695c1cd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f695419ba80 0x7f695419bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f695c1cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695419c470 con 0x7f6954100f10 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f695c1cd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695419c5e0 con 0x7f6954105fe0 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f695c1cd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695419c750 con 0x7f695419ba80 2024-01-31T05:29:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f6959f69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6954100f10 0x7f69541971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f6959f69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6954100f10 0x7f69541971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60768/0 (socket says 172.21.15.45:60768) 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f6959f69640 1 -- 172.21.15.45:0/121664389 learned_addr learned my addr 172.21.15.45:0/121664389 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f6959768640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6954105fe0 0x7f6954197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f695a76a640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f695419ba80 0x7f695419bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f6959f69640 1 -- 172.21.15.45:0/121664389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f695419ba80 msgr2=0x7f695419bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f6959f69640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f695419ba80 0x7f695419bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f6959f69640 1 -- 172.21.15.45:0/121664389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6954105fe0 msgr2=0x7f6954197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f6959f69640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6954105fe0 0x7f6954197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f6959f69640 1 -- 172.21.15.45:0/121664389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6950009560 con 0x7f6954100f10 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f695a76a640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f695419ba80 0x7f695419bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.053+0000 7f6959f69640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6954100f10 0x7f69541971c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6950005940 tx=0x7f6950004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.054+0000 7f694affd640 1 -- 172.21.15.45:0/121664389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950004250 con 0x7f6954100f10 2024-01-31T05:29:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.054+0000 7f694affd640 1 -- 172.21.15.45:0/121664389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69500043b0 con 0x7f6954100f10 2024-01-31T05:29:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.054+0000 7f695c1cd640 1 -- 172.21.15.45:0/121664389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69541a3d30 con 0x7f6954100f10 2024-01-31T05:29:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.054+0000 7f694affd640 1 -- 172.21.15.45:0/121664389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950003d00 con 0x7f6954100f10 2024-01-31T05:29:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.054+0000 7f695c1cd640 1 -- 172.21.15.45:0/121664389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69541a41b0 con 0x7f6954100f10 2024-01-31T05:29:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.056+0000 7f694affd640 1 -- 172.21.15.45:0/121664389 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f6950005000 con 0x7f6954100f10 2024-01-31T05:29:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.056+0000 7f695ef94740 1 Processor -- start 2024-01-31T05:29:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f695ef94740 1 -- start start 2024-01-31T05:29:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f695ef94740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3060 0x55f8647a3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f695ef94740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3980 0x55f8647ac210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f695ef94740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f695ef94740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f864712040 con 0x55f8647a3980 2024-01-31T05:29:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f695ef94740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f86470f730 con 0x55f8647a3060 2024-01-31T05:29:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f695ef94740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f86470b020 con 0x55f8647ac750 2024-01-31T05:29:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f694affd640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f69400a14f0 0x7f69400a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f6959f69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3980 0x55f8647ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f695a76a640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f69400a14f0 0x7f69400a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f6959768640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f6959768640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:58452/0 (socket says 172.21.15.45:58452) 2024-01-31T05:29:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f6959f69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3980 0x55f8647ac210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60772/0 (socket says 172.21.15.45:60772) 2024-01-31T05:29:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f6959768640 1 -- 172.21.15.45:0/1308994105 learned_addr learned my addr 172.21.15.45:0/1308994105 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f695a76a640 1 --2- 172.21.15.45:0/1308994105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3060 0x55f8647a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f694affd640 1 -- 172.21.15.45:0/121664389 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f69500ef6b0 con 0x7f6954100f10 2024-01-31T05:29:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f6959768640 1 -- 172.21.15.45:0/1308994105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3060 msgr2=0x55f8647a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f6959768640 1 --2- 172.21.15.45:0/1308994105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3060 0x55f8647a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f6959768640 1 -- 172.21.15.45:0/1308994105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3980 msgr2=0x55f8647ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.057+0000 7f6959768640 1 --2- 172.21.15.45:0/1308994105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3980 0x55f8647ac210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.058+0000 7f6959768640 1 -- 172.21.15.45:0/1308994105 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f864702e40 con 0x55f8647ac750 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.058+0000 7f695a76a640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f69400a14f0 0x7f69400a39b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f694c005fd0 tx=0x7f694c009db0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.058+0000 7f6959768640 1 --2- 172.21.15.45:0/1308994105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647aeb40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f694400bc70 tx=0x7f69440107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e3120defe065024 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.058+0000 7f693ffff640 1 -- 172.21.15.45:0/1308994105 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6944005220 con 0x55f8647ac750 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.058+0000 7f693ffff640 1 -- 172.21.15.45:0/1308994105 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6944015070 con 0x55f8647ac750 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.058+0000 7f693ffff640 1 -- 172.21.15.45:0/1308994105 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6944011440 con 0x55f8647ac750 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.059+0000 7f695ef94740 1 -- 172.21.15.45:0/1308994105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 msgr2=0x55f8647aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.059+0000 7f695ef94740 1 --2- 172.21.15.45:0/1308994105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647aeb40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f694400bc70 tx=0x7f69440107a0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.059+0000 7f695ef94740 1 -- 172.21.15.45:0/1308994105 shutdown_connections 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.059+0000 7f695ef94740 1 --2- 172.21.15.45:0/1308994105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3980 0x55f8647ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.059+0000 7f695ef94740 1 --2- 172.21.15.45:0/1308994105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647aeb40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.059+0000 7f695ef94740 1 --2- 172.21.15.45:0/1308994105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3060 0x55f8647a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.059+0000 7f695ef94740 1 -- 172.21.15.45:0/1308994105 >> 172.21.15.45:0/1308994105 conn(0x55f86479d980 msgr2=0x55f86479dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.059+0000 7f695ef94740 1 -- 172.21.15.45:0/1308994105 shutdown_connections 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.059+0000 7f695ef94740 1 -- 172.21.15.45:0/1308994105 wait complete. 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.059+0000 7f695ef94740 1 -- start start 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.059+0000 7f695ef94740 1 Processor -- start 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f695ef94740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 0x55f8647a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f695ef94740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3980 0x55f8647a9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f695ef94740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647aa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f695ef94740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8646c41b0 con 0x55f8647a3060 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f695ef94740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8646c3e50 con 0x55f8647a3980 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f695ef94740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f864689090 con 0x55f8647ac750 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f695ef94740 10 client.0 _ll_register_callbacks cb 0x55f8646aea90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f6959f69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 0x55f8647a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f6959768640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3980 0x55f8647a9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f6959f69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 0x55f8647a96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60778/0 (socket says 172.21.15.45:60778) 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f6959f69640 1 -- 172.21.15.45:0/2562180330 learned_addr learned my addr 172.21.15.45:0/2562180330 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f695a76a640 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f6959f69640 1 -- 172.21.15.45:0/2562180330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 msgr2=0x55f8647aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f6959f69640 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f6959f69640 1 -- 172.21.15.45:0/2562180330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3980 msgr2=0x55f8647a9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f6959f69640 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3980 0x55f8647a9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f6959f69640 1 -- 172.21.15.45:0/2562180330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694400b8a0 con 0x55f8647a3060 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f6959768640 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3980 0x55f8647a9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f6959f69640 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 0x55f8647a96f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f69500ecf10 tx=0x7f695008e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dc42adc9cdc97d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f693d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f693d7fa640 1 -- 172.21.15.45:0/2562180330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69500054b0 con 0x55f8647a3060 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f693d7fa640 1 -- 172.21.15.45:0/2562180330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f69500035f0 con 0x55f8647a3060 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.060+0000 7f695ef94740 1 -- 172.21.15.45:0/2562180330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f864704010 con 0x55f8647a3060 2024-01-31T05:29:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.061+0000 7f693d7fa640 1 -- 172.21.15.45:0/2562180330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6950003e60 con 0x55f8647a3060 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.061+0000 7f695ef94740 10 client.15969 fetch_fsmap learned FSMap version 27 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.061+0000 7f695ef94740 1 -- 172.21.15.45:0/2562180330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f8647096a0 con 0x55f8647a3060 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.061+0000 7f693d7fa640 1 -- 172.21.15.45:0/2562180330 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f864704010 con 0x55f8647a3060 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.061+0000 7f693d7fa640 1 -- 172.21.15.45:0/2562180330 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f695003c070 con 0x55f8647a3060 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.061+0000 7f695ef94740 10 client.15969 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.061+0000 7f695ef94740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.061+0000 7f695ef94740 20 client.15969 populate_metadata read hostname 'smithi045' 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.061+0000 7f695ef94740 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f8647bc3c0 0x55f8647be7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.061+0000 7f695ef94740 4 client.15969 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.061+0000 7f695ef94740 1 -- 172.21.15.45:0/2562180330 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f864703d30 con 0x55f8647bc3c0 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.061+0000 7f6959768640 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f8647bc3c0 0x55f8647be7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f6959768640 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f8647bc3c0 0x55f8647be7b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96a227d205425ec2 server_cookie=e43eba7693904603 in_seq=0 out_seq=0 2024-01-31T05:29:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f693d7fa640 10 client.15969 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f693d7fa640 1 -- 172.21.15.45:0/2562180330 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f864703d30 con 0x55f8647bc3c0 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f693d7fa640 10 client.15969 handle_command_reply: tid=0 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f695ef94740 1 client.15969 shutdown 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f695ef94740 1 -- 172.21.15.45:0/2562180330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 msgr2=0x55f8647a96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f695ef94740 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 0x55f8647a96f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f69500ecf10 tx=0x7f695008e5c0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f695ef94740 1 -- 172.21.15.45:0/2562180330 shutdown_connections 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f695ef94740 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f8647bc3c0 0x55f8647be7b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f695ef94740 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f695ef94740 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3980 0x55f8647a9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f693d7fa640 0 client.15969 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f695ef94740 1 --2- 172.21.15.45:0/2562180330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 0x55f8647a96f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f695ef94740 1 -- 172.21.15.45:0/2562180330 >> 172.21.15.45:0/2562180330 conn(0x55f86479d980 msgr2=0x55f86479f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f695ef94740 1 -- 172.21.15.45:0/2562180330 shutdown_connections 2024-01-31T05:29:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.062+0000 7f695ef94740 1 -- 172.21.15.45:0/2562180330 wait complete. 2024-01-31T05:29:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.063+0000 7f695ef94740 20 client.15969 trim_cache size 0 max 16384 2024-01-31T05:29:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f695ef94740 1 Processor -- start 2024-01-31T05:29:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f695ef94740 1 -- start start 2024-01-31T05:29:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f695ef94740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 0x55f8647abe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f695ef94740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3980 0x55f8647a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f695ef94740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647a6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f695ef94740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8647990b0 con 0x55f8647a3060 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f695ef94740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8647aeb40 con 0x55f8647a3980 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f695ef94740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8647a4550 con 0x55f8647ac750 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f6959f69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 0x55f8647abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f6959f69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 0x55f8647abe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60786/0 (socket says 172.21.15.45:60786) 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f6959f69640 1 -- 172.21.15.45:0/1035493657 learned_addr learned my addr 172.21.15.45:0/1035493657 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f695a76a640 1 --2- 172.21.15.45:0/1035493657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f6959768640 1 --2- 172.21.15.45:0/1035493657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3980 0x55f8647a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f6959f69640 1 -- 172.21.15.45:0/1035493657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 msgr2=0x55f8647a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f6959f69640 1 --2- 172.21.15.45:0/1035493657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f6959f69640 1 -- 172.21.15.45:0/1035493657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3980 msgr2=0x55f8647a9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f6959f69640 1 --2- 172.21.15.45:0/1035493657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3980 0x55f8647a9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f6959f69640 1 -- 172.21.15.45:0/1035493657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8646a6030 con 0x55f8647a3060 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f6959f69640 1 --2- 172.21.15.45:0/1035493657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 0x55f8647abe20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6950006010 tx=0x7f69500bab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bede7d370a01da server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.076+0000 7f693f7fe640 1 -- 172.21.15.45:0/1035493657 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950004520 con 0x55f8647a3060 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.077+0000 7f693f7fe640 1 -- 172.21.15.45:0/1035493657 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69500b88c0 con 0x55f8647a3060 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.077+0000 7f693f7fe640 1 -- 172.21.15.45:0/1035493657 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69500035f0 con 0x55f8647a3060 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.077+0000 7f695ef94740 1 -- 172.21.15.45:0/1035493657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 msgr2=0x55f8647abe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.077+0000 7f695ef94740 1 --2- 172.21.15.45:0/1035493657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 0x55f8647abe20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6950006010 tx=0x7f69500bab10 comp rx=0 tx=0).stop 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.077+0000 7f695ef94740 1 -- 172.21.15.45:0/1035493657 shutdown_connections 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.077+0000 7f695ef94740 1 --2- 172.21.15.45:0/1035493657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647ac750 0x55f8647a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.077+0000 7f695ef94740 1 --2- 172.21.15.45:0/1035493657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3980 0x55f8647a9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.077+0000 7f695ef94740 1 --2- 172.21.15.45:0/1035493657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647a3060 0x55f8647abe20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.077+0000 7f695ef94740 1 -- 172.21.15.45:0/1035493657 >> 172.21.15.45:0/1035493657 conn(0x55f86479d980 msgr2=0x55f8647a49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.077+0000 7f695ef94740 1 -- 172.21.15.45:0/1035493657 shutdown_connections 2024-01-31T05:29:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.077+0000 7f695ef94740 1 -- 172.21.15.45:0/1035493657 wait complete. 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.077+0000 7f695ef94740 1 -- start start 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695ef94740 1 Processor -- start 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695ef94740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3060 0x55f8647b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695ef94740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647a3980 0x55f8647b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695ef94740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647ac750 0x55f8647b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695ef94740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8647b2980 con 0x55f8647ac750 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695ef94740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8647b2af0 con 0x55f8647a3060 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695ef94740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f864812830 con 0x55f8647a3980 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695ef94740 10 client.0 _ll_register_callbacks cb 0x7f694400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695a76a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647ac750 0x55f8647b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695a76a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647ac750 0x55f8647b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60794/0 (socket says 172.21.15.45:60794) 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695a76a640 1 -- 172.21.15.45:0/1157435714 learned_addr learned my addr 172.21.15.45:0/1157435714 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f6959f69640 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3060 0x55f8647b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f6959768640 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647a3980 0x55f8647b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695a76a640 1 -- 172.21.15.45:0/1157435714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647a3980 msgr2=0x55f8647b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695a76a640 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647a3980 0x55f8647b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695a76a640 1 -- 172.21.15.45:0/1157435714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3060 msgr2=0x55f8647b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695a76a640 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3060 0x55f8647b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f695a76a640 1 -- 172.21.15.45:0/1157435714 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8647096a0 con 0x55f8647ac750 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.078+0000 7f6959f69640 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3060 0x55f8647b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f695a76a640 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647ac750 0x55f8647b22e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f694c010eb0 tx=0x7f694c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e687c4f21d2b974 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f693d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f693d7fa640 1 -- 172.21.15.45:0/1157435714 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c011ae0 con 0x55f8647ac750 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f693d7fa640 1 -- 172.21.15.45:0/1157435714 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f694c011c40 con 0x55f8647ac750 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f695ef94740 1 -- 172.21.15.45:0/1157435714 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f86440a600 con 0x55f8647ac750 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f693d7fa640 1 -- 172.21.15.45:0/1157435714 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f694c011e00 con 0x55f8647ac750 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f695ef94740 10 client.15975 fetch_fsmap learned FSMap version 27 2024-01-31T05:29:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f695ef94740 1 -- 172.21.15.45:0/1157435714 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f8646cba50 con 0x55f8647ac750 2024-01-31T05:29:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f693d7fa640 1 -- 172.21.15.45:0/1157435714 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f86440a600 con 0x55f8647ac750 2024-01-31T05:29:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f693d7fa640 1 -- 172.21.15.45:0/1157435714 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f694c029070 con 0x55f8647ac750 2024-01-31T05:29:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f695ef94740 10 client.15975 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:29:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f695ef94740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:29:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f695ef94740 20 client.15975 populate_metadata read hostname 'smithi045' 2024-01-31T05:29:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f695ef94740 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f8648140e0 0x55f864816500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:29:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f695ef94740 4 client.15975 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:29:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f695ef94740 1 -- 172.21.15.45:0/1157435714 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f86445dc50 con 0x55f8648140e0 2024-01-31T05:29:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.079+0000 7f6959f69640 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f8648140e0 0x55f864816500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:29:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.080+0000 7f693d7fa640 10 client.15975 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.080+0000 7f6959f69640 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f8648140e0 0x55f864816500 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaddd4a3e7f21fdd server_cookie=5991f4dd947f8300 in_seq=0 out_seq=0 2024-01-31T05:29:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f693d7fa640 1 -- 172.21.15.45:0/1157435714 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f86445dc50 con 0x55f8648140e0 2024-01-31T05:29:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f693d7fa640 10 client.15975 handle_command_reply: tid=0 2024-01-31T05:29:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f695ef94740 1 client.15975 shutdown 2024-01-31T05:29:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f695ef94740 1 -- 172.21.15.45:0/1157435714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647ac750 msgr2=0x55f8647b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f695ef94740 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647ac750 0x55f8647b22e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f694c010eb0 tx=0x7f694c01c000 comp rx=0 tx=0).stop 2024-01-31T05:29:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f695ef94740 1 -- 172.21.15.45:0/1157435714 shutdown_connections 2024-01-31T05:29:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f695ef94740 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f8648140e0 0x55f864816500 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f695ef94740 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8647ac750 0x55f8647b22e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f695ef94740 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f8647a3980 0x55f8647b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f693d7fa640 0 client.15975 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:29:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f695ef94740 1 --2- 172.21.15.45:0/1157435714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f8647a3060 0x55f8647b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f695ef94740 1 -- 172.21.15.45:0/1157435714 >> 172.21.15.45:0/1157435714 conn(0x55f86479d980 msgr2=0x55f86479edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f695ef94740 1 -- 172.21.15.45:0/1157435714 shutdown_connections 2024-01-31T05:29:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f695ef94740 1 -- 172.21.15.45:0/1157435714 wait complete. 2024-01-31T05:29:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.090+0000 7f695ef94740 20 client.15975 trim_cache size 0 max 16384 2024-01-31T05:29:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:29:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:29:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:29:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:29:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.091+0000 7f695c1cd640 1 -- 172.21.15.45:0/121664389 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f69400a14f0 msgr2=0x7f69400a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.091+0000 7f695c1cd640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f69400a14f0 0x7f69400a39b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f694c005fd0 tx=0x7f694c009db0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.091+0000 7f695c1cd640 1 -- 172.21.15.45:0/121664389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6954100f10 msgr2=0x7f69541971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:29:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.091+0000 7f695c1cd640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6954100f10 0x7f69541971c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6950005940 tx=0x7f6950004bd0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.091+0000 7f695c1cd640 1 -- 172.21.15.45:0/121664389 shutdown_connections 2024-01-31T05:29:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.091+0000 7f695c1cd640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f69400a14f0 0x7f69400a39b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.091+0000 7f695c1cd640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f695419ba80 0x7f695419bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.091+0000 7f695c1cd640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6954105fe0 0x7f6954197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.091+0000 7f695c1cd640 1 --2- 172.21.15.45:0/121664389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6954100f10 0x7f69541971c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:29:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.091+0000 7f695c1cd640 1 -- 172.21.15.45:0/121664389 >> 172.21.15.45:0/121664389 conn(0x7f69540fc7b0 msgr2=0x7f69540feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:29:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.091+0000 7f695c1cd640 1 -- 172.21.15.45:0/121664389 shutdown_connections 2024-01-31T05:29:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:29:50.091+0000 7f695c1cd640 1 -- 172.21.15.45:0/121664389 wait complete. 2024-01-31T05:29:50.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 747111 11.33 MB/sec execute 230 sec latency 15.558 ms 2024-01-31T05:29:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:50 smithi045 ceph-mon[31585]: pgmap v222: 97 pgs: 97 active+clean; 404 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 196 op/s 2024-01-31T05:29:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:50 smithi156 ceph-mon[35720]: pgmap v222: 97 pgs: 97 active+clean; 404 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 196 op/s 2024-01-31T05:29:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:50 smithi186 ceph-mon[35659]: pgmap v222: 97 pgs: 97 active+clean; 404 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 196 op/s 2024-01-31T05:29:51.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 749519 11.34 MB/sec execute 231 sec latency 16.644 ms 2024-01-31T05:29:52.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 751626 11.32 MB/sec execute 232 sec latency 16.836 ms 2024-01-31T05:29:52.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:52 smithi045 ceph-mon[31585]: pgmap v223: 97 pgs: 97 active+clean; 404 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 196 op/s 2024-01-31T05:29:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:52 smithi156 ceph-mon[35720]: pgmap v223: 97 pgs: 97 active+clean; 404 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 196 op/s 2024-01-31T05:29:53.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:52 smithi186 ceph-mon[35659]: pgmap v223: 97 pgs: 97 active+clean; 404 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 196 op/s 2024-01-31T05:29:53.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 753245 11.33 MB/sec execute 233 sec latency 197.103 ms 2024-01-31T05:29:53.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:53 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:29:53.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:53 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:29:53.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:53 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:29:53.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:53 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:29:53.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:53 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:29:53.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:53 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:29:53.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:53 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:29:53.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:53 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:29:54.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:53 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:29:54.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:53 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:29:54.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:53 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:29:54.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:53 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:29:54.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 755217 11.31 MB/sec execute 234 sec latency 61.718 ms 2024-01-31T05:29:54.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:54 smithi045 ceph-mon[31585]: pgmap v224: 97 pgs: 97 active+clean; 413 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.8 MiB/s wr, 299 op/s 2024-01-31T05:29:54.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:54 smithi156 ceph-mon[35720]: pgmap v224: 97 pgs: 97 active+clean; 413 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.8 MiB/s wr, 299 op/s 2024-01-31T05:29:55.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:54 smithi186 ceph-mon[35659]: pgmap v224: 97 pgs: 97 active+clean; 413 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.8 MiB/s wr, 299 op/s 2024-01-31T05:29:55.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 757144 11.31 MB/sec execute 235 sec latency 22.538 ms 2024-01-31T05:29:55.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:55 smithi045 ceph-mon[31585]: pgmap v225: 97 pgs: 97 active+clean; 413 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 200 op/s 2024-01-31T05:29:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:55 smithi156 ceph-mon[35720]: pgmap v225: 97 pgs: 97 active+clean; 413 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 200 op/s 2024-01-31T05:29:56.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:55 smithi186 ceph-mon[35659]: pgmap v225: 97 pgs: 97 active+clean; 413 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 200 op/s 2024-01-31T05:29:56.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 759693 11.32 MB/sec execute 236 sec latency 16.486 ms 2024-01-31T05:29:57.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 761395 11.32 MB/sec execute 237 sec latency 56.416 ms 2024-01-31T05:29:58.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 763704 11.33 MB/sec execute 238 sec latency 68.733 ms 2024-01-31T05:29:58.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:29:58 smithi045 ceph-mon[31585]: pgmap v226: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 295 op/s 2024-01-31T05:29:58.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:29:58 smithi156 ceph-mon[35720]: pgmap v226: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 295 op/s 2024-01-31T05:29:59.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:29:58 smithi186 ceph-mon[35659]: pgmap v226: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 295 op/s 2024-01-31T05:29:59.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 765185 11.30 MB/sec execute 239 sec latency 65.699 ms 2024-01-31T05:30:00.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 767464 11.31 MB/sec execute 240 sec latency 16.480 ms 2024-01-31T05:30:00.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:00 smithi045 ceph-mon[31585]: pgmap v227: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 198 op/s 2024-01-31T05:30:00.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:00 smithi045 ceph-mon[31585]: overall HEALTH_OK 2024-01-31T05:30:00.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:00 smithi156 ceph-mon[35720]: pgmap v227: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 198 op/s 2024-01-31T05:30:00.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:00 smithi156 ceph-mon[35720]: overall HEALTH_OK 2024-01-31T05:30:01.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:00 smithi186 ceph-mon[35659]: pgmap v227: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 198 op/s 2024-01-31T05:30:01.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:00 smithi186 ceph-mon[35659]: overall HEALTH_OK 2024-01-31T05:30:01.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 768721 11.29 MB/sec execute 241 sec latency 53.122 ms 2024-01-31T05:30:02.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 770910 11.30 MB/sec execute 242 sec latency 15.689 ms 2024-01-31T05:30:02.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:02 smithi045 ceph-mon[31585]: pgmap v228: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 198 op/s 2024-01-31T05:30:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:02 smithi156 ceph-mon[35720]: pgmap v228: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 198 op/s 2024-01-31T05:30:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:02 smithi186 ceph-mon[35659]: pgmap v228: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 198 op/s 2024-01-31T05:30:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.181+0000 7f821c448640 1 -- 172.21.15.45:0/3102905807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8214106e00 msgr2=0x7f82141071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.181+0000 7f821c448640 1 --2- 172.21.15.45:0/3102905807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8214106e00 0x7f82141071e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8204009900 tx=0x7f820402bc30 comp rx=0 tx=0).stop 2024-01-31T05:30:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.181+0000 7f821c448640 1 -- 172.21.15.45:0/3102905807 shutdown_connections 2024-01-31T05:30:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.181+0000 7f821c448640 1 --2- 172.21.15.45:0/3102905807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214068d10 0x7f821410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.181+0000 7f821c448640 1 --2- 172.21.15.45:0/3102905807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8214068370 0x7f82140687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.181+0000 7f821c448640 1 --2- 172.21.15.45:0/3102905807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8214106e00 0x7f82141071e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.181+0000 7f821c448640 1 -- 172.21.15.45:0/3102905807 >> 172.21.15.45:0/3102905807 conn(0x7f8214076a40 msgr2=0x7f8214076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.181+0000 7f821c448640 1 -- 172.21.15.45:0/3102905807 shutdown_connections 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.181+0000 7f821c448640 1 -- 172.21.15.45:0/3102905807 wait complete. 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f821c448640 1 Processor -- start 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f821c448640 1 -- start start 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f821c448640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8214068370 0x7f8214197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f821c448640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8214068d10 0x7f82141976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f821c448640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214106e00 0x7f821419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f821c448640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821419c050 con 0x7f8214068370 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f821c448640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821419c1c0 con 0x7f8214068d10 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f821c448640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821419c380 con 0x7f8214106e00 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f821a1e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8214068370 0x7f8214197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f821a1e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8214068370 0x7f8214197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40794/0 (socket says 172.21.15.45:40794) 2024-01-31T05:30:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f821a1e4640 1 -- 172.21.15.45:0/4282580633 learned_addr learned my addr 172.21.15.45:0/4282580633 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f82199e3640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8214068d10 0x7f82141976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.182+0000 7f821a9e5640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214106e00 0x7f821419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.183+0000 7f821a1e4640 1 -- 172.21.15.45:0/4282580633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214106e00 msgr2=0x7f821419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.183+0000 7f821a1e4640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214106e00 0x7f821419ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.183+0000 7f821a1e4640 1 -- 172.21.15.45:0/4282580633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8214068d10 msgr2=0x7f82141976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.183+0000 7f821a1e4640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8214068d10 0x7f82141976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.183+0000 7f821a1e4640 1 -- 172.21.15.45:0/4282580633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8204009560 con 0x7f8214068370 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.183+0000 7f821a1e4640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8214068370 0x7f8214197190 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8204006010 tx=0x7f8204005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.183+0000 7f82137fe640 1 -- 172.21.15.45:0/4282580633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204004250 con 0x7f8214068370 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.183+0000 7f82137fe640 1 -- 172.21.15.45:0/4282580633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82040043b0 con 0x7f8214068370 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.183+0000 7f821c448640 1 -- 172.21.15.45:0/4282580633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821419c600 con 0x7f8214068370 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.183+0000 7f82137fe640 1 -- 172.21.15.45:0/4282580633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204003c00 con 0x7f8214068370 2024-01-31T05:30:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.183+0000 7f821c448640 1 -- 172.21.15.45:0/4282580633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141a3ee0 con 0x7f8214068370 2024-01-31T05:30:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.186+0000 7f82137fe640 1 -- 172.21.15.45:0/4282580633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f8204005000 con 0x7f8214068370 2024-01-31T05:30:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.186+0000 7f821f207740 1 Processor -- start 2024-01-31T05:30:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.186+0000 7f82137fe640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f81fc0a14f0 0x7f81fc0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821a9e5640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f81fc0a14f0 0x7f81fc0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821f207740 1 -- start start 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f82137fe640 1 -- 172.21.15.45:0/4282580633 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f82040ef670 con 0x7f8214068370 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821f207740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa185440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821f207740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa18e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821f207740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa190b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821f207740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555efa0f4040 con 0x555efa185980 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821f207740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555efa0f1730 con 0x555efa18e750 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821f207740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555efa0ed020 con 0x555efa185060 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f82199e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa18e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821a1e4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa185440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f82199e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa18e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40808/0 (socket says 172.21.15.45:40808) 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f82199e3640 1 -- 172.21.15.45:0/1899024080 learned_addr learned my addr 172.21.15.45:0/1899024080 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821a9e5640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f81fc0a14f0 0x7f81fc0a39b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f820c005fd0 tx=0x7f820c009500 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821a9e5640 1 --2- 172.21.15.45:0/1899024080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa190b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821a1e4640 1 -- 172.21.15.45:0/1899024080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 msgr2=0x555efa190b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821a1e4640 1 --2- 172.21.15.45:0/1899024080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa190b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821a1e4640 1 -- 172.21.15.45:0/1899024080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 msgr2=0x555efa18e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821a1e4640 1 --2- 172.21.15.45:0/1899024080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa18e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821a1e4640 1 -- 172.21.15.45:0/1899024080 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555efa0e4e40 con 0x555efa185060 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f82199e3640 1 --2- 172.21.15.45:0/1899024080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa18e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.187+0000 7f821a9e5640 1 --2- 172.21.15.45:0/1899024080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa190b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.188+0000 7f821a1e4640 1 --2- 172.21.15.45:0/1899024080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa185440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82040eced0 tx=0x7f82040bbfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b0713738f016fea server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.188+0000 7f8210bfe640 1 -- 172.21.15.45:0/1899024080 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040c06d0 con 0x555efa185060 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.188+0000 7f8210bfe640 1 -- 172.21.15.45:0/1899024080 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82040c4910 con 0x555efa185060 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.188+0000 7f8210bfe640 1 -- 172.21.15.45:0/1899024080 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040bba70 con 0x555efa185060 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 -- 172.21.15.45:0/1899024080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 msgr2=0x555efa185440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 --2- 172.21.15.45:0/1899024080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa185440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82040eced0 tx=0x7f82040bbfa0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 -- 172.21.15.45:0/1899024080 shutdown_connections 2024-01-31T05:30:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 --2- 172.21.15.45:0/1899024080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa18e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 --2- 172.21.15.45:0/1899024080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa190b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 --2- 172.21.15.45:0/1899024080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa185440 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 -- 172.21.15.45:0/1899024080 >> 172.21.15.45:0/1899024080 conn(0x555efa17f980 msgr2=0x555efa17fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 -- 172.21.15.45:0/1899024080 shutdown_connections 2024-01-31T05:30:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 -- 172.21.15.45:0/1899024080 wait complete. 2024-01-31T05:30:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 -- start start 2024-01-31T05:30:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 Processor -- start 2024-01-31T05:30:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa18b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa18bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa18c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555efa0a61b0 con 0x555efa185980 2024-01-31T05:30:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555efa0a5e50 con 0x555efa18e750 2024-01-31T05:30:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555efa06b090 con 0x555efa185060 2024-01-31T05:30:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821f207740 10 client.0 _ll_register_callbacks cb 0x555efa090a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:30:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f82199e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa18bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821a1e4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa18b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f821a9e5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa18c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.189+0000 7f82199e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa18bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40822/0 (socket says 172.21.15.45:40822) 2024-01-31T05:30:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f82199e3640 1 -- 172.21.15.45:0/624691458 learned_addr learned my addr 172.21.15.45:0/624691458 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f82199e3640 1 -- 172.21.15.45:0/624691458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 msgr2=0x555efa18b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f82199e3640 1 --2- 172.21.15.45:0/624691458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa18b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f82199e3640 1 -- 172.21.15.45:0/624691458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 msgr2=0x555efa18c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f82199e3640 1 --2- 172.21.15.45:0/624691458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa18c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f82199e3640 1 -- 172.21.15.45:0/624691458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141a3ee0 con 0x555efa185980 2024-01-31T05:30:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f82199e3640 1 --2- 172.21.15.45:0/624691458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa18bc30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f820800bad0 tx=0x7f820800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6c01b3861a27e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f81f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:30:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f81f9ffb640 1 -- 172.21.15.45:0/624691458 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208016070 con 0x555efa185980 2024-01-31T05:30:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f81f9ffb640 1 -- 172.21.15.45:0/624691458 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8208003ce0 con 0x555efa185980 2024-01-31T05:30:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f821f207740 1 -- 172.21.15.45:0/624691458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555efa0e6010 con 0x555efa185980 2024-01-31T05:30:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f81f9ffb640 1 -- 172.21.15.45:0/624691458 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8208003ea0 con 0x555efa185980 2024-01-31T05:30:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f821f207740 10 client.15987 fetch_fsmap learned FSMap version 27 2024-01-31T05:30:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f821f207740 1 -- 172.21.15.45:0/624691458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555efa0eb6a0 con 0x555efa185980 2024-01-31T05:30:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.190+0000 7f81f9ffb640 1 -- 172.21.15.45:0/624691458 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8208027080 con 0x555efa185980 2024-01-31T05:30:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.191+0000 7f81f9ffb640 1 -- 172.21.15.45:0/624691458 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8208021030 con 0x555efa185980 2024-01-31T05:30:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.191+0000 7f821f207740 10 client.15987 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:30:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.191+0000 7f821f207740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:30:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.191+0000 7f821f207740 20 client.15987 populate_metadata read hostname 'smithi045' 2024-01-31T05:30:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.191+0000 7f821f207740 1 --2- 172.21.15.45:0/624691458 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555efa19e450 0x555efa1a0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.191+0000 7f821f207740 4 client.15987 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.191+0000 7f821f207740 1 -- 172.21.15.45:0/624691458 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555efa0e5d30 con 0x555efa19e450 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.191+0000 7f821a1e4640 1 --2- 172.21.15.45:0/624691458 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555efa19e450 0x555efa1a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.191+0000 7f81f9ffb640 10 client.15987 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.191+0000 7f821a1e4640 1 --2- 172.21.15.45:0/624691458 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555efa19e450 0x555efa1a0870 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d84cf7af6901d597 server_cookie=9cb13ab6fc490a2b in_seq=0 out_seq=0 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f81f9ffb640 1 -- 172.21.15.45:0/624691458 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555efa0e5d30 con 0x555efa19e450 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f81f9ffb640 10 client.15987 handle_command_reply: tid=0 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f821f207740 1 client.15987 shutdown 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f821f207740 1 -- 172.21.15.45:0/624691458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 msgr2=0x555efa18bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f821f207740 1 --2- 172.21.15.45:0/624691458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa18bc30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f820800bad0 tx=0x7f820800bed0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f821f207740 1 -- 172.21.15.45:0/624691458 shutdown_connections 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f821f207740 1 --2- 172.21.15.45:0/624691458 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555efa19e450 0x555efa1a0870 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f821f207740 1 --2- 172.21.15.45:0/624691458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa18bc30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f81f9ffb640 0 client.15987 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f821f207740 1 --2- 172.21.15.45:0/624691458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa18c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f821f207740 1 --2- 172.21.15.45:0/624691458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa18b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f821f207740 1 -- 172.21.15.45:0/624691458 >> 172.21.15.45:0/624691458 conn(0x555efa17f980 msgr2=0x555efa1816d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f821f207740 1 -- 172.21.15.45:0/624691458 shutdown_connections 2024-01-31T05:30:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f821f207740 1 -- 172.21.15.45:0/624691458 wait complete. 2024-01-31T05:30:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.192+0000 7f821f207740 20 client.15987 trim_cache size 0 max 16384 2024-01-31T05:30:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.206+0000 7f821f207740 1 Processor -- start 2024-01-31T05:30:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.206+0000 7f821f207740 1 -- start start 2024-01-31T05:30:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f821f207740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa18bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f821f207740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa189fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f821f207740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa18a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f821f207740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555efa184540 con 0x555efa185980 2024-01-31T05:30:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f821f207740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555efa180980 con 0x555efa18e750 2024-01-31T05:30:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f821f207740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555efa18aa40 con 0x555efa185060 2024-01-31T05:30:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f82199e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa189fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f82199e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa189fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40836/0 (socket says 172.21.15.45:40836) 2024-01-31T05:30:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f821a9e5640 1 --2- 172.21.15.45:0/2277396175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa18a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f82199e3640 1 -- 172.21.15.45:0/2277396175 learned_addr learned my addr 172.21.15.45:0/2277396175 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f821a1e4640 1 --2- 172.21.15.45:0/2277396175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa18bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f82199e3640 1 -- 172.21.15.45:0/2277396175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 msgr2=0x555efa18bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f82199e3640 1 --2- 172.21.15.45:0/2277396175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa18bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f82199e3640 1 -- 172.21.15.45:0/2277396175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 msgr2=0x555efa18a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f82199e3640 1 --2- 172.21.15.45:0/2277396175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa18a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f82199e3640 1 -- 172.21.15.45:0/2277396175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555efa088030 con 0x555efa185980 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f821a9e5640 1 --2- 172.21.15.45:0/2277396175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa18a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f82199e3640 1 --2- 172.21.15.45:0/2277396175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa189fc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8208000c00 tx=0x7f8208005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9e5f8ac6ae8cd2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f81fbfff640 1 -- 172.21.15.45:0/2277396175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208016070 con 0x555efa185980 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f81fbfff640 1 -- 172.21.15.45:0/2277396175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555efa0e6010 con 0x555efa185980 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.207+0000 7f81fbfff640 1 -- 172.21.15.45:0/2277396175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820801b8f0 con 0x555efa185980 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.208+0000 7f821f207740 1 -- 172.21.15.45:0/2277396175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 msgr2=0x555efa189fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.208+0000 7f821f207740 1 --2- 172.21.15.45:0/2277396175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa189fc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8208000c00 tx=0x7f8208005ed0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.208+0000 7f821f207740 1 -- 172.21.15.45:0/2277396175 shutdown_connections 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.208+0000 7f821f207740 1 --2- 172.21.15.45:0/2277396175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa185980 0x555efa189fc0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.208+0000 7f821f207740 1 --2- 172.21.15.45:0/2277396175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa18e750 0x555efa18a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.208+0000 7f821f207740 1 --2- 172.21.15.45:0/2277396175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa18bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.208+0000 7f821f207740 1 -- 172.21.15.45:0/2277396175 >> 172.21.15.45:0/2277396175 conn(0x555efa17f980 msgr2=0x555efa1869f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.208+0000 7f821f207740 1 -- 172.21.15.45:0/2277396175 shutdown_connections 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.208+0000 7f821f207740 1 -- 172.21.15.45:0/2277396175 wait complete. 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.208+0000 7f821f207740 1 -- start start 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821f207740 1 Processor -- start 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821f207740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa195230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821f207740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa185980 0x555efa193800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821f207740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa18e750 0x555efa193d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821f207740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555efa195ae0 con 0x555efa18e750 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821f207740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555efa194280 con 0x555efa185980 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821f207740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555efa194440 con 0x555efa185060 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821f207740 10 client.0 _ll_register_callbacks cb 0x7f8204005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821a9e5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa18e750 0x555efa193d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821a1e4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa195230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821a9e5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa18e750 0x555efa193d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40848/0 (socket says 172.21.15.45:40848) 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821a1e4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa195230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:35448/0 (socket says 172.21.15.45:35448) 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821a9e5640 1 -- 172.21.15.45:0/140446788 learned_addr learned my addr 172.21.15.45:0/140446788 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f82199e3640 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa185980 0x555efa193800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821a9e5640 1 -- 172.21.15.45:0/140446788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 msgr2=0x555efa195230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821a9e5640 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa195230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821a9e5640 1 -- 172.21.15.45:0/140446788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa185980 msgr2=0x555efa193800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821a9e5640 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa185980 0x555efa193800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821a9e5640 1 -- 172.21.15.45:0/140446788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c017070 con 0x555efa18e750 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f82199e3640 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa185980 0x555efa193800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f821a9e5640 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa18e750 0x555efa193d40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f820c011ca0 tx=0x7f820c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688e1cf671e8363f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.209+0000 7f81f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f81f9ffb640 1 -- 172.21.15.45:0/140446788 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c024070 con 0x555efa18e750 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f81f9ffb640 1 -- 172.21.15.45:0/140446788 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f820c01f810 con 0x555efa18e750 2024-01-31T05:30:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f821f207740 1 -- 172.21.15.45:0/140446788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82040c4910 con 0x555efa18e750 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f81f9ffb640 1 -- 172.21.15.45:0/140446788 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f820c01f9d0 con 0x555efa18e750 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f821f207740 10 client.15993 fetch_fsmap learned FSMap version 27 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f821f207740 1 -- 172.21.15.45:0/140446788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555efa0eb6a0 con 0x555efa18e750 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f81f9ffb640 1 -- 172.21.15.45:0/140446788 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f820c035020 con 0x555efa18e750 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f81f9ffb640 1 -- 172.21.15.45:0/140446788 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f820c031ba0 con 0x555efa18e750 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f821f207740 10 client.15993 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f821f207740 10 client.15993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f821f207740 20 client.15993 populate_metadata read hostname 'smithi045' 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f821f207740 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555efa1f5de0 0x555efa1f8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f821f207740 4 client.15993 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f821f207740 1 -- 172.21.15.45:0/140446788 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555ef9dec600 con 0x555efa1f5de0 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.210+0000 7f821a1e4640 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555efa1f5de0 0x555efa1f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.211+0000 7f821a1e4640 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555efa1f5de0 0x555efa1f8200 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e32022f91b965b server_cookie=5e28a7e305ab17cc in_seq=0 out_seq=0 2024-01-31T05:30:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:03.211+0000 7f81f9ffb640 10 client.15993 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:03.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 772990 11.28 MB/sec execute 243 sec latency 24.083 ms 2024-01-31T05:30:03.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:03 smithi045 ceph-mon[31585]: pgmap v229: 97 pgs: 97 active+clean; 428 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 323 op/s 2024-01-31T05:30:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:03 smithi156 ceph-mon[35720]: pgmap v229: 97 pgs: 97 active+clean; 428 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 323 op/s 2024-01-31T05:30:04.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:03 smithi186 ceph-mon[35659]: pgmap v229: 97 pgs: 97 active+clean; 428 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 323 op/s 2024-01-31T05:30:04.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 774549 11.29 MB/sec execute 244 sec latency 55.365 ms 2024-01-31T05:30:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f81f9ffb640 1 -- 172.21.15.45:0/140446788 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555ef9dec600 con 0x555efa1f5de0 2024-01-31T05:30:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f81f9ffb640 10 client.15993 handle_command_reply: tid=0 2024-01-31T05:30:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f821f207740 1 client.15993 shutdown 2024-01-31T05:30:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f821f207740 1 -- 172.21.15.45:0/140446788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa18e750 msgr2=0x555efa193d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f821f207740 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa18e750 0x555efa193d40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f820c011ca0 tx=0x7f820c004ab0 comp rx=0 tx=0).stop 2024-01-31T05:30:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f821f207740 1 -- 172.21.15.45:0/140446788 shutdown_connections 2024-01-31T05:30:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f821f207740 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555efa1f5de0 0x555efa1f8200 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f821f207740 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555efa18e750 0x555efa193d40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f821f207740 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555efa185980 0x555efa193800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f81f9ffb640 0 client.15993 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f821f207740 1 --2- 172.21.15.45:0/140446788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555efa185060 0x555efa195230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f821f207740 1 -- 172.21.15.45:0/140446788 >> 172.21.15.45:0/140446788 conn(0x555efa17f980 msgr2=0x555efa189f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f821f207740 1 -- 172.21.15.45:0/140446788 shutdown_connections 2024-01-31T05:30:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f821f207740 1 -- 172.21.15.45:0/140446788 wait complete. 2024-01-31T05:30:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.014+0000 7f821f207740 20 client.15993 trim_cache size 0 max 16384 2024-01-31T05:30:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:30:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:30:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:30:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:30:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.015+0000 7f821c448640 1 -- 172.21.15.45:0/4282580633 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f81fc0a14f0 msgr2=0x7f81fc0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.015+0000 7f821c448640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f81fc0a14f0 0x7f81fc0a39b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f820c005fd0 tx=0x7f820c009500 comp rx=0 tx=0).stop 2024-01-31T05:30:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.015+0000 7f821c448640 1 -- 172.21.15.45:0/4282580633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8214068370 msgr2=0x7f8214197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.015+0000 7f821c448640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8214068370 0x7f8214197190 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8204006010 tx=0x7f8204005e00 comp rx=0 tx=0).stop 2024-01-31T05:30:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.015+0000 7f821c448640 1 -- 172.21.15.45:0/4282580633 shutdown_connections 2024-01-31T05:30:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.015+0000 7f821c448640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f81fc0a14f0 0x7f81fc0a39b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.015+0000 7f821c448640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214106e00 0x7f821419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.015+0000 7f821c448640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8214068d10 0x7f82141976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.015+0000 7f821c448640 1 --2- 172.21.15.45:0/4282580633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8214068370 0x7f8214197190 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.015+0000 7f821c448640 1 -- 172.21.15.45:0/4282580633 >> 172.21.15.45:0/4282580633 conn(0x7f8214076a40 msgr2=0x7f82140fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.015+0000 7f821c448640 1 -- 172.21.15.45:0/4282580633 shutdown_connections 2024-01-31T05:30:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:05.015+0000 7f821c448640 1 -- 172.21.15.45:0/4282580633 wait complete. 2024-01-31T05:30:05.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 776652 11.27 MB/sec execute 245 sec latency 17.261 ms 2024-01-31T05:30:05.977 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: *** Caught signal (Segmentation fault) ** 2024-01-31T05:30:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: in thread 7f3b9eace700 thread_name:md_log_replay 2024-01-31T05:30:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:30:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3baf68ed20] 2024-01-31T05:30:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 2: /usr/bin/ceph-mds(+0x5040bf) [0x5558d7ed40bf] 2024-01-31T05:30:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5558d7ed62f3] 2024-01-31T05:30:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 4: (MDLog::_replay_thread()+0x647) [0x5558d7f32697] 2024-01-31T05:30:05.978 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5558d7f3bb31] 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f3baf6841ca] 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 7: clone() 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 2024-01-31T05:30:05.706+0000 7f3b9eace700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: in thread 7f3b9eace700 thread_name:md_log_replay 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3baf68ed20] 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 2: /usr/bin/ceph-mds(+0x5040bf) [0x5558d7ed40bf] 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5558d7ed62f3] 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 4: (MDLog::_replay_thread()+0x647) [0x5558d7f32697] 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5558d7f3bb31] 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f3baf6841ca] 2024-01-31T05:30:05.979 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 7: clone() 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 0> 2024-01-31T05:30:05.706+0000 7f3b9eace700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: in thread 7f3b9eace700 thread_name:md_log_replay 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3baf68ed20] 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 2: /usr/bin/ceph-mds(+0x5040bf) [0x5558d7ed40bf] 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5558d7ed62f3] 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 4: (MDLog::_replay_thread()+0x647) [0x5558d7f32697] 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5558d7f3bb31] 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f3baf6841ca] 2024-01-31T05:30:05.980 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 7: clone() 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 0> 2024-01-31T05:30:05.706+0000 7f3b9eace700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: in thread 7f3b9eace700 thread_name:md_log_replay 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3baf68ed20] 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 2: /usr/bin/ceph-mds(+0x5040bf) [0x5558d7ed40bf] 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5558d7ed62f3] 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 4: (MDLog::_replay_thread()+0x647) [0x5558d7f32697] 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5558d7f3bb31] 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f3baf6841ca] 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 7: clone() 2024-01-31T05:30:05.981 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:30:05.982 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[70772]: 2024-01-31T05:30:05.982 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:05 smithi156 podman[71199]: 2024-01-31 05:30:05.75268907 +0000 UTC m=+0.019551104 container died d474a44b77d08d4dd5c66da9285d20797cac701a3cd4e20f60b14296b13fe043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:30:06.340 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:06 smithi156 ceph-mon[35720]: osdmap e95: 12 total, 12 up, 12 in 2024-01-31T05:30:06.340 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:06 smithi156 ceph-mon[35720]: pgmap v231: 97 pgs: 97 active+clean; 428 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 264 op/s 2024-01-31T05:30:06.341 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:06 smithi186 ceph-mon[35659]: osdmap e95: 12 total, 12 up, 12 in 2024-01-31T05:30:06.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:06 smithi186 ceph-mon[35659]: pgmap v231: 97 pgs: 97 active+clean; 428 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 264 op/s 2024-01-31T05:30:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:06 smithi045 ceph-mon[31585]: osdmap e95: 12 total, 12 up, 12 in 2024-01-31T05:30:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:06 smithi045 ceph-mon[31585]: pgmap v231: 97 pgs: 97 active+clean; 428 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 264 op/s 2024-01-31T05:30:06.726 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:06 smithi156 podman[71199]: 2024-01-31 05:30:06.34176549 +0000 UTC m=+0.608627531 container remove d474a44b77d08d4dd5c66da9285d20797cac701a3cd4e20f60b14296b13fe043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:30:06.726 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:06 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-01-31T05:30:06.727 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:06 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-01-31T05:30:06.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 778634 11.28 MB/sec execute 246 sec latency 16.001 ms 2024-01-31T05:30:07.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 781225 11.30 MB/sec execute 247 sec latency 13.916 ms 2024-01-31T05:30:08.788 INFO:tasks.workunit.client.0.smithi045.stdout: 1 782807 11.28 MB/sec execute 248 sec latency 23.359 ms 2024-01-31T05:30:08.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:08 smithi045 ceph-mon[31585]: pgmap v232: 97 pgs: 97 active+clean; 437 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 269 op/s 2024-01-31T05:30:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:08 smithi156 ceph-mon[35720]: pgmap v232: 97 pgs: 97 active+clean; 437 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 269 op/s 2024-01-31T05:30:09.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:08 smithi186 ceph-mon[35659]: pgmap v232: 97 pgs: 97 active+clean; 437 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 269 op/s 2024-01-31T05:30:09.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 784921 11.29 MB/sec execute 249 sec latency 45.252 ms 2024-01-31T05:30:10.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 786134 11.26 MB/sec execute 250 sec latency 123.465 ms 2024-01-31T05:30:10.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:10 smithi045 ceph-mon[31585]: pgmap v233: 97 pgs: 97 active+clean; 437 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 269 op/s 2024-01-31T05:30:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:10 smithi156 ceph-mon[35720]: pgmap v233: 97 pgs: 97 active+clean; 437 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 269 op/s 2024-01-31T05:30:11.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:10 smithi186 ceph-mon[35659]: pgmap v233: 97 pgs: 97 active+clean; 437 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 269 op/s 2024-01-31T05:30:11.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 788475 11.27 MB/sec execute 251 sec latency 27.840 ms 2024-01-31T05:30:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:11 smithi156 ceph-mon[35720]: pgmap v234: 97 pgs: 97 active+clean; 437 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 269 op/s 2024-01-31T05:30:12.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:11 smithi186 ceph-mon[35659]: pgmap v234: 97 pgs: 97 active+clean; 437 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 269 op/s 2024-01-31T05:30:12.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:11 smithi045 ceph-mon[31585]: pgmap v234: 97 pgs: 97 active+clean; 437 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 269 op/s 2024-01-31T05:30:12.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 790208 11.25 MB/sec execute 252 sec latency 40.217 ms 2024-01-31T05:30:13.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 792381 11.26 MB/sec execute 253 sec latency 16.976 ms 2024-01-31T05:30:13.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:13 smithi186 ceph-mon[35659]: osdmap e96: 12 total, 12 up, 12 in 2024-01-31T05:30:13.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:13 smithi045 ceph-mon[31585]: osdmap e96: 12 total, 12 up, 12 in 2024-01-31T05:30:13.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:13 smithi156 ceph-mon[35720]: osdmap e96: 12 total, 12 up, 12 in 2024-01-31T05:30:14.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 794700 11.27 MB/sec execute 254 sec latency 58.907 ms 2024-01-31T05:30:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:14 smithi045 ceph-mon[31585]: pgmap v236: 97 pgs: 97 active+clean; 448 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 276 op/s 2024-01-31T05:30:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:14 smithi156 ceph-mon[35720]: pgmap v236: 97 pgs: 97 active+clean; 448 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 276 op/s 2024-01-31T05:30:15.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:14 smithi186 ceph-mon[35659]: pgmap v236: 97 pgs: 97 active+clean; 448 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 276 op/s 2024-01-31T05:30:15.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 796313 11.25 MB/sec execute 255 sec latency 21.589 ms 2024-01-31T05:30:16.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 798942 11.27 MB/sec execute 256 sec latency 32.544 ms 2024-01-31T05:30:16.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:16 smithi045 ceph-mon[31585]: pgmap v237: 97 pgs: 97 active+clean; 448 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 235 op/s 2024-01-31T05:30:16.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:16 smithi156 ceph-mon[35720]: pgmap v237: 97 pgs: 97 active+clean; 448 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 235 op/s 2024-01-31T05:30:16.956 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:16 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 3. 2024-01-31T05:30:16.956 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:16 smithi156 systemd[1]: Stopped Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:30:16.956 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:16 smithi156 systemd[1]: Starting Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:30:17.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:16 smithi186 ceph-mon[35659]: pgmap v237: 97 pgs: 97 active+clean; 448 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 235 op/s 2024-01-31T05:30:17.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:16 smithi156 podman[71307]: 2024-01-31 05:30:16.860283223 +0000 UTC m=+0.017409581 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:30:17.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:16 smithi156 podman[71307]: 2024-01-31 05:30:16.984101389 +0000 UTC m=+0.141227732 container create 04889acaff5f200245fb35f14d8e8372298f6a4f1befbc46c562355851f24a5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, org.label-schema.license=GPLv2) 2024-01-31T05:30:17.603 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:17 smithi156 podman[71307]: 2024-01-31 05:30:17.26165023 +0000 UTC m=+0.418776581 container init 04889acaff5f200245fb35f14d8e8372298f6a4f1befbc46c562355851f24a5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=) 2024-01-31T05:30:17.603 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:17 smithi156 podman[71307]: 2024-01-31 05:30:17.26457056 +0000 UTC m=+0.421696916 container start 04889acaff5f200245fb35f14d8e8372298f6a4f1befbc46c562355851f24a5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123) 2024-01-31T05:30:17.603 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:17 smithi156 ceph-mds[71326]: -- 172.21.15.156:0/1380341262 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55974411b520 con 0x5597440f1000 2024-01-31T05:30:17.603 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:17 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: starting mds.h at 2024-01-31T05:30:17.603 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:17 smithi156 bash[71307]: 04889acaff5f200245fb35f14d8e8372298f6a4f1befbc46c562355851f24a5d 2024-01-31T05:30:17.603 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:30:17 smithi156 systemd[1]: Started Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:30:17.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 800554 11.26 MB/sec execute 257 sec latency 20.568 ms 2024-01-31T05:30:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:17 smithi045 ceph-mon[31585]: daemon mds.h restarted 2024-01-31T05:30:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:17 smithi045 ceph-mon[31585]: osdmap e97: 12 total, 12 up, 12 in 2024-01-31T05:30:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:17 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.156:6836/455612015,v1:172.21.15.156:6837/455612015] up:boot 2024-01-31T05:30:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:17 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:30:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:17 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:30:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:17 smithi156 ceph-mon[35720]: daemon mds.h restarted 2024-01-31T05:30:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:17 smithi156 ceph-mon[35720]: osdmap e97: 12 total, 12 up, 12 in 2024-01-31T05:30:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:17 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.156:6836/455612015,v1:172.21.15.156:6837/455612015] up:boot 2024-01-31T05:30:17.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:17 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:30:17.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:17 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:30:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:17 smithi186 ceph-mon[35659]: daemon mds.h restarted 2024-01-31T05:30:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:17 smithi186 ceph-mon[35659]: osdmap e97: 12 total, 12 up, 12 in 2024-01-31T05:30:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:17 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.156:6836/455612015,v1:172.21.15.156:6837/455612015] up:boot 2024-01-31T05:30:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:17 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:30:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:17 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:30:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.104+0000 7f78505f1640 1 -- 172.21.15.45:0/588566497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7848104b90 msgr2=0x7f7848104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.104+0000 7f78505f1640 1 --2- 172.21.15.45:0/588566497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7848104b90 0x7f7848104f70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7840009ca0 tx=0x7f784002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.104+0000 7f78505f1640 1 -- 172.21.15.45:0/588566497 shutdown_connections 2024-01-31T05:30:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.104+0000 7f78505f1640 1 --2- 172.21.15.45:0/588566497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7848100800 0x7f784810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.104+0000 7f78505f1640 1 --2- 172.21.15.45:0/588566497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78480ffe40 0x7f78481002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.104+0000 7f78505f1640 1 --2- 172.21.15.45:0/588566497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7848104b90 0x7f7848104f70 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.104+0000 7f78505f1640 1 -- 172.21.15.45:0/588566497 >> 172.21.15.45:0/588566497 conn(0x7f7848076a40 msgr2=0x7f7848076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.104+0000 7f78505f1640 1 -- 172.21.15.45:0/588566497 shutdown_connections 2024-01-31T05:30:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.104+0000 7f78505f1640 1 -- 172.21.15.45:0/588566497 wait complete. 2024-01-31T05:30:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.104+0000 7f78505f1640 1 Processor -- start 2024-01-31T05:30:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f78505f1640 1 -- start start 2024-01-31T05:30:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f78505f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78480ffe40 0x7f7848199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f78505f1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7848100800 0x7f78481999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f78505f1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7848104b90 0x7f784819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f78505f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784819e330 con 0x7f78480ffe40 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f78505f1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784819e4a0 con 0x7f7848100800 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f78505f1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784819e660 con 0x7f7848104b90 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f784e38d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78480ffe40 0x7f7848199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f784e38d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78480ffe40 0x7f7848199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40734/0 (socket says 172.21.15.45:40734) 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f784e38d640 1 -- 172.21.15.45:0/1810933746 learned_addr learned my addr 172.21.15.45:0/1810933746 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f784db8c640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7848100800 0x7f78481999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f784eb8e640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7848104b90 0x7f784819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f784db8c640 1 -- 172.21.15.45:0/1810933746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7848104b90 msgr2=0x7f784819dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f784db8c640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7848104b90 0x7f784819dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f784db8c640 1 -- 172.21.15.45:0/1810933746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78480ffe40 msgr2=0x7f7848199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f784db8c640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78480ffe40 0x7f7848199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f784db8c640 1 -- 172.21.15.45:0/1810933746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7840009900 con 0x7f7848100800 2024-01-31T05:30:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f784e38d640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78480ffe40 0x7f7848199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.105+0000 7f784db8c640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7848100800 0x7f78481999b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f783c00e9f0 tx=0x7f783c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.106+0000 7f78477fe640 1 -- 172.21.15.45:0/1810933746 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c00ccc0 con 0x7f7848100800 2024-01-31T05:30:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.106+0000 7f78505f1640 1 -- 172.21.15.45:0/1810933746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784819e940 con 0x7f7848100800 2024-01-31T05:30:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.106+0000 7f78477fe640 1 -- 172.21.15.45:0/1810933746 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f783c00ce20 con 0x7f7848100800 2024-01-31T05:30:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.106+0000 7f78505f1640 1 -- 172.21.15.45:0/1810933746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78481a61c0 con 0x7f7848100800 2024-01-31T05:30:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.106+0000 7f78477fe640 1 -- 172.21.15.45:0/1810933746 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c004a00 con 0x7f7848100800 2024-01-31T05:30:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.108+0000 7f78477fe640 1 -- 172.21.15.45:0/1810933746 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f783c005020 con 0x7f7848100800 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.108+0000 7f78533ba740 1 Processor -- start 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.108+0000 7f78533ba740 1 -- start start 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f78533ba740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777cf060 0x55c2777cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f78533ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c2777d8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f78533ba740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777d8750 0x55c2777dab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f78533ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27773e040 con 0x55c2777cf980 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f78533ba740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27773b730 con 0x55c2777cf060 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f78533ba740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c277737020 con 0x55c2777d8750 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f784e38d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c2777d8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f784e38d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c2777d8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40736/0 (socket says 172.21.15.45:40736) 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f78477fe640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f78300a14f0 0x7f78300a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f784db8c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777d8750 0x55c2777dab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f784e38d640 1 -- 172.21.15.45:0/4274584709 learned_addr learned my addr 172.21.15.45:0/4274584709 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f784eb8e640 1 --2- 172.21.15.45:0/4274584709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777cf060 0x55c2777cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f784eb8e640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f78300a14f0 0x7f78300a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f784e38d640 1 -- 172.21.15.45:0/4274584709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777d8750 msgr2=0x55c2777dab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f784e38d640 1 --2- 172.21.15.45:0/4274584709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777d8750 0x55c2777dab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f784e38d640 1 -- 172.21.15.45:0/4274584709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777cf060 msgr2=0x55c2777cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f784e38d640 1 --2- 172.21.15.45:0/4274584709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777cf060 0x55c2777cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f784e38d640 1 -- 172.21.15.45:0/4274584709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c27772ee40 con 0x55c2777cf980 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f78477fe640 1 -- 172.21.15.45:0/1810933746 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f783c014070 con 0x7f7848100800 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.109+0000 7f784e38d640 1 --2- 172.21.15.45:0/4274584709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c2777d8210 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7840006010 tx=0x7f7840005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bc3f563cd50175 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.110+0000 7f784eb8e640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f78300a14f0 0x7f78300a39b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7838009660 tx=0x7f7838004730 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.110+0000 7f7844bfe640 1 -- 172.21.15.45:0/4274584709 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840032ca0 con 0x55c2777cf980 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.110+0000 7f7844bfe640 1 -- 172.21.15.45:0/4274584709 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7840032e00 con 0x55c2777cf980 2024-01-31T05:30:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.110+0000 7f7844bfe640 1 -- 172.21.15.45:0/4274584709 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840003cf0 con 0x55c2777cf980 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.110+0000 7f78533ba740 1 -- 172.21.15.45:0/4274584709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 msgr2=0x55c2777d8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.110+0000 7f78533ba740 1 --2- 172.21.15.45:0/4274584709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c2777d8210 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7840006010 tx=0x7f7840005f70 comp rx=0 tx=0).stop 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.110+0000 7f78533ba740 1 -- 172.21.15.45:0/4274584709 shutdown_connections 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.110+0000 7f78533ba740 1 --2- 172.21.15.45:0/4274584709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c2777d8210 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.110+0000 7f78533ba740 1 --2- 172.21.15.45:0/4274584709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777d8750 0x55c2777dab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.110+0000 7f78533ba740 1 --2- 172.21.15.45:0/4274584709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777cf060 0x55c2777cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f78533ba740 1 -- 172.21.15.45:0/4274584709 >> 172.21.15.45:0/4274584709 conn(0x55c2777c9980 msgr2=0x55c2777c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f78533ba740 1 -- 172.21.15.45:0/4274584709 shutdown_connections 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f78533ba740 1 -- 172.21.15.45:0/4274584709 wait complete. 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f78533ba740 1 -- start start 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f78533ba740 1 Processor -- start 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f78533ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf060 0x55c2777d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f78533ba740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777cf980 0x55c2777d5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f78533ba740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777d8750 0x55c2777d60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f78533ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2776f01b0 con 0x55c2777cf060 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f78533ba740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2776efe50 con 0x55c2777cf980 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f78533ba740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2776b5090 con 0x55c2777d8750 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f784e38d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf060 0x55c2777d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f784e38d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf060 0x55c2777d5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40742/0 (socket says 172.21.15.45:40742) 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f784e38d640 1 -- 172.21.15.45:0/1584529681 learned_addr learned my addr 172.21.15.45:0/1584529681 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f784db8c640 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777cf980 0x55c2777d5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f78533ba740 10 client.0 _ll_register_callbacks cb 0x55c2776daa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.111+0000 7f784eb8e640 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777d8750 0x55c2777d60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f784e38d640 1 -- 172.21.15.45:0/1584529681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777d8750 msgr2=0x55c2777d60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f784e38d640 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777d8750 0x55c2777d60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f784e38d640 1 -- 172.21.15.45:0/1584529681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777cf980 msgr2=0x55c2777d5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f784e38d640 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777cf980 0x55c2777d5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f784e38d640 1 -- 172.21.15.45:0/1584529681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784810d3d0 con 0x55c2777cf060 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f784db8c640 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777cf980 0x55c2777d5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f784e38d640 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf060 0x55c2777d5660 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7840009dc0 tx=0x7f7840032a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793bfb95bd163ff6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f782dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f782dffb640 1 -- 172.21.15.45:0/1584529681 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840032ca0 con 0x55c2777cf060 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f782dffb640 1 -- 172.21.15.45:0/1584529681 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f784003f690 con 0x55c2777cf060 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f78533ba740 1 -- 172.21.15.45:0/1584529681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c277730010 con 0x55c2777cf060 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f782dffb640 1 -- 172.21.15.45:0/1584529681 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7840034ea0 con 0x55c2777cf060 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f78533ba740 10 client.16011 fetch_fsmap learned FSMap version 28 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f78533ba740 1 -- 172.21.15.45:0/1584529681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c2777356a0 con 0x55c2777cf060 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.112+0000 7f782dffb640 1 -- 172.21.15.45:0/1584529681 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c277730010 con 0x55c2777cf060 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.113+0000 7f782dffb640 1 -- 172.21.15.45:0/1584529681 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f78400487c0 con 0x55c2777cf060 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.113+0000 7f78533ba740 10 client.16011 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.113+0000 7f78533ba740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:30:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.113+0000 7f78533ba740 20 client.16011 populate_metadata read hostname 'smithi045' 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.113+0000 7f78533ba740 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c2777e81b0 0x55c2777ea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.113+0000 7f78533ba740 4 client.16011 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.113+0000 7f78533ba740 1 -- 172.21.15.45:0/1584529681 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c27772fd30 con 0x55c2777e81b0 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.113+0000 7f784db8c640 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c2777e81b0 0x55c2777ea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.113+0000 7f784db8c640 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c2777e81b0 0x55c2777ea5c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f453edc798ec9 server_cookie=4d1cc84d04d2a2f4 in_seq=0 out_seq=0 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.113+0000 7f782dffb640 10 client.16011 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.114+0000 7f782dffb640 1 -- 172.21.15.45:0/1584529681 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c27772fd30 con 0x55c2777e81b0 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.114+0000 7f782dffb640 10 client.16011 handle_command_reply: tid=0 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.114+0000 7f78533ba740 1 client.16011 shutdown 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.114+0000 7f78533ba740 1 -- 172.21.15.45:0/1584529681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf060 msgr2=0x55c2777d5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.114+0000 7f78533ba740 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf060 0x55c2777d5660 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7840009dc0 tx=0x7f7840032a90 comp rx=0 tx=0).stop 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.114+0000 7f78533ba740 1 -- 172.21.15.45:0/1584529681 shutdown_connections 2024-01-31T05:30:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.114+0000 7f78533ba740 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c2777e81b0 0x55c2777ea5c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.114+0000 7f78533ba740 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777d8750 0x55c2777d60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.114+0000 7f78533ba740 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777cf980 0x55c2777d5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.114+0000 7f782dffb640 0 client.16011 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.114+0000 7f78533ba740 1 --2- 172.21.15.45:0/1584529681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf060 0x55c2777d5660 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.114+0000 7f78533ba740 1 -- 172.21.15.45:0/1584529681 >> 172.21.15.45:0/1584529681 conn(0x55c2777c9980 msgr2=0x55c2777cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.115+0000 7f78533ba740 1 -- 172.21.15.45:0/1584529681 shutdown_connections 2024-01-31T05:30:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.115+0000 7f78533ba740 1 -- 172.21.15.45:0/1584529681 wait complete. 2024-01-31T05:30:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.115+0000 7f78533ba740 20 client.16011 trim_cache size 0 max 16384 2024-01-31T05:30:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.128+0000 7f78533ba740 1 Processor -- start 2024-01-31T05:30:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.128+0000 7f78533ba740 1 -- start start 2024-01-31T05:30:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f78533ba740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777cf060 0x55c2777df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f78533ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c2777df920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f78533ba740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777d8750 0x55c2777c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f78533ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2777ca980 con 0x55c2777cf980 2024-01-31T05:30:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f78533ba740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2777c97b0 con 0x55c2777d8750 2024-01-31T05:30:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f78533ba740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2777d3bb0 con 0x55c2777cf060 2024-01-31T05:30:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784db8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c2777df920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784db8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c2777df920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40752/0 (socket says 172.21.15.45:40752) 2024-01-31T05:30:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784db8c640 1 -- 172.21.15.45:0/3400636996 learned_addr learned my addr 172.21.15.45:0/3400636996 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784e38d640 1 --2- 172.21.15.45:0/3400636996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777cf060 0x55c2777df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784eb8e640 1 --2- 172.21.15.45:0/3400636996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777d8750 0x55c2777c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784db8c640 1 -- 172.21.15.45:0/3400636996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777cf060 msgr2=0x55c2777df3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784db8c640 1 --2- 172.21.15.45:0/3400636996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777cf060 0x55c2777df3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784db8c640 1 -- 172.21.15.45:0/3400636996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777d8750 msgr2=0x55c2777c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784db8c640 1 --2- 172.21.15.45:0/3400636996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777d8750 0x55c2777c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784db8c640 1 -- 172.21.15.45:0/3400636996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2776d2030 con 0x55c2777cf980 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784eb8e640 1 --2- 172.21.15.45:0/3400636996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777d8750 0x55c2777c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784e38d640 1 --2- 172.21.15.45:0/3400636996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777cf060 0x55c2777df3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f784db8c640 1 --2- 172.21.15.45:0/3400636996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c2777df920 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f783c00c6a0 tx=0x7f783c096140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96ea310576fb1bb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f782ffff640 1 -- 172.21.15.45:0/3400636996 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c099630 con 0x55c2777cf980 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f782ffff640 1 -- 172.21.15.45:0/3400636996 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f783c09a3d0 con 0x55c2777cf980 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.129+0000 7f782ffff640 1 -- 172.21.15.45:0/3400636996 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c098070 con 0x55c2777cf980 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.130+0000 7f78533ba740 1 -- 172.21.15.45:0/3400636996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 msgr2=0x55c2777df920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.130+0000 7f78533ba740 1 --2- 172.21.15.45:0/3400636996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c2777df920 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f783c00c6a0 tx=0x7f783c096140 comp rx=0 tx=0).stop 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.130+0000 7f78533ba740 1 -- 172.21.15.45:0/3400636996 shutdown_connections 2024-01-31T05:30:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.130+0000 7f78533ba740 1 --2- 172.21.15.45:0/3400636996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c2777df920 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.130+0000 7f78533ba740 1 --2- 172.21.15.45:0/3400636996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777d8750 0x55c2777c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.130+0000 7f78533ba740 1 --2- 172.21.15.45:0/3400636996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777cf060 0x55c2777df3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.130+0000 7f78533ba740 1 -- 172.21.15.45:0/3400636996 >> 172.21.15.45:0/3400636996 conn(0x55c2777c9980 msgr2=0x55c2777c8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.130+0000 7f78533ba740 1 -- 172.21.15.45:0/3400636996 shutdown_connections 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.130+0000 7f78533ba740 1 -- 172.21.15.45:0/3400636996 wait complete. 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.130+0000 7f78533ba740 1 -- start start 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.130+0000 7f78533ba740 1 Processor -- start 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f78533ba740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777cf060 0x55c277838700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f78533ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c277838c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f78533ba740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777d8750 0x55c277832b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f78533ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c277833120 con 0x55c2777cf980 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f78533ba740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c277833290 con 0x55c2777d8750 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f78533ba740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c277833450 con 0x55c2777cf060 2024-01-31T05:30:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f78533ba740 10 client.0 _ll_register_callbacks cb 0x7f7838005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f784db8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c277838c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f784db8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c277838c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40760/0 (socket says 172.21.15.45:40760) 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f784db8c640 1 -- 172.21.15.45:0/1019789953 learned_addr learned my addr 172.21.15.45:0/1019789953 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f784e38d640 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777cf060 0x55c277838700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f784eb8e640 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777d8750 0x55c277832b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f784e38d640 1 -- 172.21.15.45:0/1019789953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777d8750 msgr2=0x55c277832b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f784e38d640 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777d8750 0x55c277832b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f784e38d640 1 -- 172.21.15.45:0/1019789953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 msgr2=0x55c277838c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f784e38d640 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c277838c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f784e38d640 1 -- 172.21.15.45:0/1019789953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783c0ce040 con 0x55c2777cf060 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f784db8c640 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c277838c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f784e38d640 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777cf060 0x55c277838700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7840002230 tx=0x7f784002c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b93b092f40c6c16c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f782dffb640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:30:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.131+0000 7f782dffb640 1 -- 172.21.15.45:0/1019789953 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840003b80 con 0x55c2777cf060 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f782dffb640 1 -- 172.21.15.45:0/1019789953 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7840003ce0 con 0x55c2777cf060 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f78533ba740 1 -- 172.21.15.45:0/1019789953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7840032e00 con 0x55c2777cf060 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f782dffb640 1 -- 172.21.15.45:0/1019789953 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7840003ea0 con 0x55c2777cf060 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f78533ba740 10 client.25570 fetch_fsmap learned FSMap version 28 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f78533ba740 1 -- 172.21.15.45:0/1019789953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c2776f7a50 con 0x55c2777cf060 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f782dffb640 1 -- 172.21.15.45:0/1019789953 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7840032e00 con 0x55c2777cf060 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f782dffb640 1 -- 172.21.15.45:0/1019789953 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f784003fb90 con 0x55c2777cf060 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f78533ba740 10 client.25570 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f78533ba740 10 client.25570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f78533ba740 20 client.25570 populate_metadata read hostname 'smithi045' 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f78533ba740 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c277840220 0x55c277842610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f78533ba740 4 client.25570 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f78533ba740 1 -- 172.21.15.45:0/1019789953 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c277436600 con 0x55c277840220 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.132+0000 7f784db8c640 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c277840220 0x55c277842610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.133+0000 7f782dffb640 10 client.25570 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.133+0000 7f784db8c640 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c277840220 0x55c277842610 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fe8478c95c33ebc server_cookie=3eb81c78d7cae99c in_seq=0 out_seq=0 2024-01-31T05:30:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f782dffb640 1 -- 172.21.15.45:0/1019789953 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c277436600 con 0x55c277840220 2024-01-31T05:30:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f782dffb640 10 client.25570 handle_command_reply: tid=0 2024-01-31T05:30:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f78533ba740 1 client.25570 shutdown 2024-01-31T05:30:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f78533ba740 1 -- 172.21.15.45:0/1019789953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777cf060 msgr2=0x55c277838700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f78533ba740 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777cf060 0x55c277838700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7840002230 tx=0x7f784002c870 comp rx=0 tx=0).stop 2024-01-31T05:30:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f78533ba740 1 -- 172.21.15.45:0/1019789953 shutdown_connections 2024-01-31T05:30:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f78533ba740 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c277840220 0x55c277842610 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f78533ba740 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2777cf980 0x55c277838c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f782dffb640 0 client.25570 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f78533ba740 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2777d8750 0x55c277832b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f78533ba740 1 --2- 172.21.15.45:0/1019789953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2777cf060 0x55c277838700 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f78533ba740 1 -- 172.21.15.45:0/1019789953 >> 172.21.15.45:0/1019789953 conn(0x55c2777c9980 msgr2=0x55c2777c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f78533ba740 1 -- 172.21.15.45:0/1019789953 shutdown_connections 2024-01-31T05:30:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f78533ba740 1 -- 172.21.15.45:0/1019789953 wait complete. 2024-01-31T05:30:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.144+0000 7f78533ba740 20 client.25570 trim_cache size 0 max 16384 2024-01-31T05:30:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:30:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:30:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:30:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:30:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.145+0000 7f78505f1640 1 -- 172.21.15.45:0/1810933746 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f78300a14f0 msgr2=0x7f78300a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.145+0000 7f78505f1640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f78300a14f0 0x7f78300a39b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7838009660 tx=0x7f7838004730 comp rx=0 tx=0).stop 2024-01-31T05:30:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.145+0000 7f78505f1640 1 -- 172.21.15.45:0/1810933746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7848100800 msgr2=0x7f78481999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.145+0000 7f78505f1640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7848100800 0x7f78481999b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f783c00e9f0 tx=0x7f783c00eeb0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.145+0000 7f78505f1640 1 -- 172.21.15.45:0/1810933746 shutdown_connections 2024-01-31T05:30:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.145+0000 7f78505f1640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f78300a14f0 0x7f78300a39b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.145+0000 7f78505f1640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7848104b90 0x7f784819dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.145+0000 7f78505f1640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7848100800 0x7f78481999b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.145+0000 7f78505f1640 1 --2- 172.21.15.45:0/1810933746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78480ffe40 0x7f7848199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.145+0000 7f78505f1640 1 -- 172.21.15.45:0/1810933746 >> 172.21.15.45:0/1810933746 conn(0x7f7848076a40 msgr2=0x7f784810b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.145+0000 7f78505f1640 1 -- 172.21.15.45:0/1810933746 shutdown_connections 2024-01-31T05:30:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:18.145+0000 7f78505f1640 1 -- 172.21.15.45:0/1810933746 wait complete. 2024-01-31T05:30:18.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 802836 11.27 MB/sec execute 258 sec latency 18.353 ms 2024-01-31T05:30:18.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:18 smithi045 ceph-mon[31585]: pgmap v238: 97 pgs: 97 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 232 op/s 2024-01-31T05:30:18.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:18 smithi156 ceph-mon[35720]: pgmap v238: 97 pgs: 97 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 232 op/s 2024-01-31T05:30:19.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:18 smithi186 ceph-mon[35659]: pgmap v238: 97 pgs: 97 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 232 op/s 2024-01-31T05:30:19.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 804835 11.25 MB/sec execute 259 sec latency 52.858 ms 2024-01-31T05:30:20.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 806512 11.25 MB/sec execute 260 sec latency 63.089 ms 2024-01-31T05:30:20.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:20 smithi045 ceph-mon[31585]: pgmap v240: 97 pgs: 97 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 290 op/s 2024-01-31T05:30:20.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:20 smithi156 ceph-mon[35720]: pgmap v240: 97 pgs: 97 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 290 op/s 2024-01-31T05:30:21.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:20 smithi186 ceph-mon[35659]: pgmap v240: 97 pgs: 97 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 290 op/s 2024-01-31T05:30:21.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 808390 11.24 MB/sec execute 261 sec latency 168.043 ms 2024-01-31T05:30:22.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 810404 11.24 MB/sec execute 262 sec latency 17.517 ms 2024-01-31T05:30:22.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:22 smithi045 ceph-mon[31585]: pgmap v241: 97 pgs: 97 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-01-31T05:30:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:22 smithi156 ceph-mon[35720]: pgmap v241: 97 pgs: 97 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-01-31T05:30:23.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:22 smithi186 ceph-mon[35659]: pgmap v241: 97 pgs: 97 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-01-31T05:30:23.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 813053 11.26 MB/sec execute 263 sec latency 14.008 ms 2024-01-31T05:30:24.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 814712 11.25 MB/sec execute 264 sec latency 25.850 ms 2024-01-31T05:30:24.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:24 smithi045 ceph-mon[31585]: pgmap v242: 97 pgs: 97 active+clean; 434 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 231 op/s 2024-01-31T05:30:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:24 smithi156 ceph-mon[35720]: pgmap v242: 97 pgs: 97 active+clean; 434 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 231 op/s 2024-01-31T05:30:25.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:24 smithi186 ceph-mon[35659]: pgmap v242: 97 pgs: 97 active+clean; 434 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 231 op/s 2024-01-31T05:30:25.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 816865 11.26 MB/sec execute 265 sec latency 100.309 ms 2024-01-31T05:30:26.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 818932 11.24 MB/sec execute 266 sec latency 21.128 ms 2024-01-31T05:30:26.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:26 smithi045 ceph-mon[31585]: pgmap v243: 97 pgs: 97 active+clean; 434 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 231 op/s 2024-01-31T05:30:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:26 smithi156 ceph-mon[35720]: pgmap v243: 97 pgs: 97 active+clean; 434 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 231 op/s 2024-01-31T05:30:27.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:26 smithi186 ceph-mon[35659]: pgmap v243: 97 pgs: 97 active+clean; 434 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 231 op/s 2024-01-31T05:30:27.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 820755 11.24 MB/sec execute 267 sec latency 14.499 ms 2024-01-31T05:30:28.741 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:28 smithi156 ceph-mon[35720]: pgmap v244: 97 pgs: 97 active+clean; 443 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 230 op/s 2024-01-31T05:30:28.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 823115 11.25 MB/sec execute 268 sec latency 14.878 ms 2024-01-31T05:30:28.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:28 smithi045 ceph-mon[31585]: pgmap v244: 97 pgs: 97 active+clean; 443 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 230 op/s 2024-01-31T05:30:28.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:28 smithi186 ceph-mon[35659]: pgmap v244: 97 pgs: 97 active+clean; 443 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 230 op/s 2024-01-31T05:30:29.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 825094 11.25 MB/sec execute 269 sec latency 16.240 ms 2024-01-31T05:30:30.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 827360 11.26 MB/sec execute 270 sec latency 95.019 ms 2024-01-31T05:30:30.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:30 smithi045 ceph-mon[31585]: pgmap v245: 97 pgs: 97 active+clean; 443 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 192 op/s 2024-01-31T05:30:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:30 smithi156 ceph-mon[35720]: pgmap v245: 97 pgs: 97 active+clean; 443 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 192 op/s 2024-01-31T05:30:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:30 smithi186 ceph-mon[35659]: pgmap v245: 97 pgs: 97 active+clean; 443 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 192 op/s 2024-01-31T05:30:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.241+0000 7fe36e6ae640 1 -- 172.21.15.45:0/2053169213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3680ffe40 msgr2=0x7fe3681002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.241+0000 7fe36e6ae640 1 --2- 172.21.15.45:0/2053169213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3680ffe40 0x7fe3681002a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009930 tx=0x7fe35c02bc80 comp rx=0 tx=0).stop 2024-01-31T05:30:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.241+0000 7fe36e6ae640 1 -- 172.21.15.45:0/2053169213 shutdown_connections 2024-01-31T05:30:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.241+0000 7fe36e6ae640 1 --2- 172.21.15.45:0/2053169213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3681007e0 0x7fe36810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.241+0000 7fe36e6ae640 1 --2- 172.21.15.45:0/2053169213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3680ffe40 0x7fe3681002a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.241+0000 7fe36e6ae640 1 --2- 172.21.15.45:0/2053169213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe368104bb0 0x7fe368104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.241+0000 7fe36e6ae640 1 -- 172.21.15.45:0/2053169213 >> 172.21.15.45:0/2053169213 conn(0x7fe368076a50 msgr2=0x7fe368076e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.241+0000 7fe36e6ae640 1 -- 172.21.15.45:0/2053169213 shutdown_connections 2024-01-31T05:30:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.241+0000 7fe36e6ae640 1 -- 172.21.15.45:0/2053169213 wait complete. 2024-01-31T05:30:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.242+0000 7fe36e6ae640 1 Processor -- start 2024-01-31T05:30:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.242+0000 7fe36e6ae640 1 -- start start 2024-01-31T05:30:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.242+0000 7fe36e6ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3680ffe40 0x7fe368195090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.242+0000 7fe36e6ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3681007e0 0x7fe3681955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.242+0000 7fe36e6ae640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe368104bb0 0x7fe368199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.242+0000 7fe36e6ae640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe368199f00 con 0x7fe3681007e0 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.242+0000 7fe36e6ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36819a070 con 0x7fe3680ffe40 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.242+0000 7fe36e6ae640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36819a230 con 0x7fe368104bb0 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe3677fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3681007e0 0x7fe3681955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe36cc4b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe368104bb0 0x7fe368199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe367fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3680ffe40 0x7fe368195090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe367fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3680ffe40 0x7fe368195090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:47396/0 (socket says 172.21.15.45:47396) 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe367fff640 1 -- 172.21.15.45:0/281993151 learned_addr learned my addr 172.21.15.45:0/281993151 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe36cc4b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe368104bb0 0x7fe368199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:39682/0 (socket says 172.21.15.45:39682) 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe3677fe640 1 -- 172.21.15.45:0/281993151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe368104bb0 msgr2=0x7fe368199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe3677fe640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe368104bb0 0x7fe368199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe3677fe640 1 -- 172.21.15.45:0/281993151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3680ffe40 msgr2=0x7fe368195090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe3677fe640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3680ffe40 0x7fe368195090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe3677fe640 1 -- 172.21.15.45:0/281993151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35800b6a0 con 0x7fe3681007e0 2024-01-31T05:30:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe36cc4b640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe368104bb0 0x7fe368199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe367fff640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3680ffe40 0x7fe368195090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe3677fe640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3681007e0 0x7fe3681955d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009230 tx=0x7fe35c02cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe3657fa640 1 -- 172.21.15.45:0/281993151 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c02c5c0 con 0x7fe3681007e0 2024-01-31T05:30:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe36e6ae640 1 -- 172.21.15.45:0/281993151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c009560 con 0x7fe3681007e0 2024-01-31T05:30:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe3657fa640 1 -- 172.21.15.45:0/281993151 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe35c0054c0 con 0x7fe3681007e0 2024-01-31T05:30:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe3657fa640 1 -- 172.21.15.45:0/281993151 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c003ac0 con 0x7fe3681007e0 2024-01-31T05:30:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.243+0000 7fe36e6ae640 1 -- 172.21.15.45:0/281993151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3681a1bb0 con 0x7fe3681007e0 2024-01-31T05:30:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.246+0000 7fe3657fa640 1 -- 172.21.15.45:0/281993151 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fe35c0042e0 con 0x7fe3681007e0 2024-01-31T05:30:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.246+0000 7fe371470740 1 Processor -- start 2024-01-31T05:30:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.246+0000 7fe371470740 1 -- start start 2024-01-31T05:30:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.246+0000 7fe371470740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e82333440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.246+0000 7fe371470740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e8233c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.246+0000 7fe371470740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8233eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.246+0000 7fe371470740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e822a2040 con 0x558e8233c750 2024-01-31T05:30:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.246+0000 7fe371470740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8229f730 con 0x558e82333060 2024-01-31T05:30:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.246+0000 7fe371470740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8229b020 con 0x558e82333980 2024-01-31T05:30:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.246+0000 7fe36cc4b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e82333440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.246+0000 7fe36cc4b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e82333440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:47398/0 (socket says 172.21.15.45:47398) 2024-01-31T05:30:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe36cc4b640 1 -- 172.21.15.45:0/3043689871 learned_addr learned my addr 172.21.15.45:0/3043689871 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe3677fe640 1 --2- 172.21.15.45:0/3043689871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8233eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe367fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e8233c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe3657fa640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe3500a14f0 0x7fe3500a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe36cc4b640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe3500a14f0 0x7fe3500a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe36cc4b640 1 -- 172.21.15.45:0/3043689871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 msgr2=0x558e8233c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe36cc4b640 1 --2- 172.21.15.45:0/3043689871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e8233c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe36cc4b640 1 -- 172.21.15.45:0/3043689871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 msgr2=0x558e8233eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe36cc4b640 1 --2- 172.21.15.45:0/3043689871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8233eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe3657fa640 1 -- 172.21.15.45:0/281993151 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe35c02de40 con 0x7fe3681007e0 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe36cc4b640 1 -- 172.21.15.45:0/3043689871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e82292e40 con 0x558e82333060 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe36cc4b640 1 --2- 172.21.15.45:0/3043689871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e82333440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe36000fe50 tx=0x7fe360013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70d75a805191a6a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe36cc4b640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe3500a14f0 0x7fe3500a39b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe36001d000 tx=0x7fe36001f000 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe34e7fc640 1 -- 172.21.15.45:0/3043689871 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36000f870 con 0x558e82333060 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe34e7fc640 1 -- 172.21.15.45:0/3043689871 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3600120d0 con 0x558e82333060 2024-01-31T05:30:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.247+0000 7fe34e7fc640 1 -- 172.21.15.45:0/3043689871 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360010be0 con 0x558e82333060 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.248+0000 7fe371470740 1 -- 172.21.15.45:0/3043689871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 msgr2=0x558e82333440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.248+0000 7fe371470740 1 --2- 172.21.15.45:0/3043689871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e82333440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe36000fe50 tx=0x7fe360013830 comp rx=0 tx=0).stop 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.248+0000 7fe371470740 1 -- 172.21.15.45:0/3043689871 shutdown_connections 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.248+0000 7fe371470740 1 --2- 172.21.15.45:0/3043689871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8233eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.248+0000 7fe371470740 1 --2- 172.21.15.45:0/3043689871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e8233c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.248+0000 7fe371470740 1 --2- 172.21.15.45:0/3043689871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e82333440 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.248+0000 7fe371470740 1 -- 172.21.15.45:0/3043689871 >> 172.21.15.45:0/3043689871 conn(0x558e8232d980 msgr2=0x558e8232dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.248+0000 7fe371470740 1 -- 172.21.15.45:0/3043689871 shutdown_connections 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.248+0000 7fe371470740 1 -- 172.21.15.45:0/3043689871 wait complete. 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe371470740 1 -- start start 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe371470740 1 Processor -- start 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe371470740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e82339690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe371470740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e82339bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe371470740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8233a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe371470740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e822541b0 con 0x558e8233c750 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe371470740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e82253e50 con 0x558e82333060 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe371470740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e82219090 con 0x558e82333980 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe371470740 10 client.0 _ll_register_callbacks cb 0x558e8223ea90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe36cc4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8233a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe3677fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e82339bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe36cc4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8233a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57050/0 (socket says 172.21.15.45:57050) 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe36cc4b640 1 -- 172.21.15.45:0/2421295278 learned_addr learned my addr 172.21.15.45:0/2421295278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe367fff640 1 --2- 172.21.15.45:0/2421295278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e82339690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe36cc4b640 1 -- 172.21.15.45:0/2421295278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 msgr2=0x558e82339bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe36cc4b640 1 --2- 172.21.15.45:0/2421295278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e82339bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe36cc4b640 1 -- 172.21.15.45:0/2421295278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 msgr2=0x558e82339690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe36cc4b640 1 --2- 172.21.15.45:0/2421295278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e82339690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.249+0000 7fe36cc4b640 1 -- 172.21.15.45:0/2421295278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36000fa30 con 0x558e8233c750 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe36cc4b640 1 --2- 172.21.15.45:0/2421295278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8233a110 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe360004020 tx=0x7fe360005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97196b172f2861d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe3477fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe3477fe640 1 -- 172.21.15.45:0/2421295278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360010cf0 con 0x558e8233c750 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe3477fe640 1 -- 172.21.15.45:0/2421295278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe36000f870 con 0x558e8233c750 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe371470740 1 -- 172.21.15.45:0/2421295278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e82294010 con 0x558e8233c750 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe3477fe640 1 -- 172.21.15.45:0/2421295278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe360010e50 con 0x558e8233c750 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe371470740 10 client.16029 fetch_fsmap learned FSMap version 28 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe371470740 1 -- 172.21.15.45:0/2421295278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558e822996a0 con 0x558e8233c750 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe3477fe640 1 -- 172.21.15.45:0/2421295278 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e82294010 con 0x558e8233c750 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe3477fe640 1 -- 172.21.15.45:0/2421295278 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe360027420 con 0x558e8233c750 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe371470740 10 client.16029 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe371470740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:30:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe371470740 20 client.16029 populate_metadata read hostname 'smithi045' 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe371470740 1 --2- 172.21.15.45:0/2421295278 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558e8234c4d0 0x558e8234e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe371470740 4 client.16029 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.250+0000 7fe371470740 1 -- 172.21.15.45:0/2421295278 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e82293d30 con 0x558e8234c4d0 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.251+0000 7fe367fff640 1 --2- 172.21.15.45:0/2421295278 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558e8234c4d0 0x558e8234e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.251+0000 7fe367fff640 1 --2- 172.21.15.45:0/2421295278 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558e8234c4d0 0x558e8234e8c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cbe9245b3f12de8 server_cookie=729dd77766e42cdb in_seq=0 out_seq=0 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.251+0000 7fe3477fe640 10 client.16029 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe3477fe640 1 -- 172.21.15.45:0/2421295278 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e82293d30 con 0x558e8234c4d0 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe3477fe640 10 client.16029 handle_command_reply: tid=0 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe371470740 1 client.16029 shutdown 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe371470740 1 -- 172.21.15.45:0/2421295278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 msgr2=0x558e8233a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe371470740 1 --2- 172.21.15.45:0/2421295278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8233a110 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe360004020 tx=0x7fe360005fb0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe371470740 1 -- 172.21.15.45:0/2421295278 shutdown_connections 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe371470740 1 --2- 172.21.15.45:0/2421295278 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558e8234c4d0 0x558e8234e8c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe371470740 1 --2- 172.21.15.45:0/2421295278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8233a110 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe371470740 1 --2- 172.21.15.45:0/2421295278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e82339bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe3477fe640 0 client.16029 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe371470740 1 --2- 172.21.15.45:0/2421295278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e82339690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe371470740 1 -- 172.21.15.45:0/2421295278 >> 172.21.15.45:0/2421295278 conn(0x558e8232d980 msgr2=0x558e8232f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe371470740 1 -- 172.21.15.45:0/2421295278 shutdown_connections 2024-01-31T05:30:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe371470740 1 -- 172.21.15.45:0/2421295278 wait complete. 2024-01-31T05:30:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.252+0000 7fe371470740 20 client.16029 trim_cache size 0 max 16384 2024-01-31T05:30:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.270+0000 7fe371470740 1 Processor -- start 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.270+0000 7fe371470740 1 -- start start 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.270+0000 7fe371470740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333060 0x558e823387f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.270+0000 7fe371470740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333980 0x558e82339f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.270+0000 7fe371470740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8232c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.270+0000 7fe371470740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e82293d30 con 0x558e8233c750 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.270+0000 7fe371470740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e82294010 con 0x558e82333980 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.270+0000 7fe371470740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8233eb40 con 0x558e82333060 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.270+0000 7fe367fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333060 0x558e823387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.270+0000 7fe367fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333060 0x558e823387f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:39698/0 (socket says 172.21.15.45:39698) 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.270+0000 7fe367fff640 1 -- 172.21.15.45:0/2472380546 learned_addr learned my addr 172.21.15.45:0/2472380546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.270+0000 7fe3677fe640 1 --2- 172.21.15.45:0/2472380546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333980 0x558e82339f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.271+0000 7fe36cc4b640 1 --2- 172.21.15.45:0/2472380546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8232c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.271+0000 7fe367fff640 1 -- 172.21.15.45:0/2472380546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333980 msgr2=0x558e82339f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.271+0000 7fe367fff640 1 --2- 172.21.15.45:0/2472380546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333980 0x558e82339f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.271+0000 7fe367fff640 1 -- 172.21.15.45:0/2472380546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 msgr2=0x558e8232c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.271+0000 7fe367fff640 1 --2- 172.21.15.45:0/2472380546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8232c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.271+0000 7fe367fff640 1 -- 172.21.15.45:0/2472380546 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e82236030 con 0x558e82333060 2024-01-31T05:30:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.271+0000 7fe36cc4b640 1 --2- 172.21.15.45:0/2472380546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8232c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:30:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.271+0000 7fe367fff640 1 --2- 172.21.15.45:0/2472380546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333060 0x558e823387f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe358016630 tx=0x7fe35800d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b4883a79e55749d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.271+0000 7fe34dffb640 1 -- 172.21.15.45:0/2472380546 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358010040 con 0x558e82333060 2024-01-31T05:30:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.271+0000 7fe34dffb640 1 -- 172.21.15.45:0/2472380546 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe358025070 con 0x558e82333060 2024-01-31T05:30:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.271+0000 7fe34dffb640 1 -- 172.21.15.45:0/2472380546 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580039d0 con 0x558e82333060 2024-01-31T05:30:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.272+0000 7fe371470740 1 -- 172.21.15.45:0/2472380546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333060 msgr2=0x558e823387f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.272+0000 7fe371470740 1 --2- 172.21.15.45:0/2472380546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333060 0x558e823387f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe358016630 tx=0x7fe35800d040 comp rx=0 tx=0).stop 2024-01-31T05:30:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.272+0000 7fe371470740 1 -- 172.21.15.45:0/2472380546 shutdown_connections 2024-01-31T05:30:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.272+0000 7fe371470740 1 --2- 172.21.15.45:0/2472380546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8232c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.272+0000 7fe371470740 1 --2- 172.21.15.45:0/2472380546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333980 0x558e82339f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.272+0000 7fe371470740 1 --2- 172.21.15.45:0/2472380546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333060 0x558e823387f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.272+0000 7fe371470740 1 -- 172.21.15.45:0/2472380546 >> 172.21.15.45:0/2472380546 conn(0x558e8232d980 msgr2=0x558e8232c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.272+0000 7fe371470740 1 -- 172.21.15.45:0/2472380546 shutdown_connections 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.272+0000 7fe371470740 1 -- 172.21.15.45:0/2472380546 wait complete. 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.272+0000 7fe371470740 1 -- start start 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe371470740 1 Processor -- start 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe371470740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e823427d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe371470740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e82342d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe371470740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8239c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe371470740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8239cf30 con 0x558e8233c750 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe371470740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8239d0a0 con 0x558e82333060 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe371470740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8239d260 con 0x558e82333980 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe371470740 10 client.0 _ll_register_callbacks cb 0x7fe360009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe367fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e823427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe367fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e823427d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:47416/0 (socket says 172.21.15.45:47416) 2024-01-31T05:30:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe36cc4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8239c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe367fff640 1 -- 172.21.15.45:0/3206212305 learned_addr learned my addr 172.21.15.45:0/3206212305 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.273+0000 7fe3677fe640 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e82342d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe367fff640 1 -- 172.21.15.45:0/3206212305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 msgr2=0x558e82342d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe367fff640 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e82342d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe367fff640 1 -- 172.21.15.45:0/3206212305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 msgr2=0x558e8239c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe367fff640 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8239c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe367fff640 1 -- 172.21.15.45:0/3206212305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe358021be0 con 0x558e82333060 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe36cc4b640 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8239c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe3677fe640 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e82342d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe367fff640 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e823427d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe35800d010 tx=0x7fe358016fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e3186cd124e8db7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe3477fe640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe3477fe640 1 -- 172.21.15.45:0/3206212305 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358010050 con 0x558e82333060 2024-01-31T05:30:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe3477fe640 1 -- 172.21.15.45:0/3206212305 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe35801e040 con 0x558e82333060 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe371470740 1 -- 172.21.15.45:0/3206212305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3600120d0 con 0x558e82333060 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe3477fe640 1 -- 172.21.15.45:0/3206212305 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe358004260 con 0x558e82333060 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe371470740 10 client.25553 fetch_fsmap learned FSMap version 28 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe371470740 1 -- 172.21.15.45:0/3206212305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558e8225ba50 con 0x558e82333060 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.274+0000 7fe3477fe640 1 -- 172.21.15.45:0/3206212305 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3600120d0 con 0x558e82333060 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.275+0000 7fe3477fe640 1 -- 172.21.15.45:0/3206212305 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe358003a90 con 0x558e82333060 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.275+0000 7fe371470740 10 client.25553 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.275+0000 7fe371470740 10 client.25553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.275+0000 7fe371470740 20 client.25553 populate_metadata read hostname 'smithi045' 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.275+0000 7fe371470740 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558e8234c4d0 0x558e823a6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.275+0000 7fe371470740 4 client.25553 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.275+0000 7fe371470740 1 -- 172.21.15.45:0/3206212305 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558e81f9a600 con 0x558e8234c4d0 2024-01-31T05:30:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.275+0000 7fe3677fe640 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558e8234c4d0 0x558e823a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.275+0000 7fe3677fe640 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558e8234c4d0 0x558e823a6920 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca8352cfbd92ae5 server_cookie=ac9a0126b31dc6dd in_seq=0 out_seq=0 2024-01-31T05:30:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.275+0000 7fe3477fe640 10 client.25553 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.285+0000 7fe3477fe640 1 -- 172.21.15.45:0/3206212305 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558e81f9a600 con 0x558e8234c4d0 2024-01-31T05:30:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.285+0000 7fe3477fe640 10 client.25553 handle_command_reply: tid=0 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.285+0000 7fe371470740 1 client.25553 shutdown 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.285+0000 7fe371470740 1 -- 172.21.15.45:0/3206212305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 msgr2=0x558e823427d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.285+0000 7fe371470740 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e823427d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe35800d010 tx=0x7fe358016fa0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.285+0000 7fe371470740 1 -- 172.21.15.45:0/3206212305 shutdown_connections 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.285+0000 7fe371470740 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558e8234c4d0 0x558e823a6920 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.285+0000 7fe371470740 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e8233c750 0x558e8239c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.285+0000 7fe371470740 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e82333980 0x558e82342d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.285+0000 7fe371470740 1 --2- 172.21.15.45:0/3206212305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e82333060 0x558e823427d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.285+0000 7fe3477fe640 0 client.25553 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.285+0000 7fe371470740 1 -- 172.21.15.45:0/3206212305 >> 172.21.15.45:0/3206212305 conn(0x558e8232d980 msgr2=0x558e82339f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe371470740 1 -- 172.21.15.45:0/3206212305 shutdown_connections 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe371470740 1 -- 172.21.15.45:0/3206212305 wait complete. 2024-01-31T05:30:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe371470740 20 client.25553 trim_cache size 0 max 16384 2024-01-31T05:30:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:30:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:30:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:30:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:30:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe36e6ae640 1 -- 172.21.15.45:0/281993151 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe3500a14f0 msgr2=0x7fe3500a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe36e6ae640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe3500a14f0 0x7fe3500a39b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe36001d000 tx=0x7fe36001f000 comp rx=0 tx=0).stop 2024-01-31T05:30:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe36e6ae640 1 -- 172.21.15.45:0/281993151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3681007e0 msgr2=0x7fe3681955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe36e6ae640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3681007e0 0x7fe3681955d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009230 tx=0x7fe35c02cd10 comp rx=0 tx=0).stop 2024-01-31T05:30:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe36e6ae640 1 -- 172.21.15.45:0/281993151 shutdown_connections 2024-01-31T05:30:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe36e6ae640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe3500a14f0 0x7fe3500a39b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe36e6ae640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3681007e0 0x7fe3681955d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe36e6ae640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe368104bb0 0x7fe368199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe36e6ae640 1 --2- 172.21.15.45:0/281993151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3680ffe40 0x7fe368195090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe36e6ae640 1 -- 172.21.15.45:0/281993151 >> 172.21.15.45:0/281993151 conn(0x7fe368076a50 msgr2=0x7fe3680fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe36e6ae640 1 -- 172.21.15.45:0/281993151 shutdown_connections 2024-01-31T05:30:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:31.286+0000 7fe36e6ae640 1 -- 172.21.15.45:0/281993151 wait complete. 2024-01-31T05:30:31.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 829143 11.24 MB/sec execute 271 sec latency 20.623 ms 2024-01-31T05:30:32.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 831052 11.25 MB/sec execute 272 sec latency 192.883 ms 2024-01-31T05:30:32.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:32 smithi045 ceph-mon[31585]: pgmap v246: 97 pgs: 97 active+clean; 443 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-01-31T05:30:32.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:32 smithi156 ceph-mon[35720]: pgmap v246: 97 pgs: 97 active+clean; 443 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-01-31T05:30:33.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:32 smithi186 ceph-mon[35659]: pgmap v246: 97 pgs: 97 active+clean; 443 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-01-31T05:30:33.789 INFO:tasks.workunit.client.0.smithi045.stdout: 1 833166 11.23 MB/sec execute 273 sec latency 16.895 ms 2024-01-31T05:30:33.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:33 smithi045 ceph-mon[31585]: pgmap v247: 97 pgs: 97 active+clean; 452 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.0 MiB/s wr, 289 op/s 2024-01-31T05:30:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:33 smithi156 ceph-mon[35720]: pgmap v247: 97 pgs: 97 active+clean; 452 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.0 MiB/s wr, 289 op/s 2024-01-31T05:30:34.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:33 smithi186 ceph-mon[35659]: pgmap v247: 97 pgs: 97 active+clean; 452 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.0 MiB/s wr, 289 op/s 2024-01-31T05:30:34.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 835104 11.24 MB/sec execute 274 sec latency 17.122 ms 2024-01-31T05:30:35.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 837379 11.24 MB/sec execute 275 sec latency 38.424 ms 2024-01-31T05:30:36.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 839062 11.23 MB/sec execute 276 sec latency 121.003 ms 2024-01-31T05:30:36.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:36 smithi045 ceph-mon[31585]: pgmap v248: 97 pgs: 97 active+clean; 452 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-01-31T05:30:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:36 smithi156 ceph-mon[35720]: pgmap v248: 97 pgs: 97 active+clean; 452 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-01-31T05:30:37.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:36 smithi186 ceph-mon[35659]: pgmap v248: 97 pgs: 97 active+clean; 452 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-01-31T05:30:37.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 841414 11.25 MB/sec execute 277 sec latency 93.491 ms 2024-01-31T05:30:38.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 843057 11.23 MB/sec execute 278 sec latency 53.019 ms 2024-01-31T05:30:38.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:38 smithi186 ceph-mon[35659]: pgmap v249: 97 pgs: 97 active+clean; 462 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 285 op/s 2024-01-31T05:30:38.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:38 smithi045 ceph-mon[31585]: pgmap v249: 97 pgs: 97 active+clean; 462 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 285 op/s 2024-01-31T05:30:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:38 smithi156 ceph-mon[35720]: pgmap v249: 97 pgs: 97 active+clean; 462 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 285 op/s 2024-01-31T05:30:39.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 845396 11.24 MB/sec execute 279 sec latency 16.905 ms 2024-01-31T05:30:40.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 847197 11.22 MB/sec execute 280 sec latency 62.996 ms 2024-01-31T05:30:40.842 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:40 smithi186 ceph-mon[35659]: pgmap v250: 97 pgs: 97 active+clean; 462 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-01-31T05:30:40.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:40 smithi045 ceph-mon[31585]: pgmap v250: 97 pgs: 97 active+clean; 462 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-01-31T05:30:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:40 smithi156 ceph-mon[35720]: pgmap v250: 97 pgs: 97 active+clean; 462 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-01-31T05:30:41.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 849124 11.23 MB/sec execute 281 sec latency 32.018 ms 2024-01-31T05:30:42.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 851172 11.22 MB/sec execute 282 sec latency 47.435 ms 2024-01-31T05:30:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:42 smithi045 ceph-mon[31585]: pgmap v251: 97 pgs: 97 active+clean; 462 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-01-31T05:30:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:42 smithi156 ceph-mon[35720]: pgmap v251: 97 pgs: 97 active+clean; 462 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-01-31T05:30:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:42 smithi186 ceph-mon[35659]: pgmap v251: 97 pgs: 97 active+clean; 462 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-01-31T05:30:43.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 852916 11.22 MB/sec execute 283 sec latency 88.693 ms 2024-01-31T05:30:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.373+0000 7f447ac67640 1 -- 172.21.15.45:0/2046157608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44741018e0 msgr2=0x7f4474105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.373+0000 7f447ac67640 1 --2- 172.21.15.45:0/2046157608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44741018e0 0x7f4474105920 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4464009930 tx=0x7f446402bc30 comp rx=0 tx=0).stop 2024-01-31T05:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.373+0000 7f447ac67640 1 -- 172.21.15.45:0/2046157608 shutdown_connections 2024-01-31T05:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.373+0000 7f447ac67640 1 --2- 172.21.15.45:0/2046157608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44741018e0 0x7f4474105920 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.373+0000 7f447ac67640 1 --2- 172.21.15.45:0/2046157608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4474106000 0x7f447410dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.373+0000 7f447ac67640 1 --2- 172.21.15.45:0/2046157608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4474100f30 0x7f4474101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.373+0000 7f447ac67640 1 -- 172.21.15.45:0/2046157608 >> 172.21.15.45:0/2046157608 conn(0x7f44740fc7d0 msgr2=0x7f44740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.373+0000 7f447ac67640 1 -- 172.21.15.45:0/2046157608 shutdown_connections 2024-01-31T05:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.373+0000 7f447ac67640 1 -- 172.21.15.45:0/2046157608 wait complete. 2024-01-31T05:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.373+0000 7f447ac67640 1 Processor -- start 2024-01-31T05:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.373+0000 7f447ac67640 1 -- start start 2024-01-31T05:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f447ac67640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4474100f30 0x7f4474197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f447ac67640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4474106000 0x7f4474197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f447ac67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f447419bac0 0x7f447419bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f447ac67640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447419c4b0 con 0x7f447419bac0 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f447ac67640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447419c620 con 0x7f4474106000 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f447ac67640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447419c790 con 0x7f4474100f30 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4479204640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f447419bac0 0x7f447419bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4479204640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f447419bac0 0x7f447419bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36182/0 (socket says 172.21.15.45:36182) 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4479204640 1 -- 172.21.15.45:0/688437441 learned_addr learned my addr 172.21.15.45:0/688437441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4478a03640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4474100f30 0x7f4474197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4473fff640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4474106000 0x7f4474197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4479204640 1 -- 172.21.15.45:0/688437441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4474100f30 msgr2=0x7f4474197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4479204640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4474100f30 0x7f4474197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4479204640 1 -- 172.21.15.45:0/688437441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4474106000 msgr2=0x7f4474197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4479204640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4474106000 0x7f4474197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4479204640 1 -- 172.21.15.45:0/688437441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464009560 con 0x7f447419bac0 2024-01-31T05:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4479204640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f447419bac0 0x7f447419bf70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f445c00d750 tx=0x7f445c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4471ffb640 1 -- 172.21.15.45:0/688437441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c009bd0 con 0x7f447419bac0 2024-01-31T05:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4471ffb640 1 -- 172.21.15.45:0/688437441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445c00de60 con 0x7f447419bac0 2024-01-31T05:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f447ac67640 1 -- 172.21.15.45:0/688437441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44741a3d70 con 0x7f447419bac0 2024-01-31T05:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.374+0000 7f4471ffb640 1 -- 172.21.15.45:0/688437441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c004e00 con 0x7f447419bac0 2024-01-31T05:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.375+0000 7f447ac67640 1 -- 172.21.15.45:0/688437441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44741a41a0 con 0x7f447419bac0 2024-01-31T05:30:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.377+0000 7f4471ffb640 1 -- 172.21.15.45:0/688437441 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f445c004f60 con 0x7f447419bac0 2024-01-31T05:30:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.377+0000 7f4471ffb640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f44600a14f0 0x7f44600a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.377+0000 7f447da21740 1 Processor -- start 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.377+0000 7f447da21740 1 -- start start 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.377+0000 7f4471ffb640 1 -- 172.21.15.45:0/688437441 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f445c0cc7b0 con 0x7f447419bac0 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.377+0000 7f4478a03640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f44600a14f0 0x7f44600a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.377+0000 7f447da21740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc060 0x5575b4ffc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.377+0000 7f447da21740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc980 0x5575b5005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f447da21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b5007b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f447da21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b4f6b040 con 0x5575b5005750 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f447da21740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b4f68730 con 0x5575b4ffc060 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f447da21740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b4f64020 con 0x5575b4ffc980 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4479204640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b5007b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4479204640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b5007b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36192/0 (socket says 172.21.15.45:36192) 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4479204640 1 -- 172.21.15.45:0/1021567550 learned_addr learned my addr 172.21.15.45:0/1021567550 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4473fff640 1 --2- 172.21.15.45:0/1021567550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc980 0x5575b5005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4478a03640 1 --2- 172.21.15.45:0/1021567550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc060 0x5575b4ffc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4478a03640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f44600a14f0 0x7f44600a39b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f446c005fd0 tx=0x7f446c009680 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4479204640 1 -- 172.21.15.45:0/1021567550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc980 msgr2=0x5575b5005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4479204640 1 --2- 172.21.15.45:0/1021567550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc980 0x5575b5005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4479204640 1 -- 172.21.15.45:0/1021567550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc060 msgr2=0x5575b4ffc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4479204640 1 --2- 172.21.15.45:0/1021567550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc060 0x5575b4ffc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4479204640 1 -- 172.21.15.45:0/1021567550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575b4f5be40 con 0x5575b5005750 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4478a03640 1 --2- 172.21.15.45:0/1021567550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc060 0x5575b4ffc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4473fff640 1 --2- 172.21.15.45:0/1021567550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc980 0x5575b5005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f4479204640 1 --2- 172.21.15.45:0/1021567550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b5007b40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f445c0960f0 tx=0x7f445c046ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d257f7bc51367a3d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f446affd640 1 -- 172.21.15.45:0/1021567550 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c09d720 con 0x5575b5005750 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f446affd640 1 -- 172.21.15.45:0/1021567550 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445c0a1960 con 0x5575b5005750 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.378+0000 7f446affd640 1 -- 172.21.15.45:0/1021567550 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c095d90 con 0x5575b5005750 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.379+0000 7f447da21740 1 -- 172.21.15.45:0/1021567550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 msgr2=0x5575b5007b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.379+0000 7f447da21740 1 --2- 172.21.15.45:0/1021567550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b5007b40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f445c0960f0 tx=0x7f445c046ff0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.379+0000 7f447da21740 1 -- 172.21.15.45:0/1021567550 shutdown_connections 2024-01-31T05:30:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.379+0000 7f447da21740 1 --2- 172.21.15.45:0/1021567550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b5007b40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.379+0000 7f447da21740 1 --2- 172.21.15.45:0/1021567550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc980 0x5575b5005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.379+0000 7f447da21740 1 --2- 172.21.15.45:0/1021567550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc060 0x5575b4ffc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.379+0000 7f447da21740 1 -- 172.21.15.45:0/1021567550 >> 172.21.15.45:0/1021567550 conn(0x5575b4ff6980 msgr2=0x5575b4ff6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.379+0000 7f447da21740 1 -- 172.21.15.45:0/1021567550 shutdown_connections 2024-01-31T05:30:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.379+0000 7f447da21740 1 -- 172.21.15.45:0/1021567550 wait complete. 2024-01-31T05:30:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.379+0000 7f447da21740 1 -- start start 2024-01-31T05:30:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.379+0000 7f447da21740 1 Processor -- start 2024-01-31T05:30:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f447da21740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc060 0x5575b500f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f447da21740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc980 0x5575b500f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f447da21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b5009760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f447da21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b4f1d1b0 con 0x5575b5005750 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f447da21740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b4f1ce50 con 0x5575b4ffc980 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f447da21740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b4ee2090 con 0x5575b4ffc060 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f447da21740 10 client.0 _ll_register_callbacks cb 0x5575b4f07a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4479204640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b5009760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4478a03640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc060 0x5575b500f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4479204640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b5009760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36196/0 (socket says 172.21.15.45:36196) 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4479204640 1 -- 172.21.15.45:0/3162177104 learned_addr learned my addr 172.21.15.45:0/3162177104 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4473fff640 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc980 0x5575b500f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4479204640 1 -- 172.21.15.45:0/3162177104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc060 msgr2=0x5575b500f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4479204640 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc060 0x5575b500f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4479204640 1 -- 172.21.15.45:0/3162177104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc980 msgr2=0x5575b500f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4479204640 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc980 0x5575b500f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4479204640 1 -- 172.21.15.45:0/3162177104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44741a41a0 con 0x5575b5005750 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4473fff640 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc980 0x5575b500f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4478a03640 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc060 0x5575b500f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4479204640 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b5009760 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f445c099610 tx=0x7f445c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae09bb974a0c398 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4453fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.380+0000 7f4453fff640 1 -- 172.21.15.45:0/3162177104 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c014070 con 0x5575b5005750 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f4453fff640 1 -- 172.21.15.45:0/3162177104 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f445c095d90 con 0x5575b5005750 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f447da21740 1 -- 172.21.15.45:0/3162177104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575b4f5d010 con 0x5575b5005750 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f4453fff640 1 -- 172.21.15.45:0/3162177104 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f445c04c070 con 0x5575b5005750 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f447da21740 10 client.16044 fetch_fsmap learned FSMap version 28 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f447da21740 1 -- 172.21.15.45:0/3162177104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5575b4f626a0 con 0x5575b5005750 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f4453fff640 1 -- 172.21.15.45:0/3162177104 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5575b4f5d010 con 0x5575b5005750 2024-01-31T05:30:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f4453fff640 1 -- 172.21.15.45:0/3162177104 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f445c0af070 con 0x5575b5005750 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f447da21740 10 client.16044 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f447da21740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f447da21740 20 client.16044 populate_metadata read hostname 'smithi045' 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f447da21740 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5575b5019490 0x5575b501b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f447da21740 4 client.16044 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f447da21740 1 -- 172.21.15.45:0/3162177104 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575b4f5cd30 con 0x5575b5019490 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.381+0000 7f4478a03640 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5575b5019490 0x5575b501b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.382+0000 7f4453fff640 10 client.16044 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.382+0000 7f4478a03640 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5575b5019490 0x5575b501b880 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=310ff9815b4eebc server_cookie=4f203925dc904ebf in_seq=0 out_seq=0 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.382+0000 7f4453fff640 1 -- 172.21.15.45:0/3162177104 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5575b4f5cd30 con 0x5575b5019490 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.382+0000 7f4453fff640 10 client.16044 handle_command_reply: tid=0 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.382+0000 7f447da21740 1 client.16044 shutdown 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.382+0000 7f447da21740 1 -- 172.21.15.45:0/3162177104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 msgr2=0x5575b5009760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.382+0000 7f447da21740 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b5009760 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f445c099610 tx=0x7f445c0057b0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.382+0000 7f447da21740 1 -- 172.21.15.45:0/3162177104 shutdown_connections 2024-01-31T05:30:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.382+0000 7f447da21740 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5575b5019490 0x5575b501b880 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.383+0000 7f447da21740 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b5009760 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.383+0000 7f447da21740 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc980 0x5575b500f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.383+0000 7f4453fff640 0 client.16044 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.383+0000 7f447da21740 1 --2- 172.21.15.45:0/3162177104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc060 0x5575b500f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.383+0000 7f447da21740 1 -- 172.21.15.45:0/3162177104 >> 172.21.15.45:0/3162177104 conn(0x5575b4ff6980 msgr2=0x5575b4ff86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.383+0000 7f447da21740 1 -- 172.21.15.45:0/3162177104 shutdown_connections 2024-01-31T05:30:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.383+0000 7f447da21740 1 -- 172.21.15.45:0/3162177104 wait complete. 2024-01-31T05:30:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.383+0000 7f447da21740 20 client.16044 trim_cache size 0 max 16384 2024-01-31T05:30:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f447da21740 1 Processor -- start 2024-01-31T05:30:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f447da21740 1 -- start start 2024-01-31T05:30:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f447da21740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc060 0x5575b5008ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f447da21740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc980 0x5575b5009530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f447da21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b500eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f447da21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b4ff64b0 con 0x5575b5005750 2024-01-31T05:30:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f447da21740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b4f5cd30 con 0x5575b4ffc060 2024-01-31T05:30:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f447da21740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b4f5d010 con 0x5575b4ffc980 2024-01-31T05:30:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4479204640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b500eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4479204640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b500eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36206/0 (socket says 172.21.15.45:36206) 2024-01-31T05:30:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4479204640 1 -- 172.21.15.45:0/4283084100 learned_addr learned my addr 172.21.15.45:0/4283084100 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4473fff640 1 --2- 172.21.15.45:0/4283084100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc980 0x5575b5009530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4478a03640 1 --2- 172.21.15.45:0/4283084100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc060 0x5575b5008ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4479204640 1 -- 172.21.15.45:0/4283084100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc980 msgr2=0x5575b5009530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4479204640 1 --2- 172.21.15.45:0/4283084100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc980 0x5575b5009530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4479204640 1 -- 172.21.15.45:0/4283084100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc060 msgr2=0x5575b5008ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4479204640 1 --2- 172.21.15.45:0/4283084100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc060 0x5575b5008ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4479204640 1 -- 172.21.15.45:0/4283084100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575b4eff030 con 0x5575b5005750 2024-01-31T05:30:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4473fff640 1 --2- 172.21.15.45:0/4283084100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc980 0x5575b5009530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:30:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4478a03640 1 --2- 172.21.15.45:0/4283084100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc060 0x5575b5008ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f4479204640 1 --2- 172.21.15.45:0/4283084100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b500eeb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f445c00d720 tx=0x7f445c099580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582e49f92d3fbe9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.398+0000 7f446a7fc640 1 -- 172.21.15.45:0/4283084100 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c014070 con 0x5575b5005750 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.399+0000 7f446a7fc640 1 -- 172.21.15.45:0/4283084100 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445c097880 con 0x5575b5005750 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.399+0000 7f446a7fc640 1 -- 172.21.15.45:0/4283084100 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c0a14d0 con 0x5575b5005750 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.399+0000 7f447da21740 1 -- 172.21.15.45:0/4283084100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 msgr2=0x5575b500eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.399+0000 7f447da21740 1 --2- 172.21.15.45:0/4283084100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b500eeb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f445c00d720 tx=0x7f445c099580 comp rx=0 tx=0).stop 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.399+0000 7f447da21740 1 -- 172.21.15.45:0/4283084100 shutdown_connections 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.399+0000 7f447da21740 1 --2- 172.21.15.45:0/4283084100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b5005750 0x5575b500eeb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.399+0000 7f447da21740 1 --2- 172.21.15.45:0/4283084100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b4ffc980 0x5575b5009530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.399+0000 7f447da21740 1 --2- 172.21.15.45:0/4283084100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc060 0x5575b5008ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.399+0000 7f447da21740 1 -- 172.21.15.45:0/4283084100 >> 172.21.15.45:0/4283084100 conn(0x5575b4ff6980 msgr2=0x5575b5001e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.399+0000 7f447da21740 1 -- 172.21.15.45:0/4283084100 shutdown_connections 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.399+0000 7f447da21740 1 -- 172.21.15.45:0/4283084100 wait complete. 2024-01-31T05:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f447da21740 1 -- start start 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f447da21740 1 Processor -- start 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f447da21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b4ffc060 0x5575b5069250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f447da21740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc980 0x5575b5069790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f447da21740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b5005750 0x5575b50638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f447da21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b5063e80 con 0x5575b4ffc060 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f447da21740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b5063ff0 con 0x5575b4ffc980 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f447da21740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b50641b0 con 0x5575b5005750 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f447da21740 10 client.0 _ll_register_callbacks cb 0x7f445c0959e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f4478a03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b4ffc060 0x5575b5069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f4478a03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b4ffc060 0x5575b5069250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36218/0 (socket says 172.21.15.45:36218) 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f4478a03640 1 -- 172.21.15.45:0/596467305 learned_addr learned my addr 172.21.15.45:0/596467305 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f4473fff640 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc980 0x5575b5069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f4479204640 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b5005750 0x5575b50638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f4478a03640 1 -- 172.21.15.45:0/596467305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b5005750 msgr2=0x5575b50638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f4478a03640 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b5005750 0x5575b50638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f4478a03640 1 -- 172.21.15.45:0/596467305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc980 msgr2=0x5575b5069790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.400+0000 7f4478a03640 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc980 0x5575b5069790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f4478a03640 1 -- 172.21.15.45:0/596467305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575b4f626a0 con 0x5575b4ffc060 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f4478a03640 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b4ffc060 0x5575b5069250 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f446c009fd0 tx=0x7f446c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2e6e127dc1a328 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f4453fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f4453fff640 1 -- 172.21.15.45:0/596467305 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c026050 con 0x5575b4ffc060 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f4453fff640 1 -- 172.21.15.45:0/596467305 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f446c028630 con 0x5575b4ffc060 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f447da21740 1 -- 172.21.15.45:0/596467305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f445c0a1960 con 0x5575b4ffc060 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f4453fff640 1 -- 172.21.15.45:0/596467305 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f446c0287f0 con 0x5575b4ffc060 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f447da21740 10 client.16050 fetch_fsmap learned FSMap version 28 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f447da21740 1 -- 172.21.15.45:0/596467305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5575b4f24a50 con 0x5575b4ffc060 2024-01-31T05:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f4453fff640 1 -- 172.21.15.45:0/596467305 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445c0a1960 con 0x5575b4ffc060 2024-01-31T05:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f4453fff640 1 -- 172.21.15.45:0/596467305 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f446c03a410 con 0x5575b4ffc060 2024-01-31T05:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f447da21740 10 client.16050 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f447da21740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f447da21740 20 client.16050 populate_metadata read hostname 'smithi045' 2024-01-31T05:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f447da21740 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5575b50710e0 0x5575b5073500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f447da21740 4 client.16050 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.401+0000 7f447da21740 1 -- 172.21.15.45:0/596467305 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5575b4c63600 con 0x5575b50710e0 2024-01-31T05:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.402+0000 7f4473fff640 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5575b50710e0 0x5575b5073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.402+0000 7f4453fff640 10 client.16050 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.402+0000 7f4473fff640 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5575b50710e0 0x5575b5073500 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58a0179a0a5c0205 server_cookie=901908acf6af685c in_seq=0 out_seq=0 2024-01-31T05:30:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f4453fff640 1 -- 172.21.15.45:0/596467305 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5575b4c63600 con 0x5575b50710e0 2024-01-31T05:30:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f4453fff640 10 client.16050 handle_command_reply: tid=0 2024-01-31T05:30:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447da21740 1 client.16050 shutdown 2024-01-31T05:30:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447da21740 1 -- 172.21.15.45:0/596467305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b4ffc060 msgr2=0x5575b5069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447da21740 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b4ffc060 0x5575b5069250 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f446c009fd0 tx=0x7f446c004ab0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447da21740 1 -- 172.21.15.45:0/596467305 shutdown_connections 2024-01-31T05:30:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447da21740 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5575b50710e0 0x5575b5073500 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447da21740 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575b5005750 0x5575b50638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f4453fff640 0 client.16050 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447da21740 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575b4ffc980 0x5575b5069790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447da21740 1 --2- 172.21.15.45:0/596467305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575b4ffc060 0x5575b5069250 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447da21740 1 -- 172.21.15.45:0/596467305 >> 172.21.15.45:0/596467305 conn(0x5575b4ff6980 msgr2=0x5575b5008ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447da21740 1 -- 172.21.15.45:0/596467305 shutdown_connections 2024-01-31T05:30:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447da21740 1 -- 172.21.15.45:0/596467305 wait complete. 2024-01-31T05:30:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447da21740 20 client.16050 trim_cache size 0 max 16384 2024-01-31T05:30:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:30:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:30:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:30:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:30:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.413+0000 7f447ac67640 1 -- 172.21.15.45:0/688437441 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f44600a14f0 msgr2=0x7f44600a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.414+0000 7f447ac67640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f44600a14f0 0x7f44600a39b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f446c005fd0 tx=0x7f446c009680 comp rx=0 tx=0).stop 2024-01-31T05:30:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.414+0000 7f447ac67640 1 -- 172.21.15.45:0/688437441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f447419bac0 msgr2=0x7f447419bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.414+0000 7f447ac67640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f447419bac0 0x7f447419bf70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f445c00d750 tx=0x7f445c00dc10 comp rx=0 tx=0).stop 2024-01-31T05:30:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.414+0000 7f447ac67640 1 -- 172.21.15.45:0/688437441 shutdown_connections 2024-01-31T05:30:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.414+0000 7f447ac67640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f44600a14f0 0x7f44600a39b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.414+0000 7f447ac67640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f447419bac0 0x7f447419bf70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.414+0000 7f447ac67640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4474106000 0x7f4474197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.414+0000 7f447ac67640 1 --2- 172.21.15.45:0/688437441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4474100f30 0x7f4474197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.414+0000 7f447ac67640 1 -- 172.21.15.45:0/688437441 >> 172.21.15.45:0/688437441 conn(0x7f44740fc7d0 msgr2=0x7f44740feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.414+0000 7f447ac67640 1 -- 172.21.15.45:0/688437441 shutdown_connections 2024-01-31T05:30:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:44.414+0000 7f447ac67640 1 -- 172.21.15.45:0/688437441 wait complete. 2024-01-31T05:30:44.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 855449 11.23 MB/sec execute 284 sec latency 15.486 ms 2024-01-31T05:30:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:44 smithi045 ceph-mon[31585]: pgmap v252: 97 pgs: 97 active+clean; 471 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 287 op/s 2024-01-31T05:30:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:44 smithi156 ceph-mon[35720]: pgmap v252: 97 pgs: 97 active+clean; 471 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 287 op/s 2024-01-31T05:30:45.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:44 smithi186 ceph-mon[35659]: pgmap v252: 97 pgs: 97 active+clean; 471 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 287 op/s 2024-01-31T05:30:45.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 856808 11.21 MB/sec execute 285 sec latency 59.857 ms 2024-01-31T05:30:46.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 859277 11.23 MB/sec execute 286 sec latency 22.681 ms 2024-01-31T05:30:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:46 smithi045 ceph-mon[31585]: pgmap v253: 97 pgs: 97 active+clean; 471 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 189 op/s 2024-01-31T05:30:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:46 smithi156 ceph-mon[35720]: pgmap v253: 97 pgs: 97 active+clean; 471 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 189 op/s 2024-01-31T05:30:47.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:46 smithi186 ceph-mon[35659]: pgmap v253: 97 pgs: 97 active+clean; 471 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 189 op/s 2024-01-31T05:30:47.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 860891 11.21 MB/sec execute 287 sec latency 14.977 ms 2024-01-31T05:30:48.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 863072 11.22 MB/sec execute 288 sec latency 23.452 ms 2024-01-31T05:30:48.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:48 smithi045 ceph-mon[31585]: pgmap v254: 97 pgs: 97 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 287 op/s 2024-01-31T05:30:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:48 smithi156 ceph-mon[35720]: pgmap v254: 97 pgs: 97 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 287 op/s 2024-01-31T05:30:49.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:48 smithi186 ceph-mon[35659]: pgmap v254: 97 pgs: 97 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 287 op/s 2024-01-31T05:30:49.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 864739 11.20 MB/sec execute 289 sec latency 73.103 ms 2024-01-31T05:30:50.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 866732 11.21 MB/sec execute 290 sec latency 61.511 ms 2024-01-31T05:30:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:50 smithi045 ceph-mon[31585]: pgmap v255: 97 pgs: 97 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 196 op/s 2024-01-31T05:30:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:50 smithi156 ceph-mon[35720]: pgmap v255: 97 pgs: 97 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 196 op/s 2024-01-31T05:30:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:50 smithi186 ceph-mon[35659]: pgmap v255: 97 pgs: 97 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 196 op/s 2024-01-31T05:30:51.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 868851 11.20 MB/sec execute 291 sec latency 28.356 ms 2024-01-31T05:30:52.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 870744 11.20 MB/sec execute 292 sec latency 20.291 ms 2024-01-31T05:30:52.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:52 smithi045 ceph-mon[31585]: pgmap v256: 97 pgs: 97 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 196 op/s 2024-01-31T05:30:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:52 smithi156 ceph-mon[35720]: pgmap v256: 97 pgs: 97 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 196 op/s 2024-01-31T05:30:53.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:52 smithi186 ceph-mon[35659]: pgmap v256: 97 pgs: 97 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 196 op/s 2024-01-31T05:30:53.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 873327 11.22 MB/sec execute 293 sec latency 17.072 ms 2024-01-31T05:30:53.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:53 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:30:53.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:53 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:30:54.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:53 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:30:54.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 874809 11.20 MB/sec execute 294 sec latency 78.596 ms 2024-01-31T05:30:54.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:54 smithi045 ceph-mon[31585]: pgmap v257: 97 pgs: 97 active+clean; 488 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 290 op/s 2024-01-31T05:30:54.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:54 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:30:54.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:54 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:30:54.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:54 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:30:54.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:54 smithi156 ceph-mon[35720]: pgmap v257: 97 pgs: 97 active+clean; 488 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 290 op/s 2024-01-31T05:30:54.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:54 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:30:54.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:54 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:30:54.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:54 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:30:55.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:54 smithi186 ceph-mon[35659]: pgmap v257: 97 pgs: 97 active+clean; 488 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 290 op/s 2024-01-31T05:30:55.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:54 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:30:55.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:54 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:30:55.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:54 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:30:55.790 INFO:tasks.workunit.client.0.smithi045.stdout: 1 876937 11.21 MB/sec execute 295 sec latency 76.253 ms 2024-01-31T05:30:56.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 878506 11.19 MB/sec execute 296 sec latency 20.009 ms 2024-01-31T05:30:56.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:56 smithi045 ceph-mon[31585]: pgmap v258: 97 pgs: 97 active+clean; 488 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 192 op/s 2024-01-31T05:30:56.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:56 smithi156 ceph-mon[35720]: pgmap v258: 97 pgs: 97 active+clean; 488 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 192 op/s 2024-01-31T05:30:57.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:56 smithi186 ceph-mon[35659]: pgmap v258: 97 pgs: 97 active+clean; 488 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 192 op/s 2024-01-31T05:30:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.504+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3858120710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c100800 msgr2=0x7f2b3c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.504+0000 7f2b40fc9640 1 --2- 172.21.15.45:0/3858120710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c100800 0x7f2b3c10ca20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009ca0 tx=0x7f2b3402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.505+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3858120710 shutdown_connections 2024-01-31T05:30:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.505+0000 7f2b40fc9640 1 --2- 172.21.15.45:0/3858120710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c100800 0x7f2b3c10ca20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.505+0000 7f2b40fc9640 1 --2- 172.21.15.45:0/3858120710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b3c0ffe40 0x7f2b3c1002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.505+0000 7f2b40fc9640 1 --2- 172.21.15.45:0/3858120710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b3c104b90 0x7f2b3c104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.505+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3858120710 >> 172.21.15.45:0/3858120710 conn(0x7f2b3c076a40 msgr2=0x7f2b3c076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.505+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3858120710 shutdown_connections 2024-01-31T05:30:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.505+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3858120710 wait complete. 2024-01-31T05:30:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.505+0000 7f2b40fc9640 1 Processor -- start 2024-01-31T05:30:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.505+0000 7f2b40fc9640 1 -- start start 2024-01-31T05:30:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b40fc9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b3c0ffe40 0x7f2b3c196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b40fc9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b3c100800 0x7f2b3c1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b40fc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c104b90 0x7f2b3c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b40fc9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c19af50 con 0x7f2b3c104b90 2024-01-31T05:30:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b40fc9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c19b0c0 con 0x7f2b3c0ffe40 2024-01-31T05:30:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b40fc9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c19b280 con 0x7f2b3c100800 2024-01-31T05:30:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b3ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c104b90 0x7f2b3c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b3ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c104b90 0x7f2b3c19a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60234/0 (socket says 172.21.15.45:60234) 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b3ad9d640 1 -- 172.21.15.45:0/3941108834 learned_addr learned my addr 172.21.15.45:0/3941108834 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b3c0ffe40 0x7f2b3c196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b33fff640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b3c100800 0x7f2b3c1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b3ad9d640 1 -- 172.21.15.45:0/3941108834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b3c100800 msgr2=0x7f2b3c1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b3ad9d640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b3c100800 0x7f2b3c1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b3ad9d640 1 -- 172.21.15.45:0/3941108834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b3c0ffe40 msgr2=0x7f2b3c196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b3ad9d640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b3c0ffe40 0x7f2b3c196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b3ad9d640 1 -- 172.21.15.45:0/3941108834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b34009900 con 0x7f2b3c104b90 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b3ad9d640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c104b90 0x7f2b3c19a950 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2b3402c700 tx=0x7f2b34032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.506+0000 7f2b337fe640 1 -- 172.21.15.45:0/3941108834 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b34032800 con 0x7f2b3c104b90 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.507+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3941108834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3c19b500 con 0x7f2b3c104b90 2024-01-31T05:30:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.507+0000 7f2b337fe640 1 -- 172.21.15.45:0/3941108834 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b34003e30 con 0x7f2b3c104b90 2024-01-31T05:30:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.507+0000 7f2b337fe640 1 -- 172.21.15.45:0/3941108834 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b340047f0 con 0x7f2b3c104b90 2024-01-31T05:30:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.507+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3941108834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3c19b7d0 con 0x7f2b3c104b90 2024-01-31T05:30:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.509+0000 7f2b43d92740 1 Processor -- start 2024-01-31T05:30:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.509+0000 7f2b337fe640 1 -- 172.21.15.45:0/3941108834 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f2b3403b040 con 0x7f2b3c104b90 2024-01-31T05:30:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b43d92740 1 -- start start 2024-01-31T05:30:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b43d92740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 0x557141005440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b43d92740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x55714100e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b43d92740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 0x557141010b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b43d92740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557140f74040 con 0x557141005060 2024-01-31T05:30:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b43d92740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557140f71730 con 0x55714100e750 2024-01-31T05:30:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b43d92740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557140f6d020 con 0x557141005980 2024-01-31T05:30:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b3a59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x55714100e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b3a59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x55714100e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:36398/0 (socket says 172.21.15.45:36398) 2024-01-31T05:30:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b3a59c640 1 -- 172.21.15.45:0/3082735561 learned_addr learned my addr 172.21.15.45:0/3082735561 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b33fff640 1 --2- 172.21.15.45:0/3082735561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 0x557141005440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b337fe640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2b280a14f0 0x7f2b280a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b3a59c640 1 -- 172.21.15.45:0/3082735561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 msgr2=0x557141010b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/3082735561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 0x557141010b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b3a59c640 1 -- 172.21.15.45:0/3082735561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 msgr2=0x557141005440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/3082735561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 0x557141005440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b3a59c640 1 -- 172.21.15.45:0/3082735561 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557140f64e40 con 0x557141005980 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b33fff640 1 --2- 172.21.15.45:0/3082735561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 0x557141005440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b3ad9d640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2b280a14f0 0x7f2b280a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b337fe640 1 -- 172.21.15.45:0/3941108834 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2b340ef6f0 con 0x7f2b3c104b90 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.510+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/3082735561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x55714100e210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009ca0 tx=0x7f2b2c0119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45d2b7e90306b292 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.511+0000 7f2b30ff9640 1 -- 172.21.15.45:0/3082735561 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c016070 con 0x557141005980 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.511+0000 7f2b3ad9d640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2b280a14f0 0x7f2b280a39b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2b340ed460 tx=0x7f2b3408e680 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.511+0000 7f2b30ff9640 1 -- 172.21.15.45:0/3082735561 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b2c01a8d0 con 0x557141005980 2024-01-31T05:30:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.511+0000 7f2b30ff9640 1 -- 172.21.15.45:0/3082735561 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c00e910 con 0x557141005980 2024-01-31T05:30:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.511+0000 7f2b43d92740 1 -- 172.21.15.45:0/3082735561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 msgr2=0x55714100e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.511+0000 7f2b43d92740 1 --2- 172.21.15.45:0/3082735561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x55714100e210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009ca0 tx=0x7f2b2c0119a0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.511+0000 7f2b43d92740 1 -- 172.21.15.45:0/3082735561 shutdown_connections 2024-01-31T05:30:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.511+0000 7f2b43d92740 1 --2- 172.21.15.45:0/3082735561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 0x557141010b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.511+0000 7f2b43d92740 1 --2- 172.21.15.45:0/3082735561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x55714100e210 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 1 --2- 172.21.15.45:0/3082735561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 0x557141005440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 1 -- 172.21.15.45:0/3082735561 >> 172.21.15.45:0/3082735561 conn(0x557140fff980 msgr2=0x557140fffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 1 -- 172.21.15.45:0/3082735561 shutdown_connections 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 1 -- 172.21.15.45:0/3082735561 wait complete. 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 1 -- start start 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 1 Processor -- start 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 0x55714100b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x55714100bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 0x55714100c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557140f261b0 con 0x557141005060 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557140f25e50 con 0x55714100e750 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557140eeb090 con 0x557141005980 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b43d92740 10 client.0 _ll_register_callbacks cb 0x557140f10a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b33fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 0x55714100b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b3a59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x55714100bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b33fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 0x55714100b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60252/0 (socket says 172.21.15.45:60252) 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.512+0000 7f2b33fff640 1 -- 172.21.15.45:0/596698898 learned_addr learned my addr 172.21.15.45:0/596698898 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b33fff640 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 0x55714100c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b3a59c640 1 -- 172.21.15.45:0/596698898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 msgr2=0x55714100c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 0x55714100c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b3a59c640 1 -- 172.21.15.45:0/596698898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 msgr2=0x55714100b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 0x55714100b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b3a59c640 1 -- 172.21.15.45:0/596698898 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c009950 con 0x557141005980 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b33fff640 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 0x55714100b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x55714100bd10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009c70 tx=0x7f2b2c01ac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9009a307913855f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b21ffb640 1 -- 172.21.15.45:0/596698898 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c016070 con 0x557141005980 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b43d92740 1 -- 172.21.15.45:0/596698898 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557140f66010 con 0x557141005980 2024-01-31T05:30:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b21ffb640 1 -- 172.21.15.45:0/596698898 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2b2c004240 con 0x557141005980 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b21ffb640 1 -- 172.21.15.45:0/596698898 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b2c01ade0 con 0x557141005980 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b43d92740 10 client.25618 fetch_fsmap learned FSMap version 28 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.513+0000 7f2b43d92740 1 -- 172.21.15.45:0/596698898 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557140f6b6a0 con 0x557141005980 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.514+0000 7f2b21ffb640 1 -- 172.21.15.45:0/596698898 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b2c030020 con 0x557141005980 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.514+0000 7f2b21ffb640 1 -- 172.21.15.45:0/596698898 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2b2c0049c0 con 0x557141005980 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.514+0000 7f2b43d92740 10 client.25618 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.514+0000 7f2b43d92740 10 client.25618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.514+0000 7f2b43d92740 20 client.25618 populate_metadata read hostname 'smithi045' 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.514+0000 7f2b43d92740 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55714101e480 0x557141020870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.514+0000 7f2b43d92740 4 client.25618 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.514+0000 7f2b43d92740 1 -- 172.21.15.45:0/596698898 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557140f65d30 con 0x55714101e480 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.514+0000 7f2b33fff640 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55714101e480 0x557141020870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.514+0000 7f2b33fff640 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55714101e480 0x557141020870 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bae5e55faaecc6 server_cookie=ca1c007bc05ab600 in_seq=0 out_seq=0 2024-01-31T05:30:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.514+0000 7f2b21ffb640 10 client.25618 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b21ffb640 1 -- 172.21.15.45:0/596698898 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557140f65d30 con 0x55714101e480 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b21ffb640 10 client.25618 handle_command_reply: tid=0 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b43d92740 1 client.25618 shutdown 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b43d92740 1 -- 172.21.15.45:0/596698898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 msgr2=0x55714100bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b43d92740 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x55714100bd10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009c70 tx=0x7f2b2c01ac70 comp rx=0 tx=0).stop 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b43d92740 1 -- 172.21.15.45:0/596698898 shutdown_connections 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b43d92740 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55714101e480 0x557141020870 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b43d92740 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 0x55714100c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b43d92740 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x55714100bd10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b21ffb640 0 client.25618 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b43d92740 1 --2- 172.21.15.45:0/596698898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005060 0x55714100b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b43d92740 1 -- 172.21.15.45:0/596698898 >> 172.21.15.45:0/596698898 conn(0x557140fff980 msgr2=0x5571410016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b43d92740 1 -- 172.21.15.45:0/596698898 shutdown_connections 2024-01-31T05:30:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b43d92740 1 -- 172.21.15.45:0/596698898 wait complete. 2024-01-31T05:30:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.515+0000 7f2b43d92740 20 client.25618 trim_cache size 0 max 16384 2024-01-31T05:30:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.529+0000 7f2b43d92740 1 Processor -- start 2024-01-31T05:30:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b43d92740 1 -- start start 2024-01-31T05:30:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b43d92740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005060 0x55714100df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b43d92740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005980 0x55714100c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b43d92740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 0x55714100a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b43d92740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714101e280 con 0x557141005980 2024-01-31T05:30:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b43d92740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557141000980 con 0x55714100e750 2024-01-31T05:30:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b43d92740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557141025a30 con 0x557141005060 2024-01-31T05:30:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b3a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005980 0x55714100c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b3a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005980 0x55714100c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60268/0 (socket says 172.21.15.45:60268) 2024-01-31T05:30:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b3a59c640 1 -- 172.21.15.45:0/1691087574 learned_addr learned my addr 172.21.15.45:0/1691087574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b33fff640 1 --2- 172.21.15.45:0/1691087574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005060 0x55714100df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b33fff640 1 --2- 172.21.15.45:0/1691087574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 0x55714100a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b3a59c640 1 -- 172.21.15.45:0/1691087574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005060 msgr2=0x55714100df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/1691087574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005060 0x55714100df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b3a59c640 1 -- 172.21.15.45:0/1691087574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 msgr2=0x55714100a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/1691087574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 0x55714100a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b3a59c640 1 -- 172.21.15.45:0/1691087574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557140f08030 con 0x557141005980 2024-01-31T05:30:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/1691087574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005980 0x55714100c070 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0058c0 tx=0x7f2b2c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cea5deee9d22c0d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b23fff640 1 -- 172.21.15.45:0/1691087574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c016040 con 0x557141005980 2024-01-31T05:30:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b23fff640 1 -- 172.21.15.45:0/1691087574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b2c030070 con 0x557141005980 2024-01-31T05:30:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.530+0000 7f2b23fff640 1 -- 172.21.15.45:0/1691087574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c004ef0 con 0x557141005980 2024-01-31T05:30:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.531+0000 7f2b43d92740 1 -- 172.21.15.45:0/1691087574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005980 msgr2=0x55714100c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.531+0000 7f2b43d92740 1 --2- 172.21.15.45:0/1691087574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005980 0x55714100c070 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0058c0 tx=0x7f2b2c009620 comp rx=0 tx=0).stop 2024-01-31T05:30:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.531+0000 7f2b43d92740 1 -- 172.21.15.45:0/1691087574 shutdown_connections 2024-01-31T05:30:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.531+0000 7f2b43d92740 1 --2- 172.21.15.45:0/1691087574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557141005980 0x55714100c070 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.531+0000 7f2b43d92740 1 --2- 172.21.15.45:0/1691087574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55714100e750 0x55714100a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.531+0000 7f2b43d92740 1 --2- 172.21.15.45:0/1691087574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005060 0x55714100df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.531+0000 7f2b43d92740 1 -- 172.21.15.45:0/1691087574 >> 172.21.15.45:0/1691087574 conn(0x557140fff980 msgr2=0x557141006a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.531+0000 7f2b43d92740 1 -- 172.21.15.45:0/1691087574 shutdown_connections 2024-01-31T05:30:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.531+0000 7f2b43d92740 1 -- 172.21.15.45:0/1691087574 wait complete. 2024-01-31T05:30:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.531+0000 7f2b43d92740 1 -- start start 2024-01-31T05:30:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.531+0000 7f2b43d92740 1 Processor -- start 2024-01-31T05:30:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b43d92740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557141005060 0x5571410153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b43d92740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x5571410158e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b43d92740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55714100e750 0x5571410138d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b43d92740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557141013fa0 con 0x55714100e750 2024-01-31T05:30:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b43d92740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557141014110 con 0x557141005060 2024-01-31T05:30:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b43d92740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571410142d0 con 0x557141005980 2024-01-31T05:30:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b43d92740 10 client.0 _ll_register_callbacks cb 0x7f2b2c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:30:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b33fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55714100e750 0x5571410138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b33fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55714100e750 0x5571410138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60276/0 (socket says 172.21.15.45:60276) 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b33fff640 1 -- 172.21.15.45:0/3556735174 learned_addr learned my addr 172.21.15.45:0/3556735174 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x5571410158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b33fff640 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557141005060 0x5571410153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b33fff640 1 -- 172.21.15.45:0/3556735174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 msgr2=0x5571410158e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b33fff640 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x5571410158e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b33fff640 1 -- 172.21.15.45:0/3556735174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557141005060 msgr2=0x5571410153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b33fff640 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557141005060 0x5571410153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b33fff640 1 -- 172.21.15.45:0/3556735174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557140f6b6a0 con 0x55714100e750 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b33fff640 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55714100e750 0x5571410138d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2b24014990 tx=0x7f2b2c01a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23391904e64f303a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b21ffb640 1 -- 172.21.15.45:0/3556735174 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24004020 con 0x55714100e750 2024-01-31T05:30:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b21ffb640 1 -- 172.21.15.45:0/3556735174 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2b24004180 con 0x55714100e750 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.532+0000 7f2b43d92740 1 -- 172.21.15.45:0/3556735174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557140ffb080 con 0x55714100e750 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.533+0000 7f2b21ffb640 1 -- 172.21.15.45:0/3556735174 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b24004340 con 0x55714100e750 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.533+0000 7f2b43d92740 10 client.16068 fetch_fsmap learned FSMap version 28 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.533+0000 7f2b43d92740 1 -- 172.21.15.45:0/3556735174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557140f2da50 con 0x55714100e750 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.533+0000 7f2b21ffb640 1 -- 172.21.15.45:0/3556735174 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b24033020 con 0x55714100e750 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.533+0000 7f2b21ffb640 1 -- 172.21.15.45:0/3556735174 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2b240037b0 con 0x55714100e750 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.533+0000 7f2b43d92740 10 client.16068 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.533+0000 7f2b43d92740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.533+0000 7f2b43d92740 20 client.16068 populate_metadata read hostname 'smithi045' 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.533+0000 7f2b43d92740 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55714107a050 0x55714107c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.533+0000 7f2b43d92740 4 client.16068 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.533+0000 7f2b43d92740 1 -- 172.21.15.45:0/3556735174 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f2b2c01a8d0 con 0x55714107a050 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.533+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55714107a050 0x55714107c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:30:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.534+0000 7f2b3a59c640 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55714107a050 0x55714107c470 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ce8e02386369dd server_cookie=c6a5483c7fc1333d in_seq=0 out_seq=0 2024-01-31T05:30:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.534+0000 7f2b21ffb640 10 client.16068 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.544+0000 7f2b21ffb640 1 -- 172.21.15.45:0/3556735174 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f2b2c01a8d0 con 0x55714107a050 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.544+0000 7f2b21ffb640 10 client.16068 handle_command_reply: tid=0 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.544+0000 7f2b43d92740 1 client.16068 shutdown 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b43d92740 1 -- 172.21.15.45:0/3556735174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55714100e750 msgr2=0x5571410138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b43d92740 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55714100e750 0x5571410138d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2b24014990 tx=0x7f2b2c01a740 comp rx=0 tx=0).stop 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b43d92740 1 -- 172.21.15.45:0/3556735174 shutdown_connections 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b43d92740 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55714107a050 0x55714107c470 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b43d92740 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55714100e750 0x5571410138d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b43d92740 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557141005980 0x5571410158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b21ffb640 0 client.16068 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b43d92740 1 --2- 172.21.15.45:0/3556735174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557141005060 0x5571410153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b43d92740 1 -- 172.21.15.45:0/3556735174 >> 172.21.15.45:0/3556735174 conn(0x557140fff980 msgr2=0x557141009fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b43d92740 1 -- 172.21.15.45:0/3556735174 shutdown_connections 2024-01-31T05:30:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b43d92740 1 -- 172.21.15.45:0/3556735174 wait complete. 2024-01-31T05:30:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b43d92740 20 client.16068 trim_cache size 0 max 16384 2024-01-31T05:30:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:30:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:30:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:30:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:30:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3941108834 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2b280a14f0 msgr2=0x7f2b280a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b40fc9640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2b280a14f0 0x7f2b280a39b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2b340ed460 tx=0x7f2b3408e680 comp rx=0 tx=0).stop 2024-01-31T05:30:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3941108834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c104b90 msgr2=0x7f2b3c19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:30:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b40fc9640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c104b90 0x7f2b3c19a950 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2b3402c700 tx=0x7f2b34032fc0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3941108834 shutdown_connections 2024-01-31T05:30:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b40fc9640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2b280a14f0 0x7f2b280a39b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.545+0000 7f2b40fc9640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c104b90 0x7f2b3c19a950 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.546+0000 7f2b40fc9640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b3c100800 0x7f2b3c1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.546+0000 7f2b40fc9640 1 --2- 172.21.15.45:0/3941108834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b3c0ffe40 0x7f2b3c196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:30:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.546+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3941108834 >> 172.21.15.45:0/3941108834 conn(0x7f2b3c076a40 msgr2=0x7f2b3c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:30:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.546+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3941108834 shutdown_connections 2024-01-31T05:30:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:30:57.546+0000 7f2b40fc9640 1 -- 172.21.15.45:0/3941108834 wait complete. 2024-01-31T05:30:57.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 880799 11.20 MB/sec execute 297 sec latency 14.628 ms 2024-01-31T05:30:58.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 882906 11.19 MB/sec execute 298 sec latency 18.265 ms 2024-01-31T05:30:58.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:30:58 smithi045 ceph-mon[31585]: pgmap v259: 97 pgs: 97 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.0 MiB/s wr, 288 op/s 2024-01-31T05:30:58.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:30:58 smithi186 ceph-mon[35659]: pgmap v259: 97 pgs: 97 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.0 MiB/s wr, 288 op/s 2024-01-31T05:30:58.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:30:58 smithi156 ceph-mon[35720]: pgmap v259: 97 pgs: 97 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.0 MiB/s wr, 288 op/s 2024-01-31T05:30:59.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 884622 11.19 MB/sec execute 299 sec latency 75.832 ms 2024-01-31T05:31:00.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 886577 11.18 MB/sec execute 300 sec latency 77.276 ms 2024-01-31T05:31:00.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:00 smithi045 ceph-mon[31585]: pgmap v260: 97 pgs: 97 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-01-31T05:31:00.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:00 smithi156 ceph-mon[35720]: pgmap v260: 97 pgs: 97 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-01-31T05:31:01.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:00 smithi186 ceph-mon[35659]: pgmap v260: 97 pgs: 97 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-01-31T05:31:01.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 888374 11.18 MB/sec execute 301 sec latency 34.871 ms 2024-01-31T05:31:02.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 890922 11.19 MB/sec execute 302 sec latency 13.873 ms 2024-01-31T05:31:02.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:02 smithi045 ceph-mon[31585]: pgmap v261: 97 pgs: 97 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-01-31T05:31:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:02 smithi156 ceph-mon[35720]: pgmap v261: 97 pgs: 97 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-01-31T05:31:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:02 smithi186 ceph-mon[35659]: pgmap v261: 97 pgs: 97 active+clean; 497 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-01-31T05:31:03.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 892581 11.19 MB/sec execute 303 sec latency 16.830 ms 2024-01-31T05:31:04.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 894933 11.20 MB/sec execute 304 sec latency 33.132 ms 2024-01-31T05:31:04.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:04 smithi045 ceph-mon[31585]: pgmap v262: 97 pgs: 97 active+clean; 506 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 305 op/s 2024-01-31T05:31:04.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:04 smithi156 ceph-mon[35720]: pgmap v262: 97 pgs: 97 active+clean; 506 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 305 op/s 2024-01-31T05:31:05.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:04 smithi186 ceph-mon[35659]: pgmap v262: 97 pgs: 97 active+clean; 506 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 305 op/s 2024-01-31T05:31:05.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 896572 11.18 MB/sec execute 305 sec latency 66.877 ms 2024-01-31T05:31:06.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 898359 11.18 MB/sec execute 306 sec latency 110.915 ms 2024-01-31T05:31:06.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:06 smithi045 ceph-mon[31585]: pgmap v263: 97 pgs: 97 active+clean; 506 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 210 op/s 2024-01-31T05:31:06.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:06 smithi156 ceph-mon[35720]: pgmap v263: 97 pgs: 97 active+clean; 506 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 210 op/s 2024-01-31T05:31:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:06 smithi186 ceph-mon[35659]: pgmap v263: 97 pgs: 97 active+clean; 506 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 210 op/s 2024-01-31T05:31:07.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 899894 11.17 MB/sec execute 307 sec latency 28.641 ms 2024-01-31T05:31:08.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 901985 11.17 MB/sec execute 308 sec latency 15.150 ms 2024-01-31T05:31:08.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:08 smithi045 ceph-mon[31585]: pgmap v264: 97 pgs: 97 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.2 MiB/s wr, 306 op/s 2024-01-31T05:31:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:08 smithi156 ceph-mon[35720]: pgmap v264: 97 pgs: 97 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.2 MiB/s wr, 306 op/s 2024-01-31T05:31:09.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:08 smithi186 ceph-mon[35659]: pgmap v264: 97 pgs: 97 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.2 MiB/s wr, 306 op/s 2024-01-31T05:31:09.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 903840 11.16 MB/sec execute 309 sec latency 21.559 ms 2024-01-31T05:31:09.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:09 smithi045 ceph-mon[31585]: pgmap v265: 97 pgs: 97 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 209 op/s 2024-01-31T05:31:09.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:09 smithi156 ceph-mon[35720]: pgmap v265: 97 pgs: 97 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 209 op/s 2024-01-31T05:31:10.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:09 smithi186 ceph-mon[35659]: pgmap v265: 97 pgs: 97 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 209 op/s 2024-01-31T05:31:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.645+0000 7fd6a3099640 1 -- 172.21.15.45:0/2908165236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c11a4d0 msgr2=0x7fd69c11c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.645+0000 7fd6a3099640 1 --2- 172.21.15.45:0/2908165236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c11a4d0 0x7fd69c11c8c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd694009ca0 tx=0x7fd69402c140 comp rx=0 tx=0).stop 2024-01-31T05:31:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.646+0000 7fd6a3099640 1 -- 172.21.15.45:0/2908165236 shutdown_connections 2024-01-31T05:31:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.646+0000 7fd6a3099640 1 --2- 172.21.15.45:0/2908165236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c11a4d0 0x7fd69c11c8c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.646+0000 7fd6a3099640 1 --2- 172.21.15.45:0/2908165236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c10f8a0 0x7fd69c10d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.646+0000 7fd6a3099640 1 --2- 172.21.15.45:0/2908165236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd69c10f430 0x7fd69c10d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.646+0000 7fd6a3099640 1 -- 172.21.15.45:0/2908165236 >> 172.21.15.45:0/2908165236 conn(0x7fd69c06bd90 msgr2=0x7fd69c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.647+0000 7fd6a3099640 1 -- 172.21.15.45:0/2908165236 shutdown_connections 2024-01-31T05:31:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.647+0000 7fd6a3099640 1 -- 172.21.15.45:0/2908165236 wait complete. 2024-01-31T05:31:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd6a3099640 1 Processor -- start 2024-01-31T05:31:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd6a3099640 1 -- start start 2024-01-31T05:31:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd6a3099640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c10f430 0x7fd69c1a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd6a3099640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd69c10f8a0 0x7fd69c1a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd6a3099640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c11a4d0 0x7fd69c1aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd6a3099640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c1aafb0 con 0x7fd69c10f8a0 2024-01-31T05:31:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd6a3099640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c1ab120 con 0x7fd69c10f430 2024-01-31T05:31:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd6a3099640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c1ab2e0 con 0x7fd69c11a4d0 2024-01-31T05:31:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd6a1636640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c11a4d0 0x7fd69c1aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd6a1636640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c11a4d0 0x7fd69c1aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:42278/0 (socket says 172.21.15.45:42278) 2024-01-31T05:31:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd6a1636640 1 -- 172.21.15.45:0/3935298793 learned_addr learned my addr 172.21.15.45:0/3935298793 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd69bfff640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd69c10f8a0 0x7fd69c1a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.648+0000 7fd6a0e35640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c10f430 0x7fd69c1a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.649+0000 7fd6a1636640 1 -- 172.21.15.45:0/3935298793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c10f430 msgr2=0x7fd69c1a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.649+0000 7fd6a1636640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c10f430 0x7fd69c1a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.649+0000 7fd6a1636640 1 -- 172.21.15.45:0/3935298793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd69c10f8a0 msgr2=0x7fd69c1a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.649+0000 7fd6a1636640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd69c10f8a0 0x7fd69c1a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.649+0000 7fd6a1636640 1 -- 172.21.15.45:0/3935298793 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd694009900 con 0x7fd69c11a4d0 2024-01-31T05:31:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.650+0000 7fd6a1636640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c11a4d0 0x7fd69c1aa9b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd694009bf0 tx=0x7fd694032980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.651+0000 7fd699ffb640 1 -- 172.21.15.45:0/3935298793 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694034850 con 0x7fd69c11a4d0 2024-01-31T05:31:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.651+0000 7fd699ffb640 1 -- 172.21.15.45:0/3935298793 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd694003e30 con 0x7fd69c11a4d0 2024-01-31T05:31:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.651+0000 7fd699ffb640 1 -- 172.21.15.45:0/3935298793 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694040450 con 0x7fd69c11a4d0 2024-01-31T05:31:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.651+0000 7fd6a3099640 1 -- 172.21.15.45:0/3935298793 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c1ab560 con 0x7fd69c11a4d0 2024-01-31T05:31:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.651+0000 7fd6a3099640 1 -- 172.21.15.45:0/3935298793 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69c10fcb0 con 0x7fd69c11a4d0 2024-01-31T05:31:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.656+0000 7fd699ffb640 1 -- 172.21.15.45:0/3935298793 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fd69400a040 con 0x7fd69c11a4d0 2024-01-31T05:31:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.656+0000 7fd6a5e59740 1 Processor -- start 2024-01-31T05:31:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.656+0000 7fd6a5e59740 1 -- start start 2024-01-31T05:31:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.656+0000 7fd6a5e59740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1060 0x560c8e4e1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.656+0000 7fd6a5e59740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 0x560c8e4ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.657+0000 7fd6a5e59740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4ea750 0x560c8e4ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.657+0000 7fd6a5e59740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8e450040 con 0x560c8e4e1980 2024-01-31T05:31:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.657+0000 7fd6a5e59740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8e44d730 con 0x560c8e4e1060 2024-01-31T05:31:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.657+0000 7fd6a5e59740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8e449020 con 0x560c8e4ea750 2024-01-31T05:31:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.657+0000 7fd699ffb640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd68c0a1720 0x7fd68c0a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.657+0000 7fd699ffb640 1 -- 172.21.15.45:0/3935298793 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd6940efbd0 con 0x7fd69c11a4d0 2024-01-31T05:31:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.658+0000 7fd6a1636640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd68c0a1720 0x7fd68c0a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.658+0000 7fd6a0e35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 0x560c8e4ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd6a0e35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 0x560c8e4ea210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45170/0 (socket says 172.21.15.45:45170) 2024-01-31T05:31:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd6a0e35640 1 -- 172.21.15.45:0/2222905901 learned_addr learned my addr 172.21.15.45:0/2222905901 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd69bfff640 1 --2- 172.21.15.45:0/2222905901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4ea750 0x560c8e4ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd6a1636640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd68c0a1720 0x7fd68c0a3be0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd6940bc9e0 tx=0x7fd69402c140 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd69bfff640 1 --2- 172.21.15.45:0/2222905901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1060 0x560c8e4e1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd6a0e35640 1 -- 172.21.15.45:0/2222905901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4ea750 msgr2=0x560c8e4ecb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd6a0e35640 1 --2- 172.21.15.45:0/2222905901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4ea750 0x560c8e4ecb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd6a0e35640 1 -- 172.21.15.45:0/2222905901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1060 msgr2=0x560c8e4e1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd6a0e35640 1 --2- 172.21.15.45:0/2222905901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1060 0x560c8e4e1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd6a0e35640 1 -- 172.21.15.45:0/2222905901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c8e440e40 con 0x560c8e4e1980 2024-01-31T05:31:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd69bfff640 1 --2- 172.21.15.45:0/2222905901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1060 0x560c8e4e1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:31:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd6a0e35640 1 --2- 172.21.15.45:0/2222905901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 0x560c8e4ea210 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd688009a40 tx=0x7fd688009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69050e23f1b4fb6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.659+0000 7fd686ffd640 1 -- 172.21.15.45:0/2222905901 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688009540 con 0x560c8e4e1980 2024-01-31T05:31:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.660+0000 7fd686ffd640 1 -- 172.21.15.45:0/2222905901 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd688013470 con 0x560c8e4e1980 2024-01-31T05:31:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.660+0000 7fd686ffd640 1 -- 172.21.15.45:0/2222905901 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688009540 con 0x560c8e4e1980 2024-01-31T05:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.660+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2222905901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 msgr2=0x560c8e4ea210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.660+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/2222905901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 0x560c8e4ea210 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd688009a40 tx=0x7fd688009f00 comp rx=0 tx=0).stop 2024-01-31T05:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.661+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2222905901 shutdown_connections 2024-01-31T05:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.661+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/2222905901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 0x560c8e4ea210 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.661+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/2222905901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4ea750 0x560c8e4ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.661+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/2222905901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1060 0x560c8e4e1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.661+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2222905901 >> 172.21.15.45:0/2222905901 conn(0x560c8e4db980 msgr2=0x560c8e4dbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.661+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2222905901 shutdown_connections 2024-01-31T05:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.661+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2222905901 wait complete. 2024-01-31T05:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.661+0000 7fd6a5e59740 1 -- start start 2024-01-31T05:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.661+0000 7fd6a5e59740 1 Processor -- start 2024-01-31T05:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.662+0000 7fd6a5e59740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4e1060 0x560c8e4e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.662+0000 7fd6a5e59740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1980 0x560c8e4e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.662+0000 7fd6a5e59740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4ea750 0x560c8e4e8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.662+0000 7fd6a5e59740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8e4021b0 con 0x560c8e4ea750 2024-01-31T05:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.662+0000 7fd6a5e59740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8e401e50 con 0x560c8e4e1980 2024-01-31T05:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.662+0000 7fd6a5e59740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8e3c7090 con 0x560c8e4e1060 2024-01-31T05:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.662+0000 7fd6a5e59740 10 client.0 _ll_register_callbacks cb 0x560c8e3eca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.662+0000 7fd69bfff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4e1060 0x560c8e4e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.662+0000 7fd69bfff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4e1060 0x560c8e4e76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:42306/0 (socket says 172.21.15.45:42306) 2024-01-31T05:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.662+0000 7fd69bfff640 1 -- 172.21.15.45:0/2197165673 learned_addr learned my addr 172.21.15.45:0/2197165673 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.662+0000 7fd69bfff640 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4ea750 0x560c8e4e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.662+0000 7fd6a0e35640 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1980 0x560c8e4e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd69bfff640 1 -- 172.21.15.45:0/2197165673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1980 msgr2=0x560c8e4e7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd69bfff640 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1980 0x560c8e4e7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd69bfff640 1 -- 172.21.15.45:0/2197165673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4ea750 msgr2=0x560c8e4e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd69bfff640 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4ea750 0x560c8e4e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd69bfff640 1 -- 172.21.15.45:0/2197165673 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6880096a0 con 0x560c8e4e1060 2024-01-31T05:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd69bfff640 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4e1060 0x560c8e4e76c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd690029e40 tx=0x7fd690004700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90028dec6f5f6b20 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd67bfff640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd67bfff640 1 -- 172.21.15.45:0/2197165673 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690022d70 con 0x560c8e4e1060 2024-01-31T05:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2197165673 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c8e442010 con 0x560c8e4e1060 2024-01-31T05:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd67bfff640 1 -- 172.21.15.45:0/2197165673 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd690026460 con 0x560c8e4e1060 2024-01-31T05:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd67bfff640 1 -- 172.21.15.45:0/2197165673 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd690022ed0 con 0x560c8e4e1060 2024-01-31T05:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd6a5e59740 10 client.25636 fetch_fsmap learned FSMap version 28 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2197165673 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560c8e4476a0 con 0x560c8e4e1060 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.663+0000 7fd67bfff640 1 -- 172.21.15.45:0/2197165673 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c8e442010 con 0x560c8e4e1060 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.664+0000 7fd67bfff640 1 -- 172.21.15.45:0/2197165673 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd690031020 con 0x560c8e4e1060 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.664+0000 7fd6a5e59740 10 client.25636 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.664+0000 7fd6a5e59740 10 client.25636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.664+0000 7fd6a5e59740 20 client.25636 populate_metadata read hostname 'smithi045' 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.664+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560c8e4fa510 0x560c8e4fc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.664+0000 7fd6a5e59740 4 client.25636 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.664+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2197165673 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c8e441d30 con 0x560c8e4fa510 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.664+0000 7fd6a0e35640 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560c8e4fa510 0x560c8e4fc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.664+0000 7fd67bfff640 10 client.25636 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.664+0000 7fd6a0e35640 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560c8e4fa510 0x560c8e4fc930 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f25efd9a2b3ee743 server_cookie=7b91c4404f918324 in_seq=0 out_seq=0 2024-01-31T05:31:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd67bfff640 1 -- 172.21.15.45:0/2197165673 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c8e441d30 con 0x560c8e4fa510 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd67bfff640 10 client.25636 handle_command_reply: tid=0 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd6a5e59740 1 client.25636 shutdown 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2197165673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4e1060 msgr2=0x560c8e4e76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4e1060 0x560c8e4e76c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd690029e40 tx=0x7fd690004700 comp rx=0 tx=0).stop 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2197165673 shutdown_connections 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560c8e4fa510 0x560c8e4fc930 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4ea750 0x560c8e4e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1980 0x560c8e4e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd67bfff640 0 client.25636 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/2197165673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4e1060 0x560c8e4e76c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2197165673 >> 172.21.15.45:0/2197165673 conn(0x560c8e4db980 msgr2=0x560c8e4dd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2197165673 shutdown_connections 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd6a5e59740 1 -- 172.21.15.45:0/2197165673 wait complete. 2024-01-31T05:31:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.665+0000 7fd6a5e59740 20 client.25636 trim_cache size 0 max 16384 2024-01-31T05:31:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.679+0000 7fd6a5e59740 1 Processor -- start 2024-01-31T05:31:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.679+0000 7fd6a5e59740 1 -- start start 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.679+0000 7fd6a5e59740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4e1060 0x560c8e4e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.679+0000 7fd6a5e59740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1980 0x560c8e4e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.679+0000 7fd6a5e59740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4ea750 0x560c8e4e4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.679+0000 7fd6a5e59740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8e4e0340 con 0x560c8e4ea750 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.679+0000 7fd6a5e59740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8e4ecb40 con 0x560c8e4e1980 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.679+0000 7fd6a5e59740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8e4ed260 con 0x560c8e4e1060 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.679+0000 7fd69bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4ea750 0x560c8e4e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.679+0000 7fd69bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4ea750 0x560c8e4e4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45186/0 (socket says 172.21.15.45:45186) 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.679+0000 7fd69bfff640 1 -- 172.21.15.45:0/4142393276 learned_addr learned my addr 172.21.15.45:0/4142393276 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.679+0000 7fd69bfff640 1 --2- 172.21.15.45:0/4142393276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4e1060 0x560c8e4e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.680+0000 7fd6a0e35640 1 --2- 172.21.15.45:0/4142393276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1980 0x560c8e4e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.680+0000 7fd69bfff640 1 -- 172.21.15.45:0/4142393276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4e1060 msgr2=0x560c8e4e5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.680+0000 7fd69bfff640 1 --2- 172.21.15.45:0/4142393276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4e1060 0x560c8e4e5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.680+0000 7fd69bfff640 1 -- 172.21.15.45:0/4142393276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1980 msgr2=0x560c8e4e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.680+0000 7fd69bfff640 1 --2- 172.21.15.45:0/4142393276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1980 0x560c8e4e64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.680+0000 7fd69bfff640 1 -- 172.21.15.45:0/4142393276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c8e3e4030 con 0x560c8e4ea750 2024-01-31T05:31:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.680+0000 7fd6a0e35640 1 --2- 172.21.15.45:0/4142393276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1980 0x560c8e4e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:31:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.680+0000 7fd69bfff640 1 --2- 172.21.15.45:0/4142393276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4ea750 0x560c8e4e4920 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd690029990 tx=0x7fd690004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55d872ca93895c5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.680+0000 7fd6867fc640 1 -- 172.21.15.45:0/4142393276 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69001ae40 con 0x560c8e4ea750 2024-01-31T05:31:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.680+0000 7fd6867fc640 1 -- 172.21.15.45:0/4142393276 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd69001a430 con 0x560c8e4ea750 2024-01-31T05:31:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.680+0000 7fd6867fc640 1 -- 172.21.15.45:0/4142393276 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690026b30 con 0x560c8e4ea750 2024-01-31T05:31:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.681+0000 7fd6a5e59740 1 -- 172.21.15.45:0/4142393276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4ea750 msgr2=0x560c8e4e4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.681+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/4142393276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4ea750 0x560c8e4e4920 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd690029990 tx=0x7fd690004770 comp rx=0 tx=0).stop 2024-01-31T05:31:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.681+0000 7fd6a5e59740 1 -- 172.21.15.45:0/4142393276 shutdown_connections 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.681+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/4142393276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4ea750 0x560c8e4e4920 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.681+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/4142393276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1980 0x560c8e4e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.681+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/4142393276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4e1060 0x560c8e4e5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.681+0000 7fd6a5e59740 1 -- 172.21.15.45:0/4142393276 >> 172.21.15.45:0/4142393276 conn(0x560c8e4db980 msgr2=0x560c8e4f5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.681+0000 7fd6a5e59740 1 -- 172.21.15.45:0/4142393276 shutdown_connections 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.681+0000 7fd6a5e59740 1 -- 172.21.15.45:0/4142393276 wait complete. 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.681+0000 7fd6a5e59740 1 -- start start 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.681+0000 7fd6a5e59740 1 Processor -- start 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a5e59740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1060 0x560c8e4f1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a5e59740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 0x560c8e4efb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a5e59740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4ea750 0x560c8e4f0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a5e59740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8e4f0630 con 0x560c8e4e1980 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a5e59740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8e4f07a0 con 0x560c8e4e1060 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a5e59740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8e4f0960 con 0x560c8e4ea750 2024-01-31T05:31:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a5e59740 10 client.0 _ll_register_callbacks cb 0x7fd688009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:31:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a0e35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 0x560c8e4efb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a0e35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 0x560c8e4efb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45200/0 (socket says 172.21.15.45:45200) 2024-01-31T05:31:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a0e35640 1 -- 172.21.15.45:0/1511191191 learned_addr learned my addr 172.21.15.45:0/1511191191 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd69bfff640 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1060 0x560c8e4f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd69bfff640 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4ea750 0x560c8e4f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a0e35640 1 -- 172.21.15.45:0/1511191191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4ea750 msgr2=0x560c8e4f0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a0e35640 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4ea750 0x560c8e4f0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a0e35640 1 -- 172.21.15.45:0/1511191191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1060 msgr2=0x560c8e4f1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a0e35640 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1060 0x560c8e4f1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a0e35640 1 -- 172.21.15.45:0/1511191191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c8e4476a0 con 0x560c8e4e1980 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a0e35640 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 0x560c8e4efb30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd688000c00 tx=0x7fd68800fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96feb9cafeb6fc1b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd67bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd67bfff640 1 -- 172.21.15.45:0/1511191191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688021040 con 0x560c8e4e1980 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd67bfff640 1 -- 172.21.15.45:0/1511191191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd6880137e0 con 0x560c8e4e1980 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a5e59740 1 -- 172.21.15.45:0/1511191191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd688013470 con 0x560c8e4e1980 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd67bfff640 1 -- 172.21.15.45:0/1511191191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6880139a0 con 0x560c8e4e1980 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a5e59740 10 client.16083 fetch_fsmap learned FSMap version 28 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.682+0000 7fd6a5e59740 1 -- 172.21.15.45:0/1511191191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560c8e409a50 con 0x560c8e4e1980 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.683+0000 7fd67bfff640 1 -- 172.21.15.45:0/1511191191 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd688013470 con 0x560c8e4e1980 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.683+0000 7fd67bfff640 1 -- 172.21.15.45:0/1511191191 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd688011070 con 0x560c8e4e1980 2024-01-31T05:31:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.683+0000 7fd6a5e59740 10 client.16083 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:31:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.683+0000 7fd6a5e59740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:31:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.683+0000 7fd6a5e59740 20 client.16083 populate_metadata read hostname 'smithi045' 2024-01-31T05:31:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.683+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560c8e4fa510 0x560c8e554100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.683+0000 7fd6a5e59740 4 client.16083 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:31:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.683+0000 7fd6a5e59740 1 -- 172.21.15.45:0/1511191191 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c8e148600 con 0x560c8e4fa510 2024-01-31T05:31:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.683+0000 7fd69bfff640 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560c8e4fa510 0x560c8e554100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.683+0000 7fd67bfff640 10 client.16083 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.683+0000 7fd69bfff640 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560c8e4fa510 0x560c8e554100 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=801040f055efe447 server_cookie=f7b0053a2b92429d in_seq=0 out_seq=0 2024-01-31T05:31:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd67bfff640 1 -- 172.21.15.45:0/1511191191 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c8e148600 con 0x560c8e4fa510 2024-01-31T05:31:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd67bfff640 10 client.16083 handle_command_reply: tid=0 2024-01-31T05:31:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd6a5e59740 1 client.16083 shutdown 2024-01-31T05:31:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd6a5e59740 1 -- 172.21.15.45:0/1511191191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 msgr2=0x560c8e4efb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 0x560c8e4efb30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd688000c00 tx=0x7fd68800fcc0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd6a5e59740 1 -- 172.21.15.45:0/1511191191 shutdown_connections 2024-01-31T05:31:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560c8e4fa510 0x560c8e554100 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8e4e1980 0x560c8e4efb30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c8e4ea750 0x560c8e4f0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd67bfff640 0 client.16083 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd6a5e59740 1 --2- 172.21.15.45:0/1511191191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c8e4e1060 0x560c8e4f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd6a5e59740 1 -- 172.21.15.45:0/1511191191 >> 172.21.15.45:0/1511191191 conn(0x560c8e4db980 msgr2=0x560c8e4f5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd6a5e59740 1 -- 172.21.15.45:0/1511191191 shutdown_connections 2024-01-31T05:31:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd6a5e59740 1 -- 172.21.15.45:0/1511191191 wait complete. 2024-01-31T05:31:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.695+0000 7fd6a5e59740 20 client.16083 trim_cache size 0 max 16384 2024-01-31T05:31:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:31:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:31:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:31:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:31:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.696+0000 7fd6a3099640 1 -- 172.21.15.45:0/3935298793 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd68c0a1720 msgr2=0x7fd68c0a3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.696+0000 7fd6a3099640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd68c0a1720 0x7fd68c0a3be0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd6940bc9e0 tx=0x7fd69402c140 comp rx=0 tx=0).stop 2024-01-31T05:31:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.696+0000 7fd6a3099640 1 -- 172.21.15.45:0/3935298793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c11a4d0 msgr2=0x7fd69c1aa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.696+0000 7fd6a3099640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c11a4d0 0x7fd69c1aa9b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd694009bf0 tx=0x7fd694032980 comp rx=0 tx=0).stop 2024-01-31T05:31:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.696+0000 7fd6a3099640 1 -- 172.21.15.45:0/3935298793 shutdown_connections 2024-01-31T05:31:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.696+0000 7fd6a3099640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd68c0a1720 0x7fd68c0a3be0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.696+0000 7fd6a3099640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd69c10f8a0 0x7fd69c1a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.696+0000 7fd6a3099640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c11a4d0 0x7fd69c1aa9b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.696+0000 7fd6a3099640 1 --2- 172.21.15.45:0/3935298793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c10f430 0x7fd69c1a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.696+0000 7fd6a3099640 1 -- 172.21.15.45:0/3935298793 >> 172.21.15.45:0/3935298793 conn(0x7fd69c06bd90 msgr2=0x7fd69c118ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.696+0000 7fd6a3099640 1 -- 172.21.15.45:0/3935298793 shutdown_connections 2024-01-31T05:31:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:10.696+0000 7fd6a3099640 1 -- 172.21.15.45:0/3935298793 wait complete. 2024-01-31T05:31:10.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 905574 11.16 MB/sec execute 310 sec latency 33.802 ms 2024-01-31T05:31:11.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 906949 11.15 MB/sec execute 311 sec latency 96.344 ms 2024-01-31T05:31:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:11 smithi045 ceph-mon[31585]: osdmap e98: 12 total, 12 up, 12 in 2024-01-31T05:31:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:11 smithi045 ceph-mon[31585]: pgmap v267: 97 pgs: 97 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 251 op/s 2024-01-31T05:31:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:11 smithi156 ceph-mon[35720]: osdmap e98: 12 total, 12 up, 12 in 2024-01-31T05:31:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:11 smithi156 ceph-mon[35720]: pgmap v267: 97 pgs: 97 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 251 op/s 2024-01-31T05:31:12.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:11 smithi186 ceph-mon[35659]: osdmap e98: 12 total, 12 up, 12 in 2024-01-31T05:31:12.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:11 smithi186 ceph-mon[35659]: pgmap v267: 97 pgs: 97 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 251 op/s 2024-01-31T05:31:12.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 908968 11.15 MB/sec execute 312 sec latency 38.807 ms 2024-01-31T05:31:13.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 910531 11.14 MB/sec execute 313 sec latency 26.866 ms 2024-01-31T05:31:14.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 912527 11.14 MB/sec execute 314 sec latency 19.266 ms 2024-01-31T05:31:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:14 smithi045 ceph-mon[31585]: pgmap v268: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 221 op/s 2024-01-31T05:31:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:14 smithi156 ceph-mon[35720]: pgmap v268: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 221 op/s 2024-01-31T05:31:15.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:14 smithi186 ceph-mon[35659]: pgmap v268: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 221 op/s 2024-01-31T05:31:15.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 914208 11.13 MB/sec execute 315 sec latency 18.129 ms 2024-01-31T05:31:16.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 916045 11.13 MB/sec execute 316 sec latency 76.215 ms 2024-01-31T05:31:16.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:16 smithi045 ceph-mon[31585]: pgmap v269: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 221 op/s 2024-01-31T05:31:16.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:16 smithi156 ceph-mon[35720]: pgmap v269: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 221 op/s 2024-01-31T05:31:17.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:16 smithi186 ceph-mon[35659]: pgmap v269: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 221 op/s 2024-01-31T05:31:17.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 916935 11.13 MB/sec execute 317 sec latency 80.560 ms 2024-01-31T05:31:18.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 918661 11.15 MB/sec execute 318 sec latency 51.253 ms 2024-01-31T05:31:18.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:18 smithi045 ceph-mon[31585]: pgmap v270: 97 pgs: 97 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-01-31T05:31:18.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:18 smithi045 ceph-mon[31585]: osdmap e99: 12 total, 12 up, 12 in 2024-01-31T05:31:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:18 smithi156 ceph-mon[35720]: pgmap v270: 97 pgs: 97 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-01-31T05:31:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:18 smithi156 ceph-mon[35720]: osdmap e99: 12 total, 12 up, 12 in 2024-01-31T05:31:19.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:18 smithi186 ceph-mon[35659]: pgmap v270: 97 pgs: 97 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-01-31T05:31:19.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:18 smithi186 ceph-mon[35659]: osdmap e99: 12 total, 12 up, 12 in 2024-01-31T05:31:19.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 920716 11.16 MB/sec execute 319 sec latency 16.131 ms 2024-01-31T05:31:20.791 INFO:tasks.workunit.client.0.smithi045.stdout: 1 922964 11.16 MB/sec execute 320 sec latency 18.734 ms 2024-01-31T05:31:20.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:20 smithi045 ceph-mon[31585]: pgmap v272: 97 pgs: 97 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 229 op/s 2024-01-31T05:31:20.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:20 smithi156 ceph-mon[35720]: pgmap v272: 97 pgs: 97 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 229 op/s 2024-01-31T05:31:21.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:20 smithi186 ceph-mon[35659]: pgmap v272: 97 pgs: 97 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 229 op/s 2024-01-31T05:31:21.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 924315 11.15 MB/sec execute 321 sec latency 60.930 ms 2024-01-31T05:31:22.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 926281 11.14 MB/sec execute 322 sec latency 31.659 ms 2024-01-31T05:31:22.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:22 smithi045 ceph-mon[31585]: pgmap v273: 97 pgs: 97 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-01-31T05:31:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:22 smithi156 ceph-mon[35720]: pgmap v273: 97 pgs: 97 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-01-31T05:31:23.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:22 smithi186 ceph-mon[35659]: pgmap v273: 97 pgs: 97 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-01-31T05:31:23.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 928126 11.14 MB/sec execute 323 sec latency 17.915 ms 2024-01-31T05:31:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.792+0000 7fb1053fb640 1 -- 172.21.15.45:0/2992153688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb100100f60 msgr2=0x7fb100104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.792+0000 7fb1053fb640 1 --2- 172.21.15.45:0/2992153688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb100100f60 0x7fb100104810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009890 tx=0x7fb0e802bb70 comp rx=0 tx=0).stop 2024-01-31T05:31:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.792+0000 7fb1053fb640 1 -- 172.21.15.45:0/2992153688 shutdown_connections 2024-01-31T05:31:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.792+0000 7fb1053fb640 1 --2- 172.21.15.45:0/2992153688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb100104ef0 0x7fb10010cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.792+0000 7fb1053fb640 1 --2- 172.21.15.45:0/2992153688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb100100f60 0x7fb100104810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.793+0000 7fb1053fb640 1 --2- 172.21.15.45:0/2992153688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1001005b0 0x7fb100100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.793+0000 7fb1053fb640 1 -- 172.21.15.45:0/2992153688 >> 172.21.15.45:0/2992153688 conn(0x7fb100076110 msgr2=0x7fb1000fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.793+0000 7fb1053fb640 1 -- 172.21.15.45:0/2992153688 shutdown_connections 2024-01-31T05:31:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.793+0000 7fb1053fb640 1 -- 172.21.15.45:0/2992153688 wait complete. 2024-01-31T05:31:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.793+0000 7fb1053fb640 1 Processor -- start 2024-01-31T05:31:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.793+0000 7fb1053fb640 1 -- start start 2024-01-31T05:31:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.793+0000 7fb1053fb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1001005b0 0x7fb100197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.793+0000 7fb1053fb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb100104ef0 0x7fb100197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.793+0000 7fb1053fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10019bae0 0x7fb10019bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.793+0000 7fb1053fb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10019c4d0 con 0x7fb10019bae0 2024-01-31T05:31:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.793+0000 7fb1053fb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10019c640 con 0x7fb1001005b0 2024-01-31T05:31:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.793+0000 7fb1053fb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10019c7b0 con 0x7fb100104ef0 2024-01-31T05:31:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10019bae0 0x7fb10019bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0feffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1001005b0 0x7fb100197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10019bae0 0x7fb10019bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50344/0 (socket says 172.21.15.45:50344) 2024-01-31T05:31:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0ff7fe640 1 -- 172.21.15.45:0/4079837580 learned_addr learned my addr 172.21.15.45:0/4079837580 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0f7fff640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb100104ef0 0x7fb100197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0ff7fe640 1 -- 172.21.15.45:0/4079837580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb100104ef0 msgr2=0x7fb100197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0ff7fe640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb100104ef0 0x7fb100197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0ff7fe640 1 -- 172.21.15.45:0/4079837580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1001005b0 msgr2=0x7fb100197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0ff7fe640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1001005b0 0x7fb100197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0ff7fe640 1 -- 172.21.15.45:0/4079837580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f8009900 con 0x7fb10019bae0 2024-01-31T05:31:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0ff7fe640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10019bae0 0x7fb10019bf90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800efc0 tx=0x7fb0f800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0fcff9640 1 -- 172.21.15.45:0/4079837580 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8003970 con 0x7fb10019bae0 2024-01-31T05:31:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb1053fb640 1 -- 172.21.15.45:0/4079837580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e8009490 con 0x7fb10019bae0 2024-01-31T05:31:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0fcff9640 1 -- 172.21.15.45:0/4079837580 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0f8004020 con 0x7fb10019bae0 2024-01-31T05:31:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb1053fb640 1 -- 172.21.15.45:0/4079837580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1001a4080 con 0x7fb10019bae0 2024-01-31T05:31:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.794+0000 7fb0fcff9640 1 -- 172.21.15.45:0/4079837580 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800b850 con 0x7fb10019bae0 2024-01-31T05:31:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.797+0000 7fb0fcff9640 1 -- 172.21.15.45:0/4079837580 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fb0f800a040 con 0x7fb10019bae0 2024-01-31T05:31:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.797+0000 7fb0fcff9640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb0ec0a1570 0x7fb0ec0a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.797+0000 7fb1081bd740 1 Processor -- start 2024-01-31T05:31:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.797+0000 7fb1081bd740 1 -- start start 2024-01-31T05:31:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.797+0000 7fb0feffd640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb0ec0a1570 0x7fb0ec0a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.797+0000 7fb0fcff9640 1 -- 172.21.15.45:0/4079837580 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb0f80ccfd0 con 0x7fb10019bae0 2024-01-31T05:31:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb1081bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72edeb80 0x55ec72edef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb1081bd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edf4a0 0x55ec72eec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb1081bd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72eecb60 0x55ec72eeef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb1081bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec72e4e040 con 0x55ec72edeb80 2024-01-31T05:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb1081bd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec72e4b730 con 0x55ec72eecb60 2024-01-31T05:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb1081bd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec72e47020 con 0x55ec72edf4a0 2024-01-31T05:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0f7fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edf4a0 0x55ec72eec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0feffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72edeb80 0x55ec72edef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72eecb60 0x55ec72eeef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0f7fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edf4a0 0x55ec72eec620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:41932/0 (socket says 172.21.15.45:41932) 2024-01-31T05:31:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0f7fff640 1 -- 172.21.15.45:0/3036533818 learned_addr learned my addr 172.21.15.45:0/3036533818 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0feffd640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb0ec0a1570 0x7fb0ec0a3a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009c20 tx=0x7fb0f0011810 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0ff7fe640 1 -- 172.21.15.45:0/3036533818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edf4a0 msgr2=0x55ec72eec620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0ff7fe640 1 --2- 172.21.15.45:0/3036533818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edf4a0 0x55ec72eec620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0ff7fe640 1 -- 172.21.15.45:0/3036533818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72edeb80 msgr2=0x55ec72edef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0ff7fe640 1 --2- 172.21.15.45:0/3036533818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72edeb80 0x55ec72edef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0ff7fe640 1 -- 172.21.15.45:0/3036533818 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec72e3ee40 con 0x55ec72eecb60 2024-01-31T05:31:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0ff7fe640 1 --2- 172.21.15.45:0/3036533818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72eecb60 0x55ec72eeef50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80ca720 tx=0x7fb0f809a590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37bc765e8d0d4f49 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0f57fa640 1 -- 172.21.15.45:0/3036533818 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80b9040 con 0x55ec72eecb60 2024-01-31T05:31:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0f57fa640 1 -- 172.21.15.45:0/3036533818 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0f80a2160 con 0x55ec72eecb60 2024-01-31T05:31:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.798+0000 7fb0f57fa640 1 -- 172.21.15.45:0/3036533818 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80c3610 con 0x55ec72eecb60 2024-01-31T05:31:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.799+0000 7fb1081bd740 1 -- 172.21.15.45:0/3036533818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72eecb60 msgr2=0x55ec72eeef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.799+0000 7fb1081bd740 1 --2- 172.21.15.45:0/3036533818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72eecb60 0x55ec72eeef50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80ca720 tx=0x7fb0f809a590 comp rx=0 tx=0).stop 2024-01-31T05:31:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.799+0000 7fb1081bd740 1 -- 172.21.15.45:0/3036533818 shutdown_connections 2024-01-31T05:31:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.799+0000 7fb1081bd740 1 --2- 172.21.15.45:0/3036533818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72eecb60 0x55ec72eeef50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.799+0000 7fb1081bd740 1 --2- 172.21.15.45:0/3036533818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edf4a0 0x55ec72eec620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.799+0000 7fb1081bd740 1 --2- 172.21.15.45:0/3036533818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72edeb80 0x55ec72edef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.799+0000 7fb1081bd740 1 -- 172.21.15.45:0/3036533818 >> 172.21.15.45:0/3036533818 conn(0x55ec72ed9980 msgr2=0x55ec72ed9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.799+0000 7fb1081bd740 1 -- 172.21.15.45:0/3036533818 shutdown_connections 2024-01-31T05:31:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.799+0000 7fb1081bd740 1 -- 172.21.15.45:0/3036533818 wait complete. 2024-01-31T05:31:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb1081bd740 1 -- start start 2024-01-31T05:31:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb1081bd740 1 Processor -- start 2024-01-31T05:31:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb1081bd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edeb80 0x55ec72ee9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb1081bd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72edf4a0 0x55ec72ee9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb1081bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72eecb60 0x55ec72eea510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb1081bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec72e001b0 con 0x55ec72eecb60 2024-01-31T05:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb1081bd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec72dffe50 con 0x55ec72edf4a0 2024-01-31T05:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb1081bd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec72dc5090 con 0x55ec72edeb80 2024-01-31T05:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb1081bd740 10 client.0 _ll_register_callbacks cb 0x55ec72deaa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb0feffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edeb80 0x55ec72ee9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb0f7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72edf4a0 0x55ec72ee9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb0feffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edeb80 0x55ec72ee9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:41948/0 (socket says 172.21.15.45:41948) 2024-01-31T05:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb0feffd640 1 -- 172.21.15.45:0/1897480255 learned_addr learned my addr 172.21.15.45:0/1897480255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.800+0000 7fb0ff7fe640 1 --2- 172.21.15.45:0/1897480255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72eecb60 0x55ec72eea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb0feffd640 1 -- 172.21.15.45:0/1897480255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72edf4a0 msgr2=0x55ec72ee9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb0feffd640 1 --2- 172.21.15.45:0/1897480255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72edf4a0 0x55ec72ee9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb0feffd640 1 -- 172.21.15.45:0/1897480255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72eecb60 msgr2=0x55ec72eea510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb0feffd640 1 --2- 172.21.15.45:0/1897480255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72eecb60 0x55ec72eea510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb0feffd640 1 -- 172.21.15.45:0/1897480255 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f80cae70 con 0x55ec72edeb80 2024-01-31T05:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb0feffd640 1 --2- 172.21.15.45:0/1897480255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edeb80 0x55ec72ee9a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb0f001ea00 tx=0x7fb0f0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2e0c78e5b1ab16e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb0e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb0e67fc640 1 -- 172.21.15.45:0/1897480255 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f001e340 con 0x55ec72edeb80 2024-01-31T05:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb1081bd740 1 -- 172.21.15.45:0/1897480255 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec72e40010 con 0x55ec72edeb80 2024-01-31T05:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb0e67fc640 1 -- 172.21.15.45:0/1897480255 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb0f000f480 con 0x55ec72edeb80 2024-01-31T05:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb0e67fc640 1 -- 172.21.15.45:0/1897480255 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0f000f640 con 0x55ec72edeb80 2024-01-31T05:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb1081bd740 10 client.25654 fetch_fsmap learned FSMap version 28 2024-01-31T05:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb1081bd740 1 -- 172.21.15.45:0/1897480255 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ec72e456a0 con 0x55ec72edeb80 2024-01-31T05:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.801+0000 7fb0e67fc640 1 -- 172.21.15.45:0/1897480255 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec72e40010 con 0x55ec72edeb80 2024-01-31T05:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.802+0000 7fb0e67fc640 1 -- 172.21.15.45:0/1897480255 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb0f0027940 con 0x55ec72edeb80 2024-01-31T05:31:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.802+0000 7fb1081bd740 10 client.25654 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:31:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.802+0000 7fb1081bd740 10 client.25654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:31:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.802+0000 7fb1081bd740 20 client.25654 populate_metadata read hostname 'smithi045' 2024-01-31T05:31:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.802+0000 7fb1081bd740 1 --2- 172.21.15.45:0/1897480255 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ec72efc9a0 0x55ec72efedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.802+0000 7fb1081bd740 4 client.25654 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:31:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.802+0000 7fb1081bd740 1 -- 172.21.15.45:0/1897480255 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec72e3fd30 con 0x55ec72efc9a0 2024-01-31T05:31:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.802+0000 7fb0f7fff640 1 --2- 172.21.15.45:0/1897480255 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ec72efc9a0 0x55ec72efedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.802+0000 7fb0e67fc640 10 client.25654 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.802+0000 7fb0f7fff640 1 --2- 172.21.15.45:0/1897480255 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ec72efc9a0 0x55ec72efedc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be4cc284c9bd19d6 server_cookie=73a2e34dc7685d68 in_seq=0 out_seq=0 2024-01-31T05:31:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb0e67fc640 1 -- 172.21.15.45:0/1897480255 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec72e3fd30 con 0x55ec72efc9a0 2024-01-31T05:31:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb0e67fc640 10 client.25654 handle_command_reply: tid=0 2024-01-31T05:31:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb1081bd740 1 client.25654 shutdown 2024-01-31T05:31:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb1081bd740 1 -- 172.21.15.45:0/1897480255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edeb80 msgr2=0x55ec72ee9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb1081bd740 1 --2- 172.21.15.45:0/1897480255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edeb80 0x55ec72ee9a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb0f001ea00 tx=0x7fb0f0004730 comp rx=0 tx=0).stop 2024-01-31T05:31:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb1081bd740 1 -- 172.21.15.45:0/1897480255 shutdown_connections 2024-01-31T05:31:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb1081bd740 1 --2- 172.21.15.45:0/1897480255 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ec72efc9a0 0x55ec72efedc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb1081bd740 1 --2- 172.21.15.45:0/1897480255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72eecb60 0x55ec72eea510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb1081bd740 1 --2- 172.21.15.45:0/1897480255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72edf4a0 0x55ec72ee9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb1081bd740 1 --2- 172.21.15.45:0/1897480255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edeb80 0x55ec72ee9a90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb0e67fc640 0 client.25654 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb1081bd740 1 -- 172.21.15.45:0/1897480255 >> 172.21.15.45:0/1897480255 conn(0x55ec72ed9980 msgr2=0x55ec72edb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb1081bd740 1 -- 172.21.15.45:0/1897480255 shutdown_connections 2024-01-31T05:31:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.803+0000 7fb1081bd740 1 -- 172.21.15.45:0/1897480255 wait complete. 2024-01-31T05:31:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.804+0000 7fb1081bd740 20 client.25654 trim_cache size 0 max 16384 2024-01-31T05:31:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb1081bd740 1 Processor -- start 2024-01-31T05:31:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb1081bd740 1 -- start start 2024-01-31T05:31:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb1081bd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72edeb80 0x55ec72eec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb1081bd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edf4a0 0x55ec72ee8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb1081bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72eecb60 0x55ec72eea330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb1081bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec72ed50b0 con 0x55ec72eecb60 2024-01-31T05:31:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb1081bd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec72e40010 con 0x55ec72edeb80 2024-01-31T05:31:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb1081bd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec72ed97f0 con 0x55ec72edf4a0 2024-01-31T05:31:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72eecb60 0x55ec72eea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb0feffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72edeb80 0x55ec72eec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb0f7fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edf4a0 0x55ec72ee8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72eecb60 0x55ec72eea330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50368/0 (socket says 172.21.15.45:50368) 2024-01-31T05:31:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb0feffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72edeb80 0x55ec72eec1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:47058/0 (socket says 172.21.15.45:47058) 2024-01-31T05:31:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb0ff7fe640 1 -- 172.21.15.45:0/2515074534 learned_addr learned my addr 172.21.15.45:0/2515074534 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb0ff7fe640 1 -- 172.21.15.45:0/2515074534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edf4a0 msgr2=0x55ec72ee8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb0ff7fe640 1 --2- 172.21.15.45:0/2515074534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edf4a0 0x55ec72ee8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb0ff7fe640 1 -- 172.21.15.45:0/2515074534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72edeb80 msgr2=0x55ec72eec1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.820+0000 7fb0ff7fe640 1 --2- 172.21.15.45:0/2515074534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72edeb80 0x55ec72eec1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.821+0000 7fb0ff7fe640 1 -- 172.21.15.45:0/2515074534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec72de2030 con 0x55ec72eecb60 2024-01-31T05:31:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.821+0000 7fb0feffd640 1 --2- 172.21.15.45:0/2515074534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72edeb80 0x55ec72eec1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:31:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.821+0000 7fb0ff7fe640 1 --2- 172.21.15.45:0/2515074534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72eecb60 0x55ec72eea330 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8005c40 tx=0x7fb0f8004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7918763a85956ad server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.821+0000 7fb0f4ff9640 1 -- 172.21.15.45:0/2515074534 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80b9040 con 0x55ec72eecb60 2024-01-31T05:31:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.821+0000 7fb0f4ff9640 1 -- 172.21.15.45:0/2515074534 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0f8004400 con 0x55ec72eecb60 2024-01-31T05:31:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.821+0000 7fb0f4ff9640 1 -- 172.21.15.45:0/2515074534 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8099520 con 0x55ec72eecb60 2024-01-31T05:31:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 -- 172.21.15.45:0/2515074534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72eecb60 msgr2=0x55ec72eea330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 --2- 172.21.15.45:0/2515074534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72eecb60 0x55ec72eea330 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8005c40 tx=0x7fb0f8004700 comp rx=0 tx=0).stop 2024-01-31T05:31:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 -- 172.21.15.45:0/2515074534 shutdown_connections 2024-01-31T05:31:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 --2- 172.21.15.45:0/2515074534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72eecb60 0x55ec72eea330 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 --2- 172.21.15.45:0/2515074534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edf4a0 0x55ec72ee8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 --2- 172.21.15.45:0/2515074534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72edeb80 0x55ec72eec1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 -- 172.21.15.45:0/2515074534 >> 172.21.15.45:0/2515074534 conn(0x55ec72ed9980 msgr2=0x55ec72ee8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 -- 172.21.15.45:0/2515074534 shutdown_connections 2024-01-31T05:31:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 -- 172.21.15.45:0/2515074534 wait complete. 2024-01-31T05:31:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 -- start start 2024-01-31T05:31:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 Processor -- start 2024-01-31T05:31:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edeb80 0x55ec72ef3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.822+0000 7fb1081bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72edf4a0 0x55ec72ef1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb1081bd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72eecb60 0x55ec72ef2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb1081bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec72ef28a0 con 0x55ec72edf4a0 2024-01-31T05:31:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb1081bd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec72ef2a10 con 0x55ec72eecb60 2024-01-31T05:31:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb1081bd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec72ef2b80 con 0x55ec72edeb80 2024-01-31T05:31:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb1081bd740 10 client.0 _ll_register_callbacks cb 0x7fb0f8096110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:31:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0f7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72edf4a0 0x55ec72ef1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0f7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72edf4a0 0x55ec72ef1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50374/0 (socket says 172.21.15.45:50374) 2024-01-31T05:31:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0f7fff640 1 -- 172.21.15.45:0/110680946 learned_addr learned my addr 172.21.15.45:0/110680946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0feffd640 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edeb80 0x55ec72ef3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0ff7fe640 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72eecb60 0x55ec72ef2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0f7fff640 1 -- 172.21.15.45:0/110680946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edeb80 msgr2=0x55ec72ef3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0f7fff640 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edeb80 0x55ec72ef3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0f7fff640 1 -- 172.21.15.45:0/110680946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72eecb60 msgr2=0x55ec72ef2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0f7fff640 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72eecb60 0x55ec72ef2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0f7fff640 1 -- 172.21.15.45:0/110680946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec72e456a0 con 0x55ec72edf4a0 2024-01-31T05:31:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0f7fff640 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72edf4a0 0x55ec72ef1e20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009860 tx=0x7fb0e802cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551e2f8a8d0efb71 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:31:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0e67fc640 1 -- 172.21.15.45:0/110680946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8046040 con 0x55ec72edf4a0 2024-01-31T05:31:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0e67fc640 1 -- 172.21.15.45:0/110680946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb0e8034ad0 con 0x55ec72edf4a0 2024-01-31T05:31:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb1081bd740 1 -- 172.21.15.45:0/110680946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0f80a2160 con 0x55ec72edf4a0 2024-01-31T05:31:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb0e67fc640 1 -- 172.21.15.45:0/110680946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0e8034c90 con 0x55ec72edf4a0 2024-01-31T05:31:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb1081bd740 10 client.16101 fetch_fsmap learned FSMap version 28 2024-01-31T05:31:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.823+0000 7fb1081bd740 1 -- 172.21.15.45:0/110680946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ec72e3ee40 con 0x55ec72edf4a0 2024-01-31T05:31:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.824+0000 7fb0e67fc640 1 -- 172.21.15.45:0/110680946 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0f80a2160 con 0x55ec72edf4a0 2024-01-31T05:31:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.824+0000 7fb0e67fc640 1 -- 172.21.15.45:0/110680946 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb0e802c510 con 0x55ec72edf4a0 2024-01-31T05:31:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.824+0000 7fb1081bd740 10 client.16101 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:31:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.824+0000 7fb1081bd740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:31:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.824+0000 7fb1081bd740 20 client.16101 populate_metadata read hostname 'smithi045' 2024-01-31T05:31:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.824+0000 7fb1081bd740 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ec72f54470 0x55ec72f56890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.824+0000 7fb1081bd740 4 client.16101 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:31:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.824+0000 7fb1081bd740 1 -- 172.21.15.45:0/110680946 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ec72b46600 con 0x55ec72f54470 2024-01-31T05:31:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.824+0000 7fb0feffd640 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ec72f54470 0x55ec72f56890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.824+0000 7fb0e67fc640 10 client.16101 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.825+0000 7fb0feffd640 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ec72f54470 0x55ec72f56890 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc627d391a9011ac server_cookie=bbb85ebb9bc86417 in_seq=0 out_seq=0 2024-01-31T05:31:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb0e67fc640 1 -- 172.21.15.45:0/110680946 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ec72b46600 con 0x55ec72f54470 2024-01-31T05:31:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb0e67fc640 10 client.16101 handle_command_reply: tid=0 2024-01-31T05:31:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb1081bd740 1 client.16101 shutdown 2024-01-31T05:31:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb1081bd740 1 -- 172.21.15.45:0/110680946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72edf4a0 msgr2=0x55ec72ef1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb1081bd740 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72edf4a0 0x55ec72ef1e20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009860 tx=0x7fb0e802cea0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb1081bd740 1 -- 172.21.15.45:0/110680946 shutdown_connections 2024-01-31T05:31:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb1081bd740 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ec72f54470 0x55ec72f56890 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb1081bd740 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec72edf4a0 0x55ec72ef1e20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb1081bd740 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec72eecb60 0x55ec72ef2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb0e67fc640 0 client.16101 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb1081bd740 1 --2- 172.21.15.45:0/110680946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec72edeb80 0x55ec72ef3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb1081bd740 1 -- 172.21.15.45:0/110680946 >> 172.21.15.45:0/110680946 conn(0x55ec72ed9980 msgr2=0x55ec72ed8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb1081bd740 1 -- 172.21.15.45:0/110680946 shutdown_connections 2024-01-31T05:31:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb1081bd740 1 -- 172.21.15.45:0/110680946 wait complete. 2024-01-31T05:31:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.836+0000 7fb1081bd740 20 client.16101 trim_cache size 0 max 16384 2024-01-31T05:31:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:31:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:31:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:31:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:31:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.837+0000 7fb1053fb640 1 -- 172.21.15.45:0/4079837580 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb0ec0a1570 msgr2=0x7fb0ec0a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.837+0000 7fb1053fb640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb0ec0a1570 0x7fb0ec0a3a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009c20 tx=0x7fb0f0011810 comp rx=0 tx=0).stop 2024-01-31T05:31:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.837+0000 7fb1053fb640 1 -- 172.21.15.45:0/4079837580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10019bae0 msgr2=0x7fb10019bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.837+0000 7fb1053fb640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10019bae0 0x7fb10019bf90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800efc0 tx=0x7fb0f800c800 comp rx=0 tx=0).stop 2024-01-31T05:31:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.837+0000 7fb1053fb640 1 -- 172.21.15.45:0/4079837580 shutdown_connections 2024-01-31T05:31:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.837+0000 7fb1053fb640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb0ec0a1570 0x7fb0ec0a3a30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.837+0000 7fb1053fb640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10019bae0 0x7fb10019bf90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.837+0000 7fb1053fb640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb100104ef0 0x7fb100197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.837+0000 7fb1053fb640 1 --2- 172.21.15.45:0/4079837580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1001005b0 0x7fb100197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.837+0000 7fb1053fb640 1 -- 172.21.15.45:0/4079837580 >> 172.21.15.45:0/4079837580 conn(0x7fb100076110 msgr2=0x7fb1000ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.837+0000 7fb1053fb640 1 -- 172.21.15.45:0/4079837580 shutdown_connections 2024-01-31T05:31:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:23.837+0000 7fb1053fb640 1 -- 172.21.15.45:0/4079837580 wait complete. 2024-01-31T05:31:24.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 930709 11.15 MB/sec execute 324 sec latency 14.246 ms 2024-01-31T05:31:24.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:24 smithi045 ceph-mon[31585]: pgmap v274: 97 pgs: 97 active+clean; 501 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 203 op/s 2024-01-31T05:31:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:24 smithi156 ceph-mon[35720]: pgmap v274: 97 pgs: 97 active+clean; 501 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 203 op/s 2024-01-31T05:31:25.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:24 smithi186 ceph-mon[35659]: pgmap v274: 97 pgs: 97 active+clean; 501 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 203 op/s 2024-01-31T05:31:25.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 932319 11.14 MB/sec execute 325 sec latency 17.679 ms 2024-01-31T05:31:26.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 934490 11.15 MB/sec execute 326 sec latency 103.685 ms 2024-01-31T05:31:26.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:26 smithi045 ceph-mon[31585]: pgmap v275: 97 pgs: 97 active+clean; 501 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 203 op/s 2024-01-31T05:31:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:26 smithi156 ceph-mon[35720]: pgmap v275: 97 pgs: 97 active+clean; 501 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 203 op/s 2024-01-31T05:31:27.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:26 smithi186 ceph-mon[35659]: pgmap v275: 97 pgs: 97 active+clean; 501 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 203 op/s 2024-01-31T05:31:27.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 935857 11.13 MB/sec execute 327 sec latency 95.588 ms 2024-01-31T05:31:28.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 938262 11.14 MB/sec execute 328 sec latency 13.373 ms 2024-01-31T05:31:28.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:28 smithi045 ceph-mon[31585]: pgmap v276: 97 pgs: 97 active+clean; 510 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 208 op/s 2024-01-31T05:31:28.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:28 smithi186 ceph-mon[35659]: pgmap v276: 97 pgs: 97 active+clean; 510 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 208 op/s 2024-01-31T05:31:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:28 smithi156 ceph-mon[35720]: pgmap v276: 97 pgs: 97 active+clean; 510 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 208 op/s 2024-01-31T05:31:29.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 940442 11.13 MB/sec execute 329 sec latency 16.125 ms 2024-01-31T05:31:30.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 942352 11.13 MB/sec execute 330 sec latency 15.775 ms 2024-01-31T05:31:30.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:30 smithi045 ceph-mon[31585]: pgmap v277: 97 pgs: 97 active+clean; 510 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 173 op/s 2024-01-31T05:31:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:30 smithi156 ceph-mon[35720]: pgmap v277: 97 pgs: 97 active+clean; 510 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 173 op/s 2024-01-31T05:31:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:30 smithi186 ceph-mon[35659]: pgmap v277: 97 pgs: 97 active+clean; 510 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 173 op/s 2024-01-31T05:31:31.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 944490 11.14 MB/sec execute 331 sec latency 106.057 ms 2024-01-31T05:31:32.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 946142 11.13 MB/sec execute 332 sec latency 19.544 ms 2024-01-31T05:31:32.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:32 smithi045 ceph-mon[31585]: pgmap v278: 97 pgs: 97 active+clean; 510 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 173 op/s 2024-01-31T05:31:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:32 smithi156 ceph-mon[35720]: pgmap v278: 97 pgs: 97 active+clean; 510 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 173 op/s 2024-01-31T05:31:33.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:32 smithi186 ceph-mon[35659]: pgmap v278: 97 pgs: 97 active+clean; 510 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 173 op/s 2024-01-31T05:31:33.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 948729 11.14 MB/sec execute 333 sec latency 15.402 ms 2024-01-31T05:31:34.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 950261 11.13 MB/sec execute 334 sec latency 16.806 ms 2024-01-31T05:31:34.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:34 smithi045 ceph-mon[31585]: pgmap v279: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 264 op/s 2024-01-31T05:31:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:34 smithi156 ceph-mon[35720]: pgmap v279: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 264 op/s 2024-01-31T05:31:35.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:34 smithi186 ceph-mon[35659]: pgmap v279: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 264 op/s 2024-01-31T05:31:35.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 952499 11.14 MB/sec execute 335 sec latency 16.002 ms 2024-01-31T05:31:36.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 954057 11.12 MB/sec execute 336 sec latency 108.388 ms 2024-01-31T05:31:36.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:36 smithi045 ceph-mon[31585]: pgmap v280: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 176 op/s 2024-01-31T05:31:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.926+0000 7f52c32df640 1 -- 172.21.15.45:0/1348194592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52bc105cd0 msgr2=0x7f52bc1060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.926+0000 7f52c32df640 1 --2- 172.21.15.45:0/1348194592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52bc105cd0 0x7f52bc1060b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009860 tx=0x7f52b802bb70 comp rx=0 tx=0).stop 2024-01-31T05:31:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.926+0000 7f52c32df640 1 -- 172.21.15.45:0/1348194592 shutdown_connections 2024-01-31T05:31:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.926+0000 7f52c32df640 1 --2- 172.21.15.45:0/1348194592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc0ff1f0 0x7f52bc0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.926+0000 7f52c32df640 1 --2- 172.21.15.45:0/1348194592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52bc0ffbb0 0x7f52bc10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.926+0000 7f52c32df640 1 --2- 172.21.15.45:0/1348194592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52bc105cd0 0x7f52bc1060b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.926+0000 7f52c32df640 1 -- 172.21.15.45:0/1348194592 >> 172.21.15.45:0/1348194592 conn(0x7f52bc076180 msgr2=0x7f52bc0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.927+0000 7f52c32df640 1 -- 172.21.15.45:0/1348194592 shutdown_connections 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.927+0000 7f52c32df640 1 -- 172.21.15.45:0/1348194592 wait complete. 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.927+0000 7f52c32df640 1 Processor -- start 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.927+0000 7f52c32df640 1 -- start start 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.927+0000 7f52c32df640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52bc0ff1f0 0x7f52bc197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.927+0000 7f52c32df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52bc0ffbb0 0x7f52bc197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.927+0000 7f52c32df640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc105cd0 0x7f52bc19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.927+0000 7f52c32df640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc19c070 con 0x7f52bc105cd0 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.927+0000 7f52c32df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc19c1e0 con 0x7f52bc0ffbb0 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.927+0000 7f52c32df640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc19c3a0 con 0x7f52bc0ff1f0 2024-01-31T05:31:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c107b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52bc0ff1f0 0x7f52bc197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c187c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc105cd0 0x7f52bc19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c107b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52bc0ff1f0 0x7f52bc197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:57594/0 (socket says 172.21.15.45:57594) 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c107b640 1 -- 172.21.15.45:0/3871795765 learned_addr learned my addr 172.21.15.45:0/3871795765 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c087a640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52bc0ffbb0 0x7f52bc197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c107b640 1 -- 172.21.15.45:0/3871795765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52bc0ffbb0 msgr2=0x7f52bc197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c107b640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52bc0ffbb0 0x7f52bc197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c107b640 1 -- 172.21.15.45:0/3871795765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc105cd0 msgr2=0x7f52bc19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c107b640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc105cd0 0x7f52bc19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c107b640 1 -- 172.21.15.45:0/3871795765 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac009560 con 0x7f52bc0ff1f0 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c187c640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc105cd0 0x7f52bc19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c087a640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52bc0ffbb0 0x7f52bc197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c107b640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52bc0ff1f0 0x7f52bc197200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f52b8037040 tx=0x7f52b8004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52b27fc640 1 -- 172.21.15.45:0/3871795765 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8034850 con 0x7f52bc0ff1f0 2024-01-31T05:31:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c32df640 1 -- 172.21.15.45:0/3871795765 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b8009490 con 0x7f52bc0ff1f0 2024-01-31T05:31:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52b27fc640 1 -- 172.21.15.45:0/3871795765 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b802c9d0 con 0x7f52bc0ff1f0 2024-01-31T05:31:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52b27fc640 1 -- 172.21.15.45:0/3871795765 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8003d40 con 0x7f52bc0ff1f0 2024-01-31T05:31:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.928+0000 7f52c32df640 1 -- 172.21.15.45:0/3871795765 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52bc1a3d20 con 0x7f52bc0ff1f0 2024-01-31T05:31:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.930+0000 7f52b27fc640 1 -- 172.21.15.45:0/3871795765 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f52b803b040 con 0x7f52bc0ff1f0 2024-01-31T05:31:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.931+0000 7f52c6181740 1 Processor -- start 2024-01-31T05:31:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.931+0000 7f52c6181740 1 -- start start 2024-01-31T05:31:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.931+0000 7f52c6181740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da060 0x562d704da440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.931+0000 7f52c6181740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 0x562d704e3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.931+0000 7f52c6181740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704e3750 0x562d704e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.931+0000 7f52c6181740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d70449040 con 0x562d704da980 2024-01-31T05:31:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.931+0000 7f52c6181740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d70446730 con 0x562d704e3750 2024-01-31T05:31:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.931+0000 7f52c6181740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d70442020 con 0x562d704da060 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.931+0000 7f52b27fc640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f52a80a14f0 0x7f52a80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.931+0000 7f52c107b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 0x562d704e3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.931+0000 7f52c187c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da060 0x562d704da440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.931+0000 7f52c107b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 0x562d704e3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42526/0 (socket says 172.21.15.45:42526) 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52c107b640 1 -- 172.21.15.45:0/2301708303 learned_addr learned my addr 172.21.15.45:0/2301708303 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52c187c640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f52a80a14f0 0x7f52a80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52c087a640 1 --2- 172.21.15.45:0/2301708303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704e3750 0x562d704e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52b27fc640 1 -- 172.21.15.45:0/3871795765 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f52b80ee830 con 0x7f52bc0ff1f0 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52c187c640 1 -- 172.21.15.45:0/2301708303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704e3750 msgr2=0x562d704e5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52c187c640 1 --2- 172.21.15.45:0/2301708303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704e3750 0x562d704e5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52c187c640 1 -- 172.21.15.45:0/2301708303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 msgr2=0x562d704e3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52c187c640 1 --2- 172.21.15.45:0/2301708303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 0x562d704e3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52c187c640 1 -- 172.21.15.45:0/2301708303 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d70439e40 con 0x562d704da060 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52c107b640 1 --2- 172.21.15.45:0/2301708303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 0x562d704e3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52c187c640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f52a80a14f0 0x7f52a80a39b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f52b40099e0 tx=0x7f52b401e000 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52c187c640 1 --2- 172.21.15.45:0/2301708303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da060 0x562d704da440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f52b400f510 tx=0x7f52b400f9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b83ed8536ae00cd1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52a77fe640 1 -- 172.21.15.45:0/2301708303 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400ee70 con 0x562d704da060 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52a77fe640 1 -- 172.21.15.45:0/2301708303 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b4003e30 con 0x562d704da060 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.932+0000 7f52a77fe640 1 -- 172.21.15.45:0/2301708303 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4026450 con 0x562d704da060 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.933+0000 7f52c6181740 1 -- 172.21.15.45:0/2301708303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da060 msgr2=0x562d704da440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.933+0000 7f52c6181740 1 --2- 172.21.15.45:0/2301708303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da060 0x562d704da440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f52b400f510 tx=0x7f52b400f9d0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.933+0000 7f52c6181740 1 -- 172.21.15.45:0/2301708303 shutdown_connections 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.933+0000 7f52c6181740 1 --2- 172.21.15.45:0/2301708303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 0x562d704e3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.933+0000 7f52c6181740 1 --2- 172.21.15.45:0/2301708303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704e3750 0x562d704e5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.933+0000 7f52c6181740 1 --2- 172.21.15.45:0/2301708303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da060 0x562d704da440 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.933+0000 7f52c6181740 1 -- 172.21.15.45:0/2301708303 >> 172.21.15.45:0/2301708303 conn(0x562d704d4980 msgr2=0x562d704d4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.933+0000 7f52c6181740 1 -- 172.21.15.45:0/2301708303 shutdown_connections 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.933+0000 7f52c6181740 1 -- 172.21.15.45:0/2301708303 wait complete. 2024-01-31T05:31:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c6181740 1 -- start start 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c6181740 1 Processor -- start 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c6181740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da060 0x562d704e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c6181740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 0x562d704e0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c6181740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704e3750 0x562d704e1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c6181740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d703fb1b0 con 0x562d704da980 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c6181740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d703fae50 con 0x562d704e3750 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c6181740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d703c0090 con 0x562d704da060 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c6181740 10 client.0 _ll_register_callbacks cb 0x562d703e5a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c087a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 0x562d704e0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c187c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704e3750 0x562d704e1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c107b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da060 0x562d704e06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c087a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 0x562d704e0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42540/0 (socket says 172.21.15.45:42540) 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c107b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da060 0x562d704e06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:57620/0 (socket says 172.21.15.45:57620) 2024-01-31T05:31:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c187c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704e3750 0x562d704e1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:49610/0 (socket says 172.21.15.45:49610) 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.934+0000 7f52c087a640 1 -- 172.21.15.45:0/2858826758 learned_addr learned my addr 172.21.15.45:0/2858826758 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52c187c640 1 -- 172.21.15.45:0/2858826758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da060 msgr2=0x562d704e06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52c187c640 1 --2- 172.21.15.45:0/2858826758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da060 0x562d704e06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52c187c640 1 -- 172.21.15.45:0/2858826758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 msgr2=0x562d704e0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52c187c640 1 --2- 172.21.15.45:0/2858826758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 0x562d704e0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52c187c640 1 -- 172.21.15.45:0/2858826758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b4019040 con 0x562d704e3750 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52c087a640 1 --2- 172.21.15.45:0/2858826758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 0x562d704e0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52c187c640 1 --2- 172.21.15.45:0/2858826758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704e3750 0x562d704e1160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52b400fed0 tx=0x7f52b4004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8248c39ee77e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2858826758 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4011580 con 0x562d704e3750 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2858826758 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f52b4004650 con 0x562d704e3750 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52c6181740 1 -- 172.21.15.45:0/2858826758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d7043b010 con 0x562d704e3750 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2858826758 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52b4004810 con 0x562d704e3750 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52c6181740 10 client.25622 fetch_fsmap learned FSMap version 28 2024-01-31T05:31:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.935+0000 7f52c6181740 1 -- 172.21.15.45:0/2858826758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562d704406a0 con 0x562d704e3750 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.936+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2858826758 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d7043b010 con 0x562d704e3750 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.936+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2858826758 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f52b402d070 con 0x562d704e3750 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.936+0000 7f52c6181740 10 client.25622 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.936+0000 7f52c6181740 10 client.25622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.936+0000 7f52c6181740 20 client.25622 populate_metadata read hostname 'smithi045' 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.936+0000 7f52c6181740 1 --2- 172.21.15.45:0/2858826758 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562d704f3550 0x562d704f5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.936+0000 7f52c6181740 4 client.25622 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.936+0000 7f52c6181740 1 -- 172.21.15.45:0/2858826758 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d7043ad30 con 0x562d704f3550 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.936+0000 7f52c107b640 1 --2- 172.21.15.45:0/2858826758 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562d704f3550 0x562d704f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.936+0000 7f52c107b640 1 --2- 172.21.15.45:0/2858826758 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562d704f3550 0x562d704f5970 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eada2c3787ce132 server_cookie=808f6e5e8ef5a61 in_seq=0 out_seq=0 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.936+0000 7f52a4ff9640 10 client.25622 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2858826758 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d7043ad30 con 0x562d704f3550 2024-01-31T05:31:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52a4ff9640 10 client.25622 handle_command_reply: tid=0 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52c6181740 1 client.25622 shutdown 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52c6181740 1 -- 172.21.15.45:0/2858826758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704e3750 msgr2=0x562d704e1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52c6181740 1 --2- 172.21.15.45:0/2858826758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704e3750 0x562d704e1160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52b400fed0 tx=0x7f52b4004430 comp rx=0 tx=0).stop 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52c6181740 1 -- 172.21.15.45:0/2858826758 shutdown_connections 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52c6181740 1 --2- 172.21.15.45:0/2858826758 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562d704f3550 0x562d704f5970 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52c6181740 1 --2- 172.21.15.45:0/2858826758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da980 0x562d704e0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52c6181740 1 --2- 172.21.15.45:0/2858826758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704e3750 0x562d704e1160 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52c6181740 1 --2- 172.21.15.45:0/2858826758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da060 0x562d704e06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52a4ff9640 0 client.25622 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52c6181740 1 -- 172.21.15.45:0/2858826758 >> 172.21.15.45:0/2858826758 conn(0x562d704d4980 msgr2=0x562d704d66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52c6181740 1 -- 172.21.15.45:0/2858826758 shutdown_connections 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52c6181740 1 -- 172.21.15.45:0/2858826758 wait complete. 2024-01-31T05:31:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.937+0000 7f52c6181740 20 client.25622 trim_cache size 0 max 16384 2024-01-31T05:31:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c6181740 1 Processor -- start 2024-01-31T05:31:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c6181740 1 -- start start 2024-01-31T05:31:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c6181740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704da060 0x562d704e2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c6181740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da980 0x562d704e0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c6181740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704e3750 0x562d704defb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c6181740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d704d00b0 con 0x562d704e3750 2024-01-31T05:31:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c6181740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7043ad30 con 0x562d704da060 2024-01-31T05:31:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c6181740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7043b010 con 0x562d704da980 2024-01-31T05:31:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c187c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704e3750 0x562d704defb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c107b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704da060 0x562d704e2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c187c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704e3750 0x562d704defb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42542/0 (socket says 172.21.15.45:42542) 2024-01-31T05:31:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c187c640 1 -- 172.21.15.45:0/2147264132 learned_addr learned my addr 172.21.15.45:0/2147264132 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c087a640 1 --2- 172.21.15.45:0/2147264132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da980 0x562d704e0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c187c640 1 -- 172.21.15.45:0/2147264132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da980 msgr2=0x562d704e0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c187c640 1 --2- 172.21.15.45:0/2147264132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da980 0x562d704e0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c187c640 1 -- 172.21.15.45:0/2147264132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704da060 msgr2=0x562d704e2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c187c640 1 --2- 172.21.15.45:0/2147264132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704da060 0x562d704e2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c187c640 1 -- 172.21.15.45:0/2147264132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d703dd030 con 0x562d704e3750 2024-01-31T05:31:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.951+0000 7f52c107b640 1 --2- 172.21.15.45:0/2147264132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704da060 0x562d704e2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:31:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.952+0000 7f52c187c640 1 --2- 172.21.15.45:0/2147264132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704e3750 0x562d704defb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f52b400f3e0 tx=0x7f52b4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a79251b051ab72 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.952+0000 7f52a6ffd640 1 -- 172.21.15.45:0/2147264132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4005a10 con 0x562d704e3750 2024-01-31T05:31:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.952+0000 7f52a6ffd640 1 -- 172.21.15.45:0/2147264132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b40096c0 con 0x562d704e3750 2024-01-31T05:31:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.952+0000 7f52a6ffd640 1 -- 172.21.15.45:0/2147264132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4011280 con 0x562d704e3750 2024-01-31T05:31:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.952+0000 7f52c6181740 1 -- 172.21.15.45:0/2147264132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704e3750 msgr2=0x562d704defb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.952+0000 7f52c6181740 1 --2- 172.21.15.45:0/2147264132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704e3750 0x562d704defb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f52b400f3e0 tx=0x7f52b4009500 comp rx=0 tx=0).stop 2024-01-31T05:31:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 -- 172.21.15.45:0/2147264132 shutdown_connections 2024-01-31T05:31:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 --2- 172.21.15.45:0/2147264132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704e3750 0x562d704defb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 --2- 172.21.15.45:0/2147264132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704da980 0x562d704e0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 --2- 172.21.15.45:0/2147264132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704da060 0x562d704e2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 -- 172.21.15.45:0/2147264132 >> 172.21.15.45:0/2147264132 conn(0x562d704d4980 msgr2=0x562d704fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 -- 172.21.15.45:0/2147264132 shutdown_connections 2024-01-31T05:31:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 -- 172.21.15.45:0/2147264132 wait complete. 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 -- start start 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 Processor -- start 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da060 0x562d704e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704da980 0x562d704e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704e3750 0x562d70548a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d70548f90 con 0x562d704da060 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d705490d0 con 0x562d704da980 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d70549210 con 0x562d704e3750 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c6181740 10 client.0 _ll_register_callbacks cb 0x7f52b4011760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c107b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da060 0x562d704e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c107b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da060 0x562d704e8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42546/0 (socket says 172.21.15.45:42546) 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.953+0000 7f52c107b640 1 -- 172.21.15.45:0/2450575510 learned_addr learned my addr 172.21.15.45:0/2450575510 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52c087a640 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704da980 0x562d704e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52c187c640 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704e3750 0x562d70548a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52c107b640 1 -- 172.21.15.45:0/2450575510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704e3750 msgr2=0x562d70548a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52c107b640 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704e3750 0x562d70548a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52c107b640 1 -- 172.21.15.45:0/2450575510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704da980 msgr2=0x562d704e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52c107b640 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704da980 0x562d704e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52c107b640 1 -- 172.21.15.45:0/2450575510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b8009490 con 0x562d704da060 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52c087a640 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704da980 0x562d704e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52c107b640 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da060 0x562d704e8610 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f52b80bb720 tx=0x7f52b80037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63570ab9c1a7b443 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2450575510 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b80035f0 con 0x562d704da060 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2450575510 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f52b80bb790 con 0x562d704da060 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52c6181740 1 -- 172.21.15.45:0/2450575510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d70141600 con 0x562d704da060 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2450575510 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52b80bf830 con 0x562d704da060 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52c6181740 10 client.16119 fetch_fsmap learned FSMap version 28 2024-01-31T05:31:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52c6181740 1 -- 172.21.15.45:0/2450575510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562d704406a0 con 0x562d704da060 2024-01-31T05:31:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.954+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2450575510 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d70141600 con 0x562d704da060 2024-01-31T05:31:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.955+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2450575510 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f52b80ec230 con 0x562d704da060 2024-01-31T05:31:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.955+0000 7f52c6181740 10 client.16119 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:31:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.955+0000 7f52c6181740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:31:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.955+0000 7f52c6181740 20 client.16119 populate_metadata read hostname 'smithi045' 2024-01-31T05:31:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.955+0000 7f52c6181740 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562d7054a670 0x562d7054ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.955+0000 7f52c6181740 4 client.16119 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:31:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.955+0000 7f52c6181740 1 -- 172.21.15.45:0/2450575510 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562d70194c50 con 0x562d7054a670 2024-01-31T05:31:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.955+0000 7f52c087a640 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562d7054a670 0x562d7054ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.955+0000 7f52a4ff9640 10 client.16119 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.955+0000 7f52c087a640 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562d7054a670 0x562d7054ca90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e15d0043bde7e2 server_cookie=8a53e8356c16d966 in_seq=0 out_seq=0 2024-01-31T05:31:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.970+0000 7f52a4ff9640 1 -- 172.21.15.45:0/2450575510 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562d70194c50 con 0x562d7054a670 2024-01-31T05:31:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.971+0000 7f52a4ff9640 10 client.16119 handle_command_reply: tid=0 2024-01-31T05:31:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.971+0000 7f52c6181740 1 client.16119 shutdown 2024-01-31T05:31:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.971+0000 7f52c6181740 1 -- 172.21.15.45:0/2450575510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da060 msgr2=0x562d704e8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.971+0000 7f52c6181740 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da060 0x562d704e8610 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f52b80bb720 tx=0x7f52b80037d0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.971+0000 7f52c6181740 1 -- 172.21.15.45:0/2450575510 shutdown_connections 2024-01-31T05:31:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.971+0000 7f52c6181740 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562d7054a670 0x562d7054ca90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.971+0000 7f52c6181740 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562d704e3750 0x562d70548a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.971+0000 7f52c6181740 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562d704da980 0x562d704e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.971+0000 7f52c6181740 1 --2- 172.21.15.45:0/2450575510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d704da060 0x562d704e8610 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.971+0000 7f52c6181740 1 -- 172.21.15.45:0/2450575510 >> 172.21.15.45:0/2450575510 conn(0x562d704d4980 msgr2=0x562d704deef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.971+0000 7f52a4ff9640 0 client.16119 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.971+0000 7f52c6181740 1 -- 172.21.15.45:0/2450575510 shutdown_connections 2024-01-31T05:31:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.972+0000 7f52c6181740 1 -- 172.21.15.45:0/2450575510 wait complete. 2024-01-31T05:31:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:31:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:31:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:31:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:31:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.972+0000 7f52c6181740 20 client.16119 trim_cache size 0 max 16384 2024-01-31T05:31:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.972+0000 7f52c32df640 1 -- 172.21.15.45:0/3871795765 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f52a80a14f0 msgr2=0x7f52a80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.972+0000 7f52c32df640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f52a80a14f0 0x7f52a80a39b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f52b40099e0 tx=0x7f52b401e000 comp rx=0 tx=0).stop 2024-01-31T05:31:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.972+0000 7f52c32df640 1 -- 172.21.15.45:0/3871795765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52bc0ff1f0 msgr2=0x7f52bc197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.972+0000 7f52c32df640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52bc0ff1f0 0x7f52bc197200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f52b8037040 tx=0x7f52b8004340 comp rx=0 tx=0).stop 2024-01-31T05:31:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.973+0000 7f52c32df640 1 -- 172.21.15.45:0/3871795765 shutdown_connections 2024-01-31T05:31:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.973+0000 7f52c32df640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f52a80a14f0 0x7f52a80a39b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.973+0000 7f52c32df640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52bc105cd0 0x7f52bc19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.973+0000 7f52c32df640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52bc0ffbb0 0x7f52bc197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.973+0000 7f52c32df640 1 --2- 172.21.15.45:0/3871795765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52bc0ff1f0 0x7f52bc197200 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.973+0000 7f52c32df640 1 -- 172.21.15.45:0/3871795765 >> 172.21.15.45:0/3871795765 conn(0x7f52bc076180 msgr2=0x7f52bc103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.974+0000 7f52c32df640 1 -- 172.21.15.45:0/3871795765 shutdown_connections 2024-01-31T05:31:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:36.974+0000 7f52c32df640 1 -- 172.21.15.45:0/3871795765 wait complete. 2024-01-31T05:31:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:36 smithi156 ceph-mon[35720]: pgmap v280: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 176 op/s 2024-01-31T05:31:37.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:36 smithi186 ceph-mon[35659]: pgmap v280: 97 pgs: 97 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 176 op/s 2024-01-31T05:31:37.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 956109 11.13 MB/sec execute 337 sec latency 25.474 ms 2024-01-31T05:31:38.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 957520 11.11 MB/sec execute 338 sec latency 163.772 ms 2024-01-31T05:31:38.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:38 smithi045 ceph-mon[31585]: pgmap v281: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-01-31T05:31:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:38 smithi156 ceph-mon[35720]: pgmap v281: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-01-31T05:31:39.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:38 smithi186 ceph-mon[35659]: pgmap v281: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-01-31T05:31:39.792 INFO:tasks.workunit.client.0.smithi045.stdout: 1 959678 11.12 MB/sec execute 339 sec latency 19.809 ms 2024-01-31T05:31:40.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 961753 11.11 MB/sec execute 340 sec latency 18.754 ms 2024-01-31T05:31:40.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:40 smithi045 ceph-mon[31585]: pgmap v282: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 182 op/s 2024-01-31T05:31:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:40 smithi156 ceph-mon[35720]: pgmap v282: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 182 op/s 2024-01-31T05:31:41.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:40 smithi186 ceph-mon[35659]: pgmap v282: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 182 op/s 2024-01-31T05:31:41.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 963432 11.11 MB/sec execute 341 sec latency 47.413 ms 2024-01-31T05:31:42.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 965757 11.11 MB/sec execute 342 sec latency 25.867 ms 2024-01-31T05:31:42.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:42 smithi045 ceph-mon[31585]: pgmap v283: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 182 op/s 2024-01-31T05:31:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:42 smithi156 ceph-mon[35720]: pgmap v283: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 182 op/s 2024-01-31T05:31:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:42 smithi186 ceph-mon[35659]: pgmap v283: 97 pgs: 97 active+clean; 527 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 182 op/s 2024-01-31T05:31:43.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 967355 11.10 MB/sec execute 343 sec latency 60.977 ms 2024-01-31T05:31:44.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 969923 11.12 MB/sec execute 344 sec latency 25.314 ms 2024-01-31T05:31:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:44 smithi045 ceph-mon[31585]: pgmap v284: 97 pgs: 97 active+clean; 537 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 265 op/s 2024-01-31T05:31:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:44 smithi156 ceph-mon[35720]: pgmap v284: 97 pgs: 97 active+clean; 537 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 265 op/s 2024-01-31T05:31:45.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:44 smithi186 ceph-mon[35659]: pgmap v284: 97 pgs: 97 active+clean; 537 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 265 op/s 2024-01-31T05:31:45.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 971592 11.11 MB/sec execute 345 sec latency 17.261 ms 2024-01-31T05:31:46.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 973839 11.11 MB/sec execute 346 sec latency 41.640 ms 2024-01-31T05:31:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:46 smithi045 ceph-mon[31585]: pgmap v285: 97 pgs: 97 active+clean; 537 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 174 op/s 2024-01-31T05:31:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:46 smithi156 ceph-mon[35720]: pgmap v285: 97 pgs: 97 active+clean; 537 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 174 op/s 2024-01-31T05:31:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:46 smithi186 ceph-mon[35659]: pgmap v285: 97 pgs: 97 active+clean; 537 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 174 op/s 2024-01-31T05:31:47.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 975755 11.10 MB/sec execute 347 sec latency 48.143 ms 2024-01-31T05:31:48.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 977605 11.11 MB/sec execute 348 sec latency 15.991 ms 2024-01-31T05:31:48.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:48 smithi045 ceph-mon[31585]: pgmap v286: 97 pgs: 97 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 264 op/s 2024-01-31T05:31:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:48 smithi156 ceph-mon[35720]: pgmap v286: 97 pgs: 97 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 264 op/s 2024-01-31T05:31:49.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:48 smithi186 ceph-mon[35659]: pgmap v286: 97 pgs: 97 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 264 op/s 2024-01-31T05:31:49.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 979779 11.11 MB/sec execute 349 sec latency 62.722 ms 2024-01-31T05:31:49.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:49 smithi156 ceph-mon[35720]: pgmap v287: 97 pgs: 97 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 172 op/s 2024-01-31T05:31:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.066+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1889156417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c104b70 msgr2=0x7f1e2c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.066+0000 7f1e33e3e640 1 --2- 172.21.15.45:0/1889156417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c104b70 0x7f1e2c104f50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009900 tx=0x7f1e2802bc30 comp rx=0 tx=0).stop 2024-01-31T05:31:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.066+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1889156417 shutdown_connections 2024-01-31T05:31:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.066+0000 7f1e33e3e640 1 --2- 172.21.15.45:0/1889156417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c1007e0 0x7f1e2c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.066+0000 7f1e33e3e640 1 --2- 172.21.15.45:0/1889156417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e2c0ffe40 0x7f1e2c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.066+0000 7f1e33e3e640 1 --2- 172.21.15.45:0/1889156417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c104b70 0x7f1e2c104f50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.066+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1889156417 >> 172.21.15.45:0/1889156417 conn(0x7f1e2c0769c0 msgr2=0x7f1e2c076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.066+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1889156417 shutdown_connections 2024-01-31T05:31:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.066+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1889156417 wait complete. 2024-01-31T05:31:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e33e3e640 1 Processor -- start 2024-01-31T05:31:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e33e3e640 1 -- start start 2024-01-31T05:31:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e33e3e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c0ffe40 0x7f1e2c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e33e3e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c1007e0 0x7f1e2c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e33e3e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e2c104b70 0x7f1e2c1a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e33e3e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c198db0 con 0x7f1e2c0ffe40 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e33e3e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c198f20 con 0x7f1e2c1007e0 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e33e3e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c1a3690 con 0x7f1e2c104b70 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e31bda640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c0ffe40 0x7f1e2c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e31bda640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c0ffe40 0x7f1e2c198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38276/0 (socket says 172.21.15.45:38276) 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e31bda640 1 -- 172.21.15.45:0/1739504100 learned_addr learned my addr 172.21.15.45:0/1739504100 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e313d9640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c1007e0 0x7f1e2c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.067+0000 7f1e323db640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e2c104b70 0x7f1e2c1a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.068+0000 7f1e31bda640 1 -- 172.21.15.45:0/1739504100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e2c104b70 msgr2=0x7f1e2c1a3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.068+0000 7f1e31bda640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e2c104b70 0x7f1e2c1a3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.068+0000 7f1e31bda640 1 -- 172.21.15.45:0/1739504100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c1007e0 msgr2=0x7f1e2c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.068+0000 7f1e31bda640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c1007e0 0x7f1e2c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.068+0000 7f1e31bda640 1 -- 172.21.15.45:0/1739504100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e28009560 con 0x7f1e2c0ffe40 2024-01-31T05:31:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.068+0000 7f1e31bda640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c0ffe40 0x7f1e2c198080 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1e28006010 tx=0x7f1e28005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.068+0000 7f1e22ffd640 1 -- 172.21.15.45:0/1739504100 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28004250 con 0x7f1e2c0ffe40 2024-01-31T05:31:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.068+0000 7f1e22ffd640 1 -- 172.21.15.45:0/1739504100 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e280043b0 con 0x7f1e2c0ffe40 2024-01-31T05:31:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.068+0000 7f1e22ffd640 1 -- 172.21.15.45:0/1739504100 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28003ba0 con 0x7f1e2c0ffe40 2024-01-31T05:31:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.068+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1739504100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2c1a3910 con 0x7f1e2c0ffe40 2024-01-31T05:31:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.068+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1739504100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2c1a3dd0 con 0x7f1e2c0ffe40 2024-01-31T05:31:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.070+0000 7f1e22ffd640 1 -- 172.21.15.45:0/1739504100 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f1e28003d00 con 0x7f1e2c0ffe40 2024-01-31T05:31:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e36c00740 1 Processor -- start 2024-01-31T05:31:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e36c00740 1 -- start start 2024-01-31T05:31:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e22ffd640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1e180a14f0 0x7f1e180a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e36c00740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51060 0x5594a7e51440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e36c00740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51980 0x5594a7e5a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e36c00740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7e5cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e36c00740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a7dc0040 con 0x5594a7e5a750 2024-01-31T05:31:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e36c00740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a7dbd730 con 0x5594a7e51060 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e36c00740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a7db9020 con 0x5594a7e51980 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e323db640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1e180a14f0 0x7f1e180a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e22ffd640 1 -- 172.21.15.45:0/1739504100 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1e280ef550 con 0x7f1e2c0ffe40 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e313d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51980 0x5594a7e5a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e323db640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7e5cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e313d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51980 0x5594a7e5a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:43586/0 (socket says 172.21.15.45:43586) 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e313d9640 1 -- 172.21.15.45:0/2769758924 learned_addr learned my addr 172.21.15.45:0/2769758924 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.071+0000 7f1e31bda640 1 --2- 172.21.15.45:0/2769758924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51060 0x5594a7e51440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.072+0000 7f1e313d9640 1 -- 172.21.15.45:0/2769758924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51060 msgr2=0x5594a7e51440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.072+0000 7f1e313d9640 1 --2- 172.21.15.45:0/2769758924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51060 0x5594a7e51440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.072+0000 7f1e313d9640 1 -- 172.21.15.45:0/2769758924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 msgr2=0x5594a7e5cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.072+0000 7f1e313d9640 1 --2- 172.21.15.45:0/2769758924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7e5cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.072+0000 7f1e313d9640 1 -- 172.21.15.45:0/2769758924 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594a7db0e40 con 0x5594a7e51980 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.072+0000 7f1e323db640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1e180a14f0 0x7f1e180a39b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e24005fd0 tx=0x7f1e24009e70 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.072+0000 7f1e313d9640 1 --2- 172.21.15.45:0/2769758924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51980 0x5594a7e5a210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009ca0 tx=0x7f1e1c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcebeceed6749fa8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.072+0000 7f1e17fff640 1 -- 172.21.15.45:0/2769758924 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c019630 con 0x5594a7e51980 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.072+0000 7f1e17fff640 1 -- 172.21.15.45:0/2769758924 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e1c019c50 con 0x5594a7e51980 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.072+0000 7f1e17fff640 1 -- 172.21.15.45:0/2769758924 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c00ea30 con 0x5594a7e51980 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.073+0000 7f1e36c00740 1 -- 172.21.15.45:0/2769758924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51980 msgr2=0x5594a7e5a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.073+0000 7f1e36c00740 1 --2- 172.21.15.45:0/2769758924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51980 0x5594a7e5a210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009ca0 tx=0x7f1e1c011810 comp rx=0 tx=0).stop 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.073+0000 7f1e36c00740 1 -- 172.21.15.45:0/2769758924 shutdown_connections 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.073+0000 7f1e36c00740 1 --2- 172.21.15.45:0/2769758924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7e5cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.073+0000 7f1e36c00740 1 --2- 172.21.15.45:0/2769758924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51980 0x5594a7e5a210 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.073+0000 7f1e36c00740 1 --2- 172.21.15.45:0/2769758924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51060 0x5594a7e51440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.073+0000 7f1e36c00740 1 -- 172.21.15.45:0/2769758924 >> 172.21.15.45:0/2769758924 conn(0x5594a7e4b980 msgr2=0x5594a7e4bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.073+0000 7f1e36c00740 1 -- 172.21.15.45:0/2769758924 shutdown_connections 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.073+0000 7f1e36c00740 1 -- 172.21.15.45:0/2769758924 wait complete. 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.073+0000 7f1e36c00740 1 -- start start 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.073+0000 7f1e36c00740 1 Processor -- start 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e36c00740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 0x5594a7e57700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e36c00740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e51980 0x5594a7e57c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e36c00740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e5a750 0x5594a7e58180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e36c00740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a7d721b0 con 0x5594a7e51980 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e36c00740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a7d71e50 con 0x5594a7e5a750 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e36c00740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a7d37090 con 0x5594a7e51060 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e36c00740 10 client.0 _ll_register_callbacks cb 0x5594a7d5ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e313d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e51980 0x5594a7e57c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e31bda640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 0x5594a7e57700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e313d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e51980 0x5594a7e57c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38288/0 (socket says 172.21.15.45:38288) 2024-01-31T05:31:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e313d9640 1 -- 172.21.15.45:0/1321852057 learned_addr learned my addr 172.21.15.45:0/1321852057 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e323db640 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e5a750 0x5594a7e58180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e313d9640 1 -- 172.21.15.45:0/1321852057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 msgr2=0x5594a7e57700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e313d9640 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 0x5594a7e57700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e313d9640 1 -- 172.21.15.45:0/1321852057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e5a750 msgr2=0x5594a7e58180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e313d9640 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e5a750 0x5594a7e58180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e313d9640 1 -- 172.21.15.45:0/1321852057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c0098d0 con 0x5594a7e51980 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e31bda640 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 0x5594a7e57700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e313d9640 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e51980 0x5594a7e57c40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c011d10 tx=0x7f1e1c0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3357b2fcb8242f4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.074+0000 7f1e157fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e157fa640 1 -- 172.21.15.45:0/1321852057 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c019840 con 0x5594a7e51980 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e157fa640 1 -- 172.21.15.45:0/1321852057 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1e1c019dc0 con 0x5594a7e51980 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e36c00740 1 -- 172.21.15.45:0/1321852057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594a7db2010 con 0x5594a7e51980 2024-01-31T05:31:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e157fa640 1 -- 172.21.15.45:0/1321852057 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e1c005c50 con 0x5594a7e51980 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e36c00740 10 client.16131 fetch_fsmap learned FSMap version 28 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e36c00740 1 -- 172.21.15.45:0/1321852057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5594a7db76a0 con 0x5594a7e51980 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e157fa640 1 -- 172.21.15.45:0/1321852057 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5594a7db2010 con 0x5594a7e51980 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e157fa640 1 -- 172.21.15.45:0/1321852057 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1e1c004880 con 0x5594a7e51980 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e36c00740 10 client.16131 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e36c00740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e36c00740 20 client.16131 populate_metadata read hostname 'smithi045' 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e36c00740 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5594a7e6a3d0 0x5594a7e6c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e36c00740 4 client.16131 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e36c00740 1 -- 172.21.15.45:0/1321852057 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594a7db1d30 con 0x5594a7e6a3d0 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.075+0000 7f1e31bda640 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5594a7e6a3d0 0x5594a7e6c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.076+0000 7f1e31bda640 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5594a7e6a3d0 0x5594a7e6c7e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f02a5e1f1e2480 server_cookie=673a7e9d45b4a283 in_seq=0 out_seq=0 2024-01-31T05:31:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.076+0000 7f1e157fa640 10 client.16131 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.076+0000 7f1e157fa640 1 -- 172.21.15.45:0/1321852057 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5594a7db1d30 con 0x5594a7e6a3d0 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.076+0000 7f1e157fa640 10 client.16131 handle_command_reply: tid=0 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.076+0000 7f1e36c00740 1 client.16131 shutdown 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.077+0000 7f1e36c00740 1 -- 172.21.15.45:0/1321852057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e51980 msgr2=0x5594a7e57c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.077+0000 7f1e36c00740 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e51980 0x5594a7e57c40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c011d10 tx=0x7f1e1c0056e0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.077+0000 7f1e36c00740 1 -- 172.21.15.45:0/1321852057 shutdown_connections 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.077+0000 7f1e36c00740 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5594a7e6a3d0 0x5594a7e6c7e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.077+0000 7f1e36c00740 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e51980 0x5594a7e57c40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.077+0000 7f1e36c00740 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e5a750 0x5594a7e58180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.077+0000 7f1e157fa640 0 client.16131 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.077+0000 7f1e36c00740 1 --2- 172.21.15.45:0/1321852057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 0x5594a7e57700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.077+0000 7f1e36c00740 1 -- 172.21.15.45:0/1321852057 >> 172.21.15.45:0/1321852057 conn(0x5594a7e4b980 msgr2=0x5594a7e4d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.077+0000 7f1e36c00740 1 -- 172.21.15.45:0/1321852057 shutdown_connections 2024-01-31T05:31:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.077+0000 7f1e36c00740 1 -- 172.21.15.45:0/1321852057 wait complete. 2024-01-31T05:31:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.077+0000 7f1e36c00740 20 client.16131 trim_cache size 0 max 16384 2024-01-31T05:31:50.096 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:49 smithi186 ceph-mon[35659]: pgmap v287: 97 pgs: 97 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 172 op/s 2024-01-31T05:31:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e36c00740 1 Processor -- start 2024-01-31T05:31:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e36c00740 1 -- start start 2024-01-31T05:31:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e36c00740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51060 0x5594a7e59e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e36c00740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51980 0x5594a7e57fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e36c00740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7e54960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e36c00740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a7e67050 con 0x5594a7e5a750 2024-01-31T05:31:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e36c00740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a7e5cb40 con 0x5594a7e51060 2024-01-31T05:31:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e36c00740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a7e52550 con 0x5594a7e51980 2024-01-31T05:31:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e323db640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7e54960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e323db640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7e54960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38294/0 (socket says 172.21.15.45:38294) 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e323db640 1 -- 172.21.15.45:0/2436138043 learned_addr learned my addr 172.21.15.45:0/2436138043 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e313d9640 1 --2- 172.21.15.45:0/2436138043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51980 0x5594a7e57fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.090+0000 7f1e31bda640 1 --2- 172.21.15.45:0/2436138043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51060 0x5594a7e59e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.091+0000 7f1e323db640 1 -- 172.21.15.45:0/2436138043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51980 msgr2=0x5594a7e57fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.091+0000 7f1e323db640 1 --2- 172.21.15.45:0/2436138043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51980 0x5594a7e57fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.091+0000 7f1e323db640 1 -- 172.21.15.45:0/2436138043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51060 msgr2=0x5594a7e59e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.091+0000 7f1e323db640 1 --2- 172.21.15.45:0/2436138043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51060 0x5594a7e59e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.091+0000 7f1e323db640 1 -- 172.21.15.45:0/2436138043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594a7d54030 con 0x5594a7e5a750 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.091+0000 7f1e31bda640 1 --2- 172.21.15.45:0/2436138043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51060 0x5594a7e59e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.091+0000 7f1e323db640 1 --2- 172.21.15.45:0/2436138043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7e54960 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1e24011650 tx=0x7f1e24021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f21685f7815f90 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.091+0000 7f1e177fe640 1 -- 172.21.15.45:0/2436138043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e24016070 con 0x5594a7e5a750 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.091+0000 7f1e177fe640 1 -- 172.21.15.45:0/2436138043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e24004040 con 0x5594a7e5a750 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.091+0000 7f1e177fe640 1 -- 172.21.15.45:0/2436138043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e24029680 con 0x5594a7e5a750 2024-01-31T05:31:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 -- 172.21.15.45:0/2436138043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 msgr2=0x5594a7e54960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 --2- 172.21.15.45:0/2436138043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7e54960 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1e24011650 tx=0x7f1e24021000 comp rx=0 tx=0).stop 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 -- 172.21.15.45:0/2436138043 shutdown_connections 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 --2- 172.21.15.45:0/2436138043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7e54960 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 --2- 172.21.15.45:0/2436138043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51980 0x5594a7e57fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 --2- 172.21.15.45:0/2436138043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51060 0x5594a7e59e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 -- 172.21.15.45:0/2436138043 >> 172.21.15.45:0/2436138043 conn(0x5594a7e4b980 msgr2=0x5594a7e529f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 -- 172.21.15.45:0/2436138043 shutdown_connections 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 -- 172.21.15.45:0/2436138043 wait complete. 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 -- start start 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 Processor -- start 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 0x5594a7e60700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51980 0x5594a7e60c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7ec0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a7e613e0 con 0x5594a7e5a750 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a7ec0e80 con 0x5594a7e51980 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a7ec1040 con 0x5594a7e51060 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.092+0000 7f1e36c00740 10 client.0 _ll_register_callbacks cb 0x7f1e1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e323db640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7ec0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e31bda640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 0x5594a7e60700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e313d9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51980 0x5594a7e60c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e31bda640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 0x5594a7e60700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:43618/0 (socket says 172.21.15.45:43618) 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e313d9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51980 0x5594a7e60c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:57340/0 (socket says 172.21.15.45:57340) 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e323db640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7ec0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38296/0 (socket says 172.21.15.45:38296) 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e313d9640 1 -- 172.21.15.45:0/3575682652 learned_addr learned my addr 172.21.15.45:0/3575682652 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e313d9640 1 -- 172.21.15.45:0/3575682652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 msgr2=0x5594a7e60700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e313d9640 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 0x5594a7e60700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e313d9640 1 -- 172.21.15.45:0/3575682652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 msgr2=0x5594a7ec0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:31:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e313d9640 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7ec0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e313d9640 1 -- 172.21.15.45:0/3575682652 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e24019040 con 0x5594a7e51980 2024-01-31T05:31:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e31bda640 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 0x5594a7e60700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:31:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e323db640 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7ec0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:31:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e313d9640 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51980 0x5594a7e60c40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5594a7e56f60 tx=0x7f1e1c019870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c30e309b24fcb65f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:31:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e157fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:31:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e157fa640 1 -- 172.21.15.45:0/3575682652 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c004ee0 con 0x5594a7e51980 2024-01-31T05:31:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e36c00740 1 -- 172.21.15.45:0/3575682652 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e1c019c50 con 0x5594a7e51980 2024-01-31T05:31:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e157fa640 1 -- 172.21.15.45:0/3575682652 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1e1c005c50 con 0x5594a7e51980 2024-01-31T05:31:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e157fa640 1 -- 172.21.15.45:0/3575682652 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e1c004d60 con 0x5594a7e51980 2024-01-31T05:31:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e36c00740 10 client.25646 fetch_fsmap learned FSMap version 28 2024-01-31T05:31:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.093+0000 7f1e36c00740 1 -- 172.21.15.45:0/3575682652 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5594a7d79a50 con 0x5594a7e51980 2024-01-31T05:31:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.094+0000 7f1e157fa640 1 -- 172.21.15.45:0/3575682652 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e1c019c50 con 0x5594a7e51980 2024-01-31T05:31:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.094+0000 7f1e157fa640 1 -- 172.21.15.45:0/3575682652 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1e1c0035f0 con 0x5594a7e51980 2024-01-31T05:31:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.094+0000 7f1e36c00740 10 client.25646 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:31:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.094+0000 7f1e36c00740 10 client.25646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:31:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.094+0000 7f1e36c00740 20 client.25646 populate_metadata read hostname 'smithi045' 2024-01-31T05:31:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.094+0000 7f1e36c00740 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5594a7ec2340 0x5594a7ec4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:31:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.094+0000 7f1e36c00740 4 client.25646 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:31:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.094+0000 7f1e36c00740 1 -- 172.21.15.45:0/3575682652 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5594a7ab8600 con 0x5594a7ec2340 2024-01-31T05:31:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.094+0000 7f1e31bda640 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5594a7ec2340 0x5594a7ec4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:31:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.094+0000 7f1e31bda640 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5594a7ec2340 0x5594a7ec4730 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6540647db80af4e1 server_cookie=6facff2ece988b62 in_seq=0 out_seq=0 2024-01-31T05:31:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.094+0000 7f1e157fa640 10 client.25646 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e157fa640 1 -- 172.21.15.45:0/3575682652 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5594a7ab8600 con 0x5594a7ec2340 2024-01-31T05:31:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e157fa640 10 client.25646 handle_command_reply: tid=0 2024-01-31T05:31:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e36c00740 1 client.25646 shutdown 2024-01-31T05:31:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e36c00740 1 -- 172.21.15.45:0/3575682652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51980 msgr2=0x5594a7e60c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e36c00740 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51980 0x5594a7e60c40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5594a7e56f60 tx=0x7f1e1c019870 comp rx=0 tx=0).stop 2024-01-31T05:31:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e36c00740 1 -- 172.21.15.45:0/3575682652 shutdown_connections 2024-01-31T05:31:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e36c00740 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5594a7ec2340 0x5594a7ec4730 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e36c00740 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594a7e5a750 0x5594a7ec0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e36c00740 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594a7e51980 0x5594a7e60c40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e36c00740 1 --2- 172.21.15.45:0/3575682652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594a7e51060 0x5594a7e60700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e157fa640 0 client.25646 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:31:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e36c00740 1 -- 172.21.15.45:0/3575682652 >> 172.21.15.45:0/3575682652 conn(0x5594a7e4b980 msgr2=0x5594a7e4cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e36c00740 1 -- 172.21.15.45:0/3575682652 shutdown_connections 2024-01-31T05:31:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e36c00740 1 -- 172.21.15.45:0/3575682652 wait complete. 2024-01-31T05:31:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.106+0000 7f1e36c00740 20 client.25646 trim_cache size 0 max 16384 2024-01-31T05:31:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:31:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:31:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:31:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:31:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.107+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1739504100 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1e180a14f0 msgr2=0x7f1e180a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.107+0000 7f1e33e3e640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1e180a14f0 0x7f1e180a39b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e24005fd0 tx=0x7f1e24009e70 comp rx=0 tx=0).stop 2024-01-31T05:31:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.107+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1739504100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c0ffe40 msgr2=0x7f1e2c198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:31:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.107+0000 7f1e33e3e640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c0ffe40 0x7f1e2c198080 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1e28006010 tx=0x7f1e28005d90 comp rx=0 tx=0).stop 2024-01-31T05:31:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.107+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1739504100 shutdown_connections 2024-01-31T05:31:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.107+0000 7f1e33e3e640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1e180a14f0 0x7f1e180a39b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.107+0000 7f1e33e3e640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e2c104b70 0x7f1e2c1a3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.107+0000 7f1e33e3e640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c1007e0 0x7f1e2c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.107+0000 7f1e33e3e640 1 --2- 172.21.15.45:0/1739504100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c0ffe40 0x7f1e2c198080 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:31:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.107+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1739504100 >> 172.21.15.45:0/1739504100 conn(0x7f1e2c0769c0 msgr2=0x7f1e2c0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:31:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.107+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1739504100 shutdown_connections 2024-01-31T05:31:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:31:50.107+0000 7f1e33e3e640 1 -- 172.21.15.45:0/1739504100 wait complete. 2024-01-31T05:31:50.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:49 smithi045 ceph-mon[31585]: pgmap v287: 97 pgs: 97 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 172 op/s 2024-01-31T05:31:50.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 981626 11.10 MB/sec execute 350 sec latency 17.140 ms 2024-01-31T05:31:51.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 984195 11.12 MB/sec execute 351 sec latency 17.165 ms 2024-01-31T05:31:51.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:51 smithi156 ceph-mon[35720]: pgmap v288: 97 pgs: 97 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 172 op/s 2024-01-31T05:31:52.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:51 smithi186 ceph-mon[35659]: pgmap v288: 97 pgs: 97 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 172 op/s 2024-01-31T05:31:52.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:51 smithi045 ceph-mon[31585]: pgmap v288: 97 pgs: 97 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 172 op/s 2024-01-31T05:31:52.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 985561 11.10 MB/sec execute 352 sec latency 51.685 ms 2024-01-31T05:31:53.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 987724 11.11 MB/sec execute 353 sec latency 23.190 ms 2024-01-31T05:31:54.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 989036 11.09 MB/sec execute 354 sec latency 22.052 ms 2024-01-31T05:31:54.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:54 smithi045 ceph-mon[31585]: pgmap v289: 97 pgs: 97 active+clean; 543 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 251 op/s 2024-01-31T05:31:54.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:54 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:31:54.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:54 smithi156 ceph-mon[35720]: pgmap v289: 97 pgs: 97 active+clean; 543 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 251 op/s 2024-01-31T05:31:54.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:54 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:31:55.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:54 smithi186 ceph-mon[35659]: pgmap v289: 97 pgs: 97 active+clean; 543 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 251 op/s 2024-01-31T05:31:55.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:54 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:31:55.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 991312 11.10 MB/sec execute 355 sec latency 23.215 ms 2024-01-31T05:31:55.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:55 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:31:55.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:55 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:31:55.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:55 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:31:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:55 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:31:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:55 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:31:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:55 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:31:56.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:55 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:31:56.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:55 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:31:56.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:55 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:31:56.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 992719 11.09 MB/sec execute 356 sec latency 35.932 ms 2024-01-31T05:31:56.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:56 smithi045 ceph-mon[31585]: pgmap v290: 97 pgs: 97 active+clean; 543 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 169 op/s 2024-01-31T05:31:56.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:56 smithi156 ceph-mon[35720]: pgmap v290: 97 pgs: 97 active+clean; 543 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 169 op/s 2024-01-31T05:31:57.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:56 smithi186 ceph-mon[35659]: pgmap v290: 97 pgs: 97 active+clean; 543 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 169 op/s 2024-01-31T05:31:57.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 994784 11.09 MB/sec execute 357 sec latency 126.545 ms 2024-01-31T05:31:58.793 INFO:tasks.workunit.client.0.smithi045.stdout: 1 995812 11.07 MB/sec execute 358 sec latency 35.704 ms 2024-01-31T05:31:58.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:31:58 smithi045 ceph-mon[31585]: pgmap v291: 97 pgs: 97 active+clean; 554 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 253 op/s 2024-01-31T05:31:58.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:31:58 smithi186 ceph-mon[35659]: pgmap v291: 97 pgs: 97 active+clean; 554 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 253 op/s 2024-01-31T05:31:58.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:31:58 smithi156 ceph-mon[35720]: pgmap v291: 97 pgs: 97 active+clean; 554 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 253 op/s 2024-01-31T05:31:59.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 998316 11.08 MB/sec execute 359 sec latency 19.726 ms 2024-01-31T05:32:00.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 999002 11.05 MB/sec execute 360 sec latency 119.323 ms 2024-01-31T05:32:00.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:00 smithi045 ceph-mon[31585]: pgmap v292: 97 pgs: 97 active+clean; 554 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 163 op/s 2024-01-31T05:32:00.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:00 smithi156 ceph-mon[35720]: pgmap v292: 97 pgs: 97 active+clean; 554 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 163 op/s 2024-01-31T05:32:01.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:00 smithi186 ceph-mon[35659]: pgmap v292: 97 pgs: 97 active+clean; 554 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 163 op/s 2024-01-31T05:32:01.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1000465 11.04 MB/sec execute 361 sec latency 38.274 ms 2024-01-31T05:32:02.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1002006 11.04 MB/sec execute 362 sec latency 46.939 ms 2024-01-31T05:32:02.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:02 smithi045 ceph-mon[31585]: pgmap v293: 97 pgs: 97 active+clean; 554 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 163 op/s 2024-01-31T05:32:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:02 smithi156 ceph-mon[35720]: pgmap v293: 97 pgs: 97 active+clean; 554 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 163 op/s 2024-01-31T05:32:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:02 smithi186 ceph-mon[35659]: pgmap v293: 97 pgs: 97 active+clean; 554 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 163 op/s 2024-01-31T05:32:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.202+0000 7f300b789640 1 -- 172.21.15.45:0/3649545618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3004068d10 msgr2=0x7f300410db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.202+0000 7f300b789640 1 --2- 172.21.15.45:0/3649545618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3004068d10 0x7f300410db80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3000009ca0 tx=0x7f300002c180 comp rx=0 tx=0).stop 2024-01-31T05:32:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.202+0000 7f300b789640 1 -- 172.21.15.45:0/3649545618 shutdown_connections 2024-01-31T05:32:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.202+0000 7f300b789640 1 --2- 172.21.15.45:0/3649545618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3004068d10 0x7f300410db80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.202+0000 7f300b789640 1 --2- 172.21.15.45:0/3649545618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3004068370 0x7f30040687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.202+0000 7f300b789640 1 --2- 172.21.15.45:0/3649545618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3004106de0 0x7f30041071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.202+0000 7f300b789640 1 -- 172.21.15.45:0/3649545618 >> 172.21.15.45:0/3649545618 conn(0x7f3004076a40 msgr2=0x7f3004076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.203+0000 7f300b789640 1 -- 172.21.15.45:0/3649545618 shutdown_connections 2024-01-31T05:32:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.203+0000 7f300b789640 1 -- 172.21.15.45:0/3649545618 wait complete. 2024-01-31T05:32:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.203+0000 7f300b789640 1 Processor -- start 2024-01-31T05:32:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.203+0000 7f300b789640 1 -- start start 2024-01-31T05:32:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.203+0000 7f300b789640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3004068370 0x7f3004197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.203+0000 7f300b789640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3004068d10 0x7f3004197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.203+0000 7f300b789640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3004106de0 0x7f300419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.203+0000 7f300b789640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300419bf70 con 0x7f3004106de0 2024-01-31T05:32:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.203+0000 7f300b789640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300419c0e0 con 0x7f3004068d10 2024-01-31T05:32:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.203+0000 7f300b789640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300419c2a0 con 0x7f3004068370 2024-01-31T05:32:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3009d26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3004106de0 0x7f300419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3008d24640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3004068d10 0x7f3004197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3009d26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3004106de0 0x7f300419b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36570/0 (socket says 172.21.15.45:36570) 2024-01-31T05:32:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3009d26640 1 -- 172.21.15.45:0/521200969 learned_addr learned my addr 172.21.15.45:0/521200969 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3009525640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3004068370 0x7f3004197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3009d26640 1 -- 172.21.15.45:0/521200969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3004068370 msgr2=0x7f3004197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3009d26640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3004068370 0x7f3004197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3009d26640 1 -- 172.21.15.45:0/521200969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3004068d10 msgr2=0x7f3004197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3009d26640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3004068d10 0x7f3004197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3009d26640 1 -- 172.21.15.45:0/521200969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3000009900 con 0x7f3004106de0 2024-01-31T05:32:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3009525640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3004068370 0x7f3004197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3008d24640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3004068d10 0x7f3004197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f3009d26640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3004106de0 0x7f300419b970 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f300002c790 tx=0x7f3000034740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f2ffa7fc640 1 -- 172.21.15.45:0/521200969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000003d80 con 0x7f3004106de0 2024-01-31T05:32:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f300b789640 1 -- 172.21.15.45:0/521200969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300419c520 con 0x7f3004106de0 2024-01-31T05:32:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f2ffa7fc640 1 -- 172.21.15.45:0/521200969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3000032b70 con 0x7f3004106de0 2024-01-31T05:32:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f2ffa7fc640 1 -- 172.21.15.45:0/521200969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30000046a0 con 0x7f3004106de0 2024-01-31T05:32:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.204+0000 7f300b789640 1 -- 172.21.15.45:0/521200969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300419c7f0 con 0x7f3004106de0 2024-01-31T05:32:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.207+0000 7f2ffa7fc640 1 -- 172.21.15.45:0/521200969 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f300000a040 con 0x7f3004106de0 2024-01-31T05:32:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.207+0000 7f300e623740 1 Processor -- start 2024-01-31T05:32:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.207+0000 7f300e623740 1 -- start start 2024-01-31T05:32:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.207+0000 7f2ffa7fc640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2ff00a14f0 0x7f2ff00a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.207+0000 7f300e623740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff060 0x557b8b8ff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.207+0000 7f300e623740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff980 0x557b8b908210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f300e623740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 0x557b8b90ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3008d24640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2ff00a14f0 0x7f2ff00a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f300e623740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8b86e040 con 0x557b8b908750 2024-01-31T05:32:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f300e623740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8b86b730 con 0x557b8b8ff060 2024-01-31T05:32:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f2ffa7fc640 1 -- 172.21.15.45:0/521200969 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f30000ef8e0 con 0x7f3004106de0 2024-01-31T05:32:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f300e623740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8b867020 con 0x557b8b8ff980 2024-01-31T05:32:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3009d26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 0x557b8b90ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3009525640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff060 0x557b8b8ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3009d26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 0x557b8b90ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36578/0 (socket says 172.21.15.45:36578) 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3009d26640 1 -- 172.21.15.45:0/2646519361 learned_addr learned my addr 172.21.15.45:0/2646519361 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3008d24640 1 --2- 172.21.15.45:0/2646519361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff980 0x557b8b908210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3009525640 1 -- 172.21.15.45:0/2646519361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff980 msgr2=0x557b8b908210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3009525640 1 --2- 172.21.15.45:0/2646519361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff980 0x557b8b908210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3009525640 1 -- 172.21.15.45:0/2646519361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 msgr2=0x557b8b90ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3009525640 1 --2- 172.21.15.45:0/2646519361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 0x557b8b90ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3009525640 1 -- 172.21.15.45:0/2646519361 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b8b85ee40 con 0x557b8b8ff060 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3009d26640 1 --2- 172.21.15.45:0/2646519361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 0x557b8b90ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3008d24640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2ff00a14f0 0x7f2ff00a39b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400bfd0 tx=0x7f2ff4014cd0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f3009525640 1 --2- 172.21.15.45:0/2646519361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff060 0x557b8b8ff440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009fd0 tx=0x7f2ffc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba44a198f68d5e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f2fef7fe640 1 -- 172.21.15.45:0/2646519361 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc004310 con 0x557b8b8ff060 2024-01-31T05:32:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f2fef7fe640 1 -- 172.21.15.45:0/2646519361 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ffc004470 con 0x557b8b8ff060 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.208+0000 7f2fef7fe640 1 -- 172.21.15.45:0/2646519361 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc004e50 con 0x557b8b8ff060 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.209+0000 7f300e623740 1 -- 172.21.15.45:0/2646519361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff060 msgr2=0x557b8b8ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.209+0000 7f300e623740 1 --2- 172.21.15.45:0/2646519361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff060 0x557b8b8ff440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009fd0 tx=0x7f2ffc011810 comp rx=0 tx=0).stop 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.209+0000 7f300e623740 1 -- 172.21.15.45:0/2646519361 shutdown_connections 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.209+0000 7f300e623740 1 --2- 172.21.15.45:0/2646519361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 0x557b8b90ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.209+0000 7f300e623740 1 --2- 172.21.15.45:0/2646519361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff980 0x557b8b908210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.209+0000 7f300e623740 1 --2- 172.21.15.45:0/2646519361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff060 0x557b8b8ff440 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.209+0000 7f300e623740 1 -- 172.21.15.45:0/2646519361 >> 172.21.15.45:0/2646519361 conn(0x557b8b8f9980 msgr2=0x557b8b8f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.209+0000 7f300e623740 1 -- 172.21.15.45:0/2646519361 shutdown_connections 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.209+0000 7f300e623740 1 -- 172.21.15.45:0/2646519361 wait complete. 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f300e623740 1 -- start start 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f300e623740 1 Processor -- start 2024-01-31T05:32:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f300e623740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff060 0x557b8b905660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f300e623740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff980 0x557b8b905ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f300e623740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b908750 0x557b8b9060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f300e623740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8b8201b0 con 0x557b8b8ff060 2024-01-31T05:32:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f300e623740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8b81fe50 con 0x557b8b8ff980 2024-01-31T05:32:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f300e623740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8b7e5090 con 0x557b8b908750 2024-01-31T05:32:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f300e623740 10 client.0 _ll_register_callbacks cb 0x557b8b80aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:32:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f3009525640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff060 0x557b8b905660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f3009525640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff060 0x557b8b905660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36584/0 (socket says 172.21.15.45:36584) 2024-01-31T05:32:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f3009525640 1 -- 172.21.15.45:0/2844519337 learned_addr learned my addr 172.21.15.45:0/2844519337 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f3008d24640 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff980 0x557b8b905ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f3009d26640 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b908750 0x557b8b9060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f3009525640 1 -- 172.21.15.45:0/2844519337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b908750 msgr2=0x557b8b9060e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f3009525640 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b908750 0x557b8b9060e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f3009525640 1 -- 172.21.15.45:0/2844519337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff980 msgr2=0x557b8b905ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f3009525640 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff980 0x557b8b905ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f3009525640 1 -- 172.21.15.45:0/2844519337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc009c00 con 0x557b8b8ff060 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.210+0000 7f3008d24640 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff980 0x557b8b905ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.211+0000 7f3009525640 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff060 0x557b8b905660 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc005890 tx=0x7f2ffc0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1d73731e99e384 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.211+0000 7f2fecff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.211+0000 7f2fecff9640 1 -- 172.21.15.45:0/2844519337 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc003bf0 con 0x557b8b8ff060 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.211+0000 7f300e623740 1 -- 172.21.15.45:0/2844519337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b8b860010 con 0x557b8b8ff060 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.211+0000 7f2fecff9640 1 -- 172.21.15.45:0/2844519337 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2ffc003d50 con 0x557b8b8ff060 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.211+0000 7f2fecff9640 1 -- 172.21.15.45:0/2844519337 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ffc023430 con 0x557b8b8ff060 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.211+0000 7f300e623740 10 client.16149 fetch_fsmap learned FSMap version 28 2024-01-31T05:32:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.211+0000 7f300e623740 1 -- 172.21.15.45:0/2844519337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557b8b8656a0 con 0x557b8b8ff060 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.211+0000 7f2fecff9640 1 -- 172.21.15.45:0/2844519337 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b8b860010 con 0x557b8b8ff060 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.211+0000 7f2fecff9640 1 -- 172.21.15.45:0/2844519337 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2ffc021040 con 0x557b8b8ff060 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.211+0000 7f300e623740 10 client.16149 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.211+0000 7f300e623740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.212+0000 7f300e623740 20 client.16149 populate_metadata read hostname 'smithi045' 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.212+0000 7f300e623740 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557b8b91c880 0x557b8b91ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.212+0000 7f300e623740 4 client.16149 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.212+0000 7f300e623740 1 -- 172.21.15.45:0/2844519337 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b8b85fd30 con 0x557b8b91c880 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.212+0000 7f3008d24640 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557b8b91c880 0x557b8b91ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.212+0000 7f2fecff9640 10 client.16149 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.212+0000 7f3008d24640 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557b8b91c880 0x557b8b91ecc0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b2f5ff019f01e9 server_cookie=75de127a35a7c80f in_seq=0 out_seq=0 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f2fecff9640 1 -- 172.21.15.45:0/2844519337 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b8b85fd30 con 0x557b8b91c880 2024-01-31T05:32:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f2fecff9640 10 client.16149 handle_command_reply: tid=0 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f300e623740 1 client.16149 shutdown 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f300e623740 1 -- 172.21.15.45:0/2844519337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff060 msgr2=0x557b8b905660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f300e623740 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff060 0x557b8b905660 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc005890 tx=0x7f2ffc0035f0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f300e623740 1 -- 172.21.15.45:0/2844519337 shutdown_connections 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f300e623740 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557b8b91c880 0x557b8b91ecc0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f2fecff9640 0 client.16149 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f300e623740 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b908750 0x557b8b9060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f300e623740 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff980 0x557b8b905ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f300e623740 1 --2- 172.21.15.45:0/2844519337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff060 0x557b8b905660 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f300e623740 1 -- 172.21.15.45:0/2844519337 >> 172.21.15.45:0/2844519337 conn(0x557b8b8f9980 msgr2=0x557b8b8fb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f300e623740 1 -- 172.21.15.45:0/2844519337 shutdown_connections 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f300e623740 1 -- 172.21.15.45:0/2844519337 wait complete. 2024-01-31T05:32:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.213+0000 7f300e623740 20 client.16149 trim_cache size 0 max 16384 2024-01-31T05:32:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.228+0000 7f300e623740 1 Processor -- start 2024-01-31T05:32:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.228+0000 7f300e623740 1 -- start start 2024-01-31T05:32:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f300e623740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff060 0x557b8b907d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f300e623740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff980 0x557b8b905f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f300e623740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 0x557b8b9028c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f300e623740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8b8f52b0 con 0x557b8b908750 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f300e623740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8b8fa980 con 0x557b8b8ff980 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f300e623740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8b923ed0 con 0x557b8b8ff060 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f3009d26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 0x557b8b9028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f3009525640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff060 0x557b8b907d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f3009d26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 0x557b8b9028c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36588/0 (socket says 172.21.15.45:36588) 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f3009d26640 1 -- 172.21.15.45:0/1171265053 learned_addr learned my addr 172.21.15.45:0/1171265053 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f3008d24640 1 --2- 172.21.15.45:0/1171265053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff980 0x557b8b905f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f3009d26640 1 -- 172.21.15.45:0/1171265053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff060 msgr2=0x557b8b907d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f3009d26640 1 --2- 172.21.15.45:0/1171265053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff060 0x557b8b907d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f3009d26640 1 -- 172.21.15.45:0/1171265053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff980 msgr2=0x557b8b905f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f3009d26640 1 --2- 172.21.15.45:0/1171265053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff980 0x557b8b905f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f3009d26640 1 -- 172.21.15.45:0/1171265053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b8b802030 con 0x557b8b908750 2024-01-31T05:32:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f3009d26640 1 --2- 172.21.15.45:0/1171265053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 0x557b8b9028c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3000004af0 tx=0x7f3000004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db95b4f414e2dde8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f2feeffd640 1 -- 172.21.15.45:0/1171265053 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30000bc150 con 0x557b8b908750 2024-01-31T05:32:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f2feeffd640 1 -- 172.21.15.45:0/1171265053 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30000bcef0 con 0x557b8b908750 2024-01-31T05:32:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.229+0000 7f2feeffd640 1 -- 172.21.15.45:0/1171265053 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30000ed2b0 con 0x557b8b908750 2024-01-31T05:32:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.230+0000 7f300e623740 1 -- 172.21.15.45:0/1171265053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 msgr2=0x557b8b9028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.230+0000 7f300e623740 1 --2- 172.21.15.45:0/1171265053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 0x557b8b9028c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3000004af0 tx=0x7f3000004140 comp rx=0 tx=0).stop 2024-01-31T05:32:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.230+0000 7f300e623740 1 -- 172.21.15.45:0/1171265053 shutdown_connections 2024-01-31T05:32:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.230+0000 7f300e623740 1 --2- 172.21.15.45:0/1171265053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b908750 0x557b8b9028c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.230+0000 7f300e623740 1 --2- 172.21.15.45:0/1171265053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b8ff980 0x557b8b905f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.230+0000 7f300e623740 1 --2- 172.21.15.45:0/1171265053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff060 0x557b8b907d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.230+0000 7f300e623740 1 -- 172.21.15.45:0/1171265053 >> 172.21.15.45:0/1171265053 conn(0x557b8b8f9980 msgr2=0x557b8b8f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.230+0000 7f300e623740 1 -- 172.21.15.45:0/1171265053 shutdown_connections 2024-01-31T05:32:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.230+0000 7f300e623740 1 -- 172.21.15.45:0/1171265053 wait complete. 2024-01-31T05:32:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.230+0000 7f300e623740 1 -- start start 2024-01-31T05:32:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.230+0000 7f300e623740 1 Processor -- start 2024-01-31T05:32:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f300e623740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff060 0x557b8b911770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f300e623740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff980 0x557b8b911cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f300e623740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b908750 0x557b8b971da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f300e623740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8b912560 con 0x557b8b8ff980 2024-01-31T05:32:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f300e623740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8b9722e0 con 0x557b8b908750 2024-01-31T05:32:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f300e623740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8b972420 con 0x557b8b8ff060 2024-01-31T05:32:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f300e623740 10 client.0 _ll_register_callbacks cb 0x7f2ffc009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:32:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f3008d24640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff980 0x557b8b911cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f3009525640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff060 0x557b8b911770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f3008d24640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff980 0x557b8b911cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36596/0 (socket says 172.21.15.45:36596) 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f3008d24640 1 -- 172.21.15.45:0/3132342017 learned_addr learned my addr 172.21.15.45:0/3132342017 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f3009525640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff060 0x557b8b911770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:49490/0 (socket says 172.21.15.45:49490) 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f3009d26640 1 --2- 172.21.15.45:0/3132342017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b908750 0x557b8b971da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f3008d24640 1 -- 172.21.15.45:0/3132342017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff060 msgr2=0x557b8b911770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f3008d24640 1 --2- 172.21.15.45:0/3132342017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff060 0x557b8b911770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f3008d24640 1 -- 172.21.15.45:0/3132342017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b908750 msgr2=0x557b8b971da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f3008d24640 1 --2- 172.21.15.45:0/3132342017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b908750 0x557b8b971da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.231+0000 7f3008d24640 1 -- 172.21.15.45:0/3132342017 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300419c520 con 0x557b8b8ff980 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f3008d24640 1 --2- 172.21.15.45:0/3132342017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff980 0x557b8b911cb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4014320 tx=0x7f2ff4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c897ec4fbc5820 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f2fecff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f2fecff9640 1 -- 172.21.15.45:0/3132342017 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4027070 con 0x557b8b8ff980 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f2fecff9640 1 -- 172.21.15.45:0/3132342017 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2ff4028d10 con 0x557b8b8ff980 2024-01-31T05:32:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f300e623740 1 -- 172.21.15.45:0/3132342017 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ffc004470 con 0x557b8b8ff980 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f2fecff9640 1 -- 172.21.15.45:0/3132342017 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ff4004020 con 0x557b8b8ff980 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f300e623740 10 client.16155 fetch_fsmap learned FSMap version 28 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f300e623740 1 -- 172.21.15.45:0/3132342017 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557b8b827a50 con 0x557b8b8ff980 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f2fecff9640 1 -- 172.21.15.45:0/3132342017 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ffc004470 con 0x557b8b8ff980 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f2fecff9640 1 -- 172.21.15.45:0/3132342017 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2ff4028250 con 0x557b8b8ff980 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f300e623740 10 client.16155 fetch_fsmap finished waiting for FSMap version 28 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f300e623740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f300e623740 20 client.16155 populate_metadata read hostname 'smithi045' 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f300e623740 1 --2- 172.21.15.45:0/3132342017 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557b8b91c880 0x557b8b975cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f300e623740 4 client.16155 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.232+0000 7f300e623740 1 -- 172.21.15.45:0/3132342017 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557b8b566600 con 0x557b8b91c880 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.233+0000 7f3009525640 1 --2- 172.21.15.45:0/3132342017 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557b8b91c880 0x557b8b975cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.233+0000 7f3009525640 1 --2- 172.21.15.45:0/3132342017 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557b8b91c880 0x557b8b975cb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba8da784e9ae47fe server_cookie=8e1458777b291653 in_seq=0 out_seq=0 2024-01-31T05:32:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:03.233+0000 7f2fecff9640 10 client.16155 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:03.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1003473 11.03 MB/sec execute 363 sec latency 20.118 ms 2024-01-31T05:32:04.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1005543 11.03 MB/sec execute 364 sec latency 18.981 ms 2024-01-31T05:32:04.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:04 smithi045 ceph-mon[31585]: pgmap v294: 97 pgs: 97 active+clean; 562 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.4 MiB/s wr, 281 op/s 2024-01-31T05:32:04.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:04 smithi156 ceph-mon[35720]: pgmap v294: 97 pgs: 97 active+clean; 562 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.4 MiB/s wr, 281 op/s 2024-01-31T05:32:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f2fecff9640 1 -- 172.21.15.45:0/3132342017 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557b8b566600 con 0x557b8b91c880 2024-01-31T05:32:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f2fecff9640 10 client.16155 handle_command_reply: tid=0 2024-01-31T05:32:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f300e623740 1 client.16155 shutdown 2024-01-31T05:32:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f300e623740 1 -- 172.21.15.45:0/3132342017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff980 msgr2=0x557b8b911cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f300e623740 1 --2- 172.21.15.45:0/3132342017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff980 0x557b8b911cb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4014320 tx=0x7f2ff4004730 comp rx=0 tx=0).stop 2024-01-31T05:32:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f300e623740 1 -- 172.21.15.45:0/3132342017 shutdown_connections 2024-01-31T05:32:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f300e623740 1 --2- 172.21.15.45:0/3132342017 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557b8b91c880 0x557b8b975cb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f300e623740 1 --2- 172.21.15.45:0/3132342017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b8b8ff980 0x557b8b911cb0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f300e623740 1 --2- 172.21.15.45:0/3132342017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b8b908750 0x557b8b971da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f300e623740 1 --2- 172.21.15.45:0/3132342017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b8b8ff060 0x557b8b911770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f2fecff9640 0 client.16155 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f300e623740 1 -- 172.21.15.45:0/3132342017 >> 172.21.15.45:0/3132342017 conn(0x557b8b8f9980 msgr2=0x557b8b903f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f300e623740 1 -- 172.21.15.45:0/3132342017 shutdown_connections 2024-01-31T05:32:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f300e623740 1 -- 172.21.15.45:0/3132342017 wait complete. 2024-01-31T05:32:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.021+0000 7f300e623740 20 client.16155 trim_cache size 0 max 16384 2024-01-31T05:32:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:32:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:32:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:32:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:32:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.022+0000 7f300b789640 1 -- 172.21.15.45:0/521200969 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2ff00a14f0 msgr2=0x7f2ff00a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.022+0000 7f300b789640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2ff00a14f0 0x7f2ff00a39b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400bfd0 tx=0x7f2ff4014cd0 comp rx=0 tx=0).stop 2024-01-31T05:32:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.022+0000 7f300b789640 1 -- 172.21.15.45:0/521200969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3004106de0 msgr2=0x7f300419b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.022+0000 7f300b789640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3004106de0 0x7f300419b970 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f300002c790 tx=0x7f3000034740 comp rx=0 tx=0).stop 2024-01-31T05:32:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.022+0000 7f300b789640 1 -- 172.21.15.45:0/521200969 shutdown_connections 2024-01-31T05:32:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.022+0000 7f300b789640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2ff00a14f0 0x7f2ff00a39b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.022+0000 7f300b789640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3004106de0 0x7f300419b970 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.022+0000 7f300b789640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3004068d10 0x7f3004197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.022+0000 7f300b789640 1 --2- 172.21.15.45:0/521200969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3004068370 0x7f3004197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.022+0000 7f300b789640 1 -- 172.21.15.45:0/521200969 >> 172.21.15.45:0/521200969 conn(0x7f3004076a40 msgr2=0x7f30040feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.022+0000 7f300b789640 1 -- 172.21.15.45:0/521200969 shutdown_connections 2024-01-31T05:32:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:05.022+0000 7f300b789640 1 -- 172.21.15.45:0/521200969 wait complete. 2024-01-31T05:32:05.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:04 smithi186 ceph-mon[35659]: pgmap v294: 97 pgs: 97 active+clean; 562 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.4 MiB/s wr, 281 op/s 2024-01-31T05:32:05.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1006849 11.02 MB/sec execute 365 sec latency 67.479 ms 2024-01-31T05:32:06.044 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: *** Caught signal (Segmentation fault) ** 2024-01-31T05:32:06.044 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: in thread 7fdcd54e2700 thread_name:md_log_replay 2024-01-31T05:32:06.044 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:32:06.044 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdce60a2d20] 2024-01-31T05:32:06.044 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 2: /usr/bin/ceph-mds(+0x5040bf) [0x5597420210bf] 2024-01-31T05:32:06.044 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5597420232f3] 2024-01-31T05:32:06.044 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 4: (MDLog::_replay_thread()+0x647) [0x55974207f697] 2024-01-31T05:32:06.044 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559742088b31] 2024-01-31T05:32:06.044 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fdce60981ca] 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 7: clone() 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 2024-01-31T05:32:05.735+0000 7fdcd54e2700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: in thread 7fdcd54e2700 thread_name:md_log_replay 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdce60a2d20] 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 2: /usr/bin/ceph-mds(+0x5040bf) [0x5597420210bf] 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5597420232f3] 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 4: (MDLog::_replay_thread()+0x647) [0x55974207f697] 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559742088b31] 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fdce60981ca] 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 7: clone() 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:32:06.045 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 0> 2024-01-31T05:32:05.735+0000 7fdcd54e2700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: in thread 7fdcd54e2700 thread_name:md_log_replay 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdce60a2d20] 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 2: /usr/bin/ceph-mds(+0x5040bf) [0x5597420210bf] 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5597420232f3] 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 4: (MDLog::_replay_thread()+0x647) [0x55974207f697] 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559742088b31] 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fdce60981ca] 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 7: clone() 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 0> 2024-01-31T05:32:05.735+0000 7fdcd54e2700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:32:06.046 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: in thread 7fdcd54e2700 thread_name:md_log_replay 2024-01-31T05:32:06.047 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 2024-01-31T05:32:06.047 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:32:06.047 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdce60a2d20] 2024-01-31T05:32:06.047 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 2: /usr/bin/ceph-mds(+0x5040bf) [0x5597420210bf] 2024-01-31T05:32:06.047 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5597420232f3] 2024-01-31T05:32:06.047 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 4: (MDLog::_replay_thread()+0x647) [0x55974207f697] 2024-01-31T05:32:06.047 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559742088b31] 2024-01-31T05:32:06.047 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fdce60981ca] 2024-01-31T05:32:06.047 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 7: clone() 2024-01-31T05:32:06.047 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:32:06.047 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71322]: 2024-01-31T05:32:06.047 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:05 smithi156 podman[71755]: 2024-01-31 05:32:05.78638506 +0000 UTC m=+0.020153436 container died 04889acaff5f200245fb35f14d8e8372298f6a4f1befbc46c562355851f24a5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-01-31T05:32:06.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:06 smithi156 ceph-mon[35720]: pgmap v295: 97 pgs: 97 active+clean; 562 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 202 op/s 2024-01-31T05:32:06.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:06 smithi156 ceph-mon[35720]: osdmap e100: 12 total, 12 up, 12 in 2024-01-31T05:32:06.727 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:06 smithi156 podman[71755]: 2024-01-31 05:32:06.453912238 +0000 UTC m=+0.687680591 container remove 04889acaff5f200245fb35f14d8e8372298f6a4f1befbc46c562355851f24a5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD) 2024-01-31T05:32:06.727 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:06 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-01-31T05:32:06.727 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:06 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-01-31T05:32:06.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1009030 11.03 MB/sec execute 366 sec latency 22.696 ms 2024-01-31T05:32:06.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:06 smithi045 ceph-mon[31585]: pgmap v295: 97 pgs: 97 active+clean; 562 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 202 op/s 2024-01-31T05:32:06.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:06 smithi045 ceph-mon[31585]: osdmap e100: 12 total, 12 up, 12 in 2024-01-31T05:32:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:06 smithi186 ceph-mon[35659]: pgmap v295: 97 pgs: 97 active+clean; 562 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 202 op/s 2024-01-31T05:32:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:06 smithi186 ceph-mon[35659]: osdmap e100: 12 total, 12 up, 12 in 2024-01-31T05:32:07.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1010178 11.01 MB/sec execute 367 sec latency 69.347 ms 2024-01-31T05:32:08.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1012510 11.02 MB/sec execute 368 sec latency 34.158 ms 2024-01-31T05:32:08.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:08 smithi045 ceph-mon[31585]: pgmap v297: 97 pgs: 97 active+clean; 567 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-01-31T05:32:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:08 smithi156 ceph-mon[35720]: pgmap v297: 97 pgs: 97 active+clean; 567 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-01-31T05:32:09.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:08 smithi186 ceph-mon[35659]: pgmap v297: 97 pgs: 97 active+clean; 567 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-01-31T05:32:09.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1013725 11.00 MB/sec execute 369 sec latency 26.580 ms 2024-01-31T05:32:10.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1016060 11.01 MB/sec execute 370 sec latency 21.971 ms 2024-01-31T05:32:10.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:10 smithi045 ceph-mon[31585]: pgmap v298: 97 pgs: 97 active+clean; 567 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-01-31T05:32:10.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:10 smithi156 ceph-mon[35720]: pgmap v298: 97 pgs: 97 active+clean; 567 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-01-31T05:32:11.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:10 smithi186 ceph-mon[35659]: pgmap v298: 97 pgs: 97 active+clean; 567 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-01-31T05:32:11.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1017122 10.99 MB/sec execute 371 sec latency 85.785 ms 2024-01-31T05:32:12.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1019386 10.99 MB/sec execute 372 sec latency 87.503 ms 2024-01-31T05:32:12.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:12 smithi045 ceph-mon[31585]: pgmap v299: 97 pgs: 97 active+clean; 567 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-01-31T05:32:12.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:12 smithi045 ceph-mon[31585]: osdmap e101: 12 total, 12 up, 12 in 2024-01-31T05:32:12.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:12 smithi156 ceph-mon[35720]: pgmap v299: 97 pgs: 97 active+clean; 567 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-01-31T05:32:12.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:12 smithi156 ceph-mon[35720]: osdmap e101: 12 total, 12 up, 12 in 2024-01-31T05:32:13.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:12 smithi186 ceph-mon[35659]: pgmap v299: 97 pgs: 97 active+clean; 567 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-01-31T05:32:13.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:12 smithi186 ceph-mon[35659]: osdmap e101: 12 total, 12 up, 12 in 2024-01-31T05:32:13.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1020407 10.98 MB/sec execute 373 sec latency 40.778 ms 2024-01-31T05:32:14.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1022972 10.99 MB/sec execute 374 sec latency 20.644 ms 2024-01-31T05:32:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:14 smithi045 ceph-mon[31585]: pgmap v301: 97 pgs: 97 active+clean; 577 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 255 op/s 2024-01-31T05:32:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:14 smithi156 ceph-mon[35720]: pgmap v301: 97 pgs: 97 active+clean; 577 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 255 op/s 2024-01-31T05:32:15.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:14 smithi186 ceph-mon[35659]: pgmap v301: 97 pgs: 97 active+clean; 577 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 255 op/s 2024-01-31T05:32:15.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1024053 10.97 MB/sec execute 375 sec latency 17.024 ms 2024-01-31T05:32:16.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1026510 10.98 MB/sec execute 376 sec latency 19.667 ms 2024-01-31T05:32:16.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:16 smithi045 ceph-mon[31585]: pgmap v302: 97 pgs: 97 active+clean; 577 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 205 op/s 2024-01-31T05:32:16.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:16 smithi156 ceph-mon[35720]: pgmap v302: 97 pgs: 97 active+clean; 577 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 205 op/s 2024-01-31T05:32:16.952 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:16 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 4. 2024-01-31T05:32:16.952 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:16 smithi156 systemd[1]: Stopped Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:32:16.952 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:16 smithi156 systemd[1]: Starting Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:32:17.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:16 smithi186 ceph-mon[35659]: pgmap v302: 97 pgs: 97 active+clean; 577 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 205 op/s 2024-01-31T05:32:17.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:16 smithi156 podman[71860]: 2024-01-31 05:32:16.855276726 +0000 UTC m=+0.018570918 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:32:17.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:16 smithi156 podman[71860]: 2024-01-31 05:32:16.979983508 +0000 UTC m=+0.143281958 container create fa207d71408a5ac1576c418179239a826ffc4c7b547db49447453d8472200699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:32:17.520 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:17 smithi156 podman[71860]: 2024-01-31 05:32:17.232921717 +0000 UTC m=+0.396215897 container init fa207d71408a5ac1576c418179239a826ffc4c7b547db49447453d8472200699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-01-31T05:32:17.520 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:17 smithi156 podman[71860]: 2024-01-31 05:32:17.236416769 +0000 UTC m=+0.399710956 container start fa207d71408a5ac1576c418179239a826ffc4c7b547db49447453d8472200699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=) 2024-01-31T05:32:17.520 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:17 smithi156 ceph-mds[71879]: -- 172.21.15.156:0/3238902855 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562ca4b33520 con 0x562ca4b09000 2024-01-31T05:32:17.520 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:17 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: starting mds.h at 2024-01-31T05:32:17.520 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:17 smithi156 bash[71860]: fa207d71408a5ac1576c418179239a826ffc4c7b547db49447453d8472200699 2024-01-31T05:32:17.521 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:32:17 smithi156 systemd[1]: Started Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:32:17.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1027246 10.96 MB/sec execute 377 sec latency 98.675 ms 2024-01-31T05:32:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:17 smithi045 ceph-mon[31585]: daemon mds.h restarted 2024-01-31T05:32:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:17 smithi045 ceph-mon[31585]: pgmap v303: 97 pgs: 97 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 202 op/s 2024-01-31T05:32:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:17 smithi045 ceph-mon[31585]: osdmap e102: 12 total, 12 up, 12 in 2024-01-31T05:32:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:17 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.156:6836/2718382318,v1:172.21.15.156:6837/2718382318] up:boot 2024-01-31T05:32:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:17 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:32:17.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:17 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:32:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:17 smithi156 ceph-mon[35720]: daemon mds.h restarted 2024-01-31T05:32:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:17 smithi156 ceph-mon[35720]: pgmap v303: 97 pgs: 97 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 202 op/s 2024-01-31T05:32:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:17 smithi156 ceph-mon[35720]: osdmap e102: 12 total, 12 up, 12 in 2024-01-31T05:32:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:17 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.156:6836/2718382318,v1:172.21.15.156:6837/2718382318] up:boot 2024-01-31T05:32:17.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:17 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:32:17.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:17 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:32:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:17 smithi186 ceph-mon[35659]: daemon mds.h restarted 2024-01-31T05:32:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:17 smithi186 ceph-mon[35659]: pgmap v303: 97 pgs: 97 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 202 op/s 2024-01-31T05:32:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:17 smithi186 ceph-mon[35659]: osdmap e102: 12 total, 12 up, 12 in 2024-01-31T05:32:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:17 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.156:6836/2718382318,v1:172.21.15.156:6837/2718382318] up:boot 2024-01-31T05:32:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:17 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:32:18.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:17 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:32:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.129+0000 7f5b5c52f640 1 -- 172.21.15.45:0/102750522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b540ffdb0 msgr2=0x7f5b54105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.129+0000 7f5b5c52f640 1 --2- 172.21.15.45:0/102750522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b540ffdb0 0x7f5b54105940 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009930 tx=0x7f5b4402bc30 comp rx=0 tx=0).stop 2024-01-31T05:32:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.129+0000 7f5b5c52f640 1 -- 172.21.15.45:0/102750522 shutdown_connections 2024-01-31T05:32:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.129+0000 7f5b5c52f640 1 --2- 172.21.15.45:0/102750522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b540ffdb0 0x7f5b54105940 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.129+0000 7f5b5c52f640 1 --2- 172.21.15.45:0/102750522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b54106070 0x7f5b5410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.129+0000 7f5b5c52f640 1 --2- 172.21.15.45:0/102750522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b540ff400 0x7f5b540ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.129+0000 7f5b5c52f640 1 -- 172.21.15.45:0/102750522 >> 172.21.15.45:0/102750522 conn(0x7f5b54076110 msgr2=0x7f5b540fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.129+0000 7f5b5c52f640 1 -- 172.21.15.45:0/102750522 shutdown_connections 2024-01-31T05:32:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.129+0000 7f5b5c52f640 1 -- 172.21.15.45:0/102750522 wait complete. 2024-01-31T05:32:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b5c52f640 1 Processor -- start 2024-01-31T05:32:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b5c52f640 1 -- start start 2024-01-31T05:32:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b5c52f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b540ff400 0x7f5b541972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b5c52f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b54106070 0x7f5b541977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b5c52f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5419bb60 0x7f5b5419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b5c52f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419c550 con 0x7f5b5419bb60 2024-01-31T05:32:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b5c52f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419c6c0 con 0x7f5b540ff400 2024-01-31T05:32:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b5c52f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419c830 con 0x7f5b54106070 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b5aacc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5419bb60 0x7f5b5419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b5aacc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5419bb60 0x7f5b5419c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48480/0 (socket says 172.21.15.45:48480) 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b5aacc640 1 -- 172.21.15.45:0/463493719 learned_addr learned my addr 172.21.15.45:0/463493719 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b59aca640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b54106070 0x7f5b541977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.130+0000 7f5b5a2cb640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b540ff400 0x7f5b541972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.131+0000 7f5b5aacc640 1 -- 172.21.15.45:0/463493719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b54106070 msgr2=0x7f5b541977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.131+0000 7f5b5aacc640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b54106070 0x7f5b541977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.131+0000 7f5b5aacc640 1 -- 172.21.15.45:0/463493719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b540ff400 msgr2=0x7f5b541972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.131+0000 7f5b5aacc640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b540ff400 0x7f5b541972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.131+0000 7f5b5aacc640 1 -- 172.21.15.45:0/463493719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b44009560 con 0x7f5b5419bb60 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.131+0000 7f5b5aacc640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5419bb60 0x7f5b5419c010 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00eb10 tx=0x7f5b4c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.131+0000 7f5b4b7fe640 1 -- 172.21.15.45:0/463493719 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c004b20 con 0x7f5b5419bb60 2024-01-31T05:32:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.131+0000 7f5b5c52f640 1 -- 172.21.15.45:0/463493719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b541a3e10 con 0x7f5b5419bb60 2024-01-31T05:32:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.131+0000 7f5b4b7fe640 1 -- 172.21.15.45:0/463493719 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b4c004c80 con 0x7f5b5419bb60 2024-01-31T05:32:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.131+0000 7f5b4b7fe640 1 -- 172.21.15.45:0/463493719 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c010690 con 0x7f5b5419bb60 2024-01-31T05:32:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.131+0000 7f5b5c52f640 1 -- 172.21.15.45:0/463493719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b541a4240 con 0x7f5b5419bb60 2024-01-31T05:32:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.133+0000 7f5b4b7fe640 1 -- 172.21.15.45:0/463493719 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f5b4c005020 con 0x7f5b5419bb60 2024-01-31T05:32:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5f3cd740 1 Processor -- start 2024-01-31T05:32:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5f3cd740 1 -- start start 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b4b7fe640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5b400a14f0 0x7f5b400a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5f3cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f335925440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5f3cd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925980 0x55f33592e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5f3cd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 0x55f335930b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5f3cd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f335894040 con 0x55f33592e750 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5f3cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f335891730 con 0x55f335925060 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5f3cd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33588d020 con 0x55f335925980 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b59aca640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5b400a14f0 0x7f5b400a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5aacc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 0x55f335930b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5a2cb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f335925440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5aacc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 0x55f335930b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48492/0 (socket says 172.21.15.45:48492) 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5aacc640 1 -- 172.21.15.45:0/2653492968 learned_addr learned my addr 172.21.15.45:0/2653492968 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b4b7fe640 1 -- 172.21.15.45:0/463493719 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5b4c014070 con 0x7f5b5419bb60 2024-01-31T05:32:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b59aca640 1 --2- 172.21.15.45:0/2653492968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925980 0x55f33592e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5a2cb640 1 -- 172.21.15.45:0/2653492968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925980 msgr2=0x55f33592e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5a2cb640 1 --2- 172.21.15.45:0/2653492968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925980 0x55f33592e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5a2cb640 1 -- 172.21.15.45:0/2653492968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 msgr2=0x55f335930b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5a2cb640 1 --2- 172.21.15.45:0/2653492968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 0x55f335930b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b5a2cb640 1 -- 172.21.15.45:0/2653492968 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f335884e40 con 0x55f335925060 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.134+0000 7f5b59aca640 1 --2- 172.21.15.45:0/2653492968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925980 0x55f33592e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.135+0000 7f5b5a2cb640 1 --2- 172.21.15.45:0/2653492968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f335925440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009c00 tx=0x7f5b50011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=467be83247a40359 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.135+0000 7f5b59aca640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5b400a14f0 0x7f5b400a39b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009230 tx=0x7f5b44005fd0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.135+0000 7f5b48bfe640 1 -- 172.21.15.45:0/2653492968 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50004230 con 0x55f335925060 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.135+0000 7f5b48bfe640 1 -- 172.21.15.45:0/2653492968 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b50009e70 con 0x55f335925060 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.135+0000 7f5b48bfe640 1 -- 172.21.15.45:0/2653492968 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50004c60 con 0x55f335925060 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/2653492968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 msgr2=0x55f335925440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/2653492968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f335925440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009c00 tx=0x7f5b50011810 comp rx=0 tx=0).stop 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/2653492968 shutdown_connections 2024-01-31T05:32:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/2653492968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 0x55f335930b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/2653492968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925980 0x55f33592e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/2653492968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f335925440 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/2653492968 >> 172.21.15.45:0/2653492968 conn(0x55f33591f980 msgr2=0x55f33591fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/2653492968 shutdown_connections 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/2653492968 wait complete. 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 -- start start 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 Processor -- start 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925060 0x55f33592b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.136+0000 7f5b5f3cd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 0x55f33592bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5f3cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f33592e750 0x55f33592c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5f3cd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3358461b0 con 0x55f335925980 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5f3cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f335845e50 con 0x55f33592e750 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5f3cd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33580b090 con 0x55f335925060 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b59aca640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 0x55f33592bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5f3cd740 10 client.0 _ll_register_callbacks cb 0x55f335830a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:32:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5a2cb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925060 0x55f33592b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b59aca640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 0x55f33592bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48498/0 (socket says 172.21.15.45:48498) 2024-01-31T05:32:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b59aca640 1 -- 172.21.15.45:0/3637440371 learned_addr learned my addr 172.21.15.45:0/3637440371 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5aacc640 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f33592e750 0x55f33592c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5a2cb640 1 -- 172.21.15.45:0/3637440371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f33592e750 msgr2=0x55f33592c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5a2cb640 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f33592e750 0x55f33592c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5a2cb640 1 -- 172.21.15.45:0/3637440371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 msgr2=0x55f33592bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5a2cb640 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 0x55f33592bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5a2cb640 1 -- 172.21.15.45:0/3637440371 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b50009830 con 0x55f335925060 2024-01-31T05:32:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b59aca640 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 0x55f33592bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5a2cb640 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925060 0x55f33592b670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009bd0 tx=0x7f5b50005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53091b35a98fdb7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b3dffb640 1 -- 172.21.15.45:0/3637440371 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b500038f0 con 0x55f335925060 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/3637440371 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f335886010 con 0x55f335925060 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.137+0000 7f5b3dffb640 1 -- 172.21.15.45:0/3637440371 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5b50003a50 con 0x55f335925060 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.138+0000 7f5b3dffb640 1 -- 172.21.15.45:0/3637440371 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b50003c10 con 0x55f335925060 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.138+0000 7f5b5f3cd740 10 client.25732 fetch_fsmap learned FSMap version 29 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.138+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/3637440371 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f33588b6a0 con 0x55f335925060 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.138+0000 7f5b3dffb640 1 -- 172.21.15.45:0/3637440371 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f335886010 con 0x55f335925060 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.138+0000 7f5b3dffb640 1 -- 172.21.15.45:0/3637440371 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5b50023410 con 0x55f335925060 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.138+0000 7f5b5f3cd740 10 client.25732 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.138+0000 7f5b5f3cd740 10 client.25732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.138+0000 7f5b5f3cd740 20 client.25732 populate_metadata read hostname 'smithi045' 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.138+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f33593e430 0x55f335940840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.138+0000 7f5b5f3cd740 4 client.25732 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:32:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.138+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/3637440371 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f335885d30 con 0x55f33593e430 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.139+0000 7f5b59aca640 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f33593e430 0x55f335940840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.139+0000 7f5b59aca640 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f33593e430 0x55f335940840 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=393fe27abd5d2694 server_cookie=65f9c78fcb53ced3 in_seq=0 out_seq=0 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.139+0000 7f5b3dffb640 10 client.25732 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.139+0000 7f5b3dffb640 1 -- 172.21.15.45:0/3637440371 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f335885d30 con 0x55f33593e430 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.139+0000 7f5b3dffb640 10 client.25732 handle_command_reply: tid=0 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b5f3cd740 1 client.25732 shutdown 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/3637440371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925060 msgr2=0x55f33592b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925060 0x55f33592b670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009bd0 tx=0x7f5b50005cf0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/3637440371 shutdown_connections 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f33593e430 0x55f335940840 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 0x55f33592bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b3dffb640 0 client.25732 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f33592e750 0x55f33592c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/3637440371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925060 0x55f33592b670 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/3637440371 >> 172.21.15.45:0/3637440371 conn(0x55f33591f980 msgr2=0x55f3359216d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/3637440371 shutdown_connections 2024-01-31T05:32:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/3637440371 wait complete. 2024-01-31T05:32:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.140+0000 7f5b5f3cd740 20 client.25732 trim_cache size 0 max 16384 2024-01-31T05:32:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.153+0000 7f5b5f3cd740 1 Processor -- start 2024-01-31T05:32:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.153+0000 7f5b5f3cd740 1 -- start start 2024-01-31T05:32:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.153+0000 7f5b5f3cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f33592dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.153+0000 7f5b5f3cd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 0x55f335935440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.153+0000 7f5b5f3cd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f33592e750 0x55f335935980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.153+0000 7f5b5f3cd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33591b0b0 con 0x55f335925980 2024-01-31T05:32:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.153+0000 7f5b5f3cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33592c660 con 0x55f335925060 2024-01-31T05:32:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.153+0000 7f5b5f3cd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f335885d30 con 0x55f33592e750 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b59aca640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 0x55f335935440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b59aca640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 0x55f335935440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48506/0 (socket says 172.21.15.45:48506) 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b59aca640 1 -- 172.21.15.45:0/4115011799 learned_addr learned my addr 172.21.15.45:0/4115011799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b5a2cb640 1 --2- 172.21.15.45:0/4115011799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f33592dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b5aacc640 1 --2- 172.21.15.45:0/4115011799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f33592e750 0x55f335935980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b59aca640 1 -- 172.21.15.45:0/4115011799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f33592e750 msgr2=0x55f335935980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b59aca640 1 --2- 172.21.15.45:0/4115011799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f33592e750 0x55f335935980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b59aca640 1 -- 172.21.15.45:0/4115011799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 msgr2=0x55f33592dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b59aca640 1 --2- 172.21.15.45:0/4115011799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f33592dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b59aca640 1 -- 172.21.15.45:0/4115011799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f335828030 con 0x55f335925980 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b5aacc640 1 --2- 172.21.15.45:0/4115011799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f33592e750 0x55f335935980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b59aca640 1 --2- 172.21.15.45:0/4115011799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 0x55f335935440 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5b44036040 tx=0x7f5b44041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b3cfa1473a1496 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b3ffff640 1 -- 172.21.15.45:0/4115011799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44055b00 con 0x55f335925980 2024-01-31T05:32:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b3ffff640 1 -- 172.21.15.45:0/4115011799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b44055040 con 0x55f335925980 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.154+0000 7f5b3ffff640 1 -- 172.21.15.45:0/4115011799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b440052f0 con 0x55f335925980 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4115011799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 msgr2=0x55f335935440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/4115011799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 0x55f335935440 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5b44036040 tx=0x7f5b44041000 comp rx=0 tx=0).stop 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4115011799 shutdown_connections 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/4115011799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f335925980 0x55f335935440 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/4115011799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f33592e750 0x55f335935980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/4115011799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f33592dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4115011799 >> 172.21.15.45:0/4115011799 conn(0x55f33591f980 msgr2=0x55f33592bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4115011799 shutdown_connections 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4115011799 wait complete. 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 -- start start 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 Processor -- start 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f33598e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925980 0x55f33598eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 0x55f335988a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f335989020 con 0x55f33592e750 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f335989190 con 0x55f335925060 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.155+0000 7f5b5f3cd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f335989350 con 0x55f335925980 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5f3cd740 10 client.0 _ll_register_callbacks cb 0x7f5b50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5aacc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 0x55f335988a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5a2cb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f33598e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b59aca640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925980 0x55f33598eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5aacc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 0x55f335988a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48514/0 (socket says 172.21.15.45:48514) 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5aacc640 1 -- 172.21.15.45:0/4049463702 learned_addr learned my addr 172.21.15.45:0/4049463702 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5a2cb640 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f33598e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:43830/0 (socket says 172.21.15.45:43830) 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5aacc640 1 -- 172.21.15.45:0/4049463702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925980 msgr2=0x55f33598eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5aacc640 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925980 0x55f33598eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5aacc640 1 -- 172.21.15.45:0/4049463702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 msgr2=0x55f33598e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5aacc640 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f33598e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5aacc640 1 -- 172.21.15.45:0/4049463702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5410e400 con 0x55f33592e750 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5a2cb640 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f33598e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5aacc640 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 0x55f335988a60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0049d0 tx=0x7f5b4c099c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a403593e9fa57cfe server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b3dffb640 1 -- 172.21.15.45:0/4049463702 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c09e170 con 0x55f33592e750 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b3dffb640 1 -- 172.21.15.45:0/4049463702 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5b4c09a230 con 0x55f33592e750 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4049463702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b50009e70 con 0x55f33592e750 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b3dffb640 1 -- 172.21.15.45:0/4049463702 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b4c09e380 con 0x55f33592e750 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5f3cd740 10 client.16179 fetch_fsmap learned FSMap version 29 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.156+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4049463702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f33584da50 con 0x55f33592e750 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.157+0000 7f5b3dffb640 1 -- 172.21.15.45:0/4049463702 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b50009e70 con 0x55f33592e750 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.157+0000 7f5b3dffb640 1 -- 172.21.15.45:0/4049463702 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5b4c01f060 con 0x55f33592e750 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.157+0000 7f5b5f3cd740 10 client.16179 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.157+0000 7f5b5f3cd740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:32:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.157+0000 7f5b5f3cd740 20 client.16179 populate_metadata read hostname 'smithi045' 2024-01-31T05:32:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.157+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f3359964f0 0x55f335998910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.157+0000 7f5b5f3cd740 4 client.16179 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:32:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.157+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4049463702 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f33558c600 con 0x55f3359964f0 2024-01-31T05:32:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.157+0000 7f5b5a2cb640 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f3359964f0 0x55f335998910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.157+0000 7f5b5a2cb640 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f3359964f0 0x55f335998910 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=245f147d2af88bbd server_cookie=793bed3c35ba4d97 in_seq=0 out_seq=0 2024-01-31T05:32:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.157+0000 7f5b3dffb640 10 client.16179 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b3dffb640 1 -- 172.21.15.45:0/4049463702 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f33558c600 con 0x55f3359964f0 2024-01-31T05:32:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b3dffb640 10 client.16179 handle_command_reply: tid=0 2024-01-31T05:32:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b5f3cd740 1 client.16179 shutdown 2024-01-31T05:32:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4049463702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 msgr2=0x55f335988a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 0x55f335988a60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0049d0 tx=0x7f5b4c099c50 comp rx=0 tx=0).stop 2024-01-31T05:32:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4049463702 shutdown_connections 2024-01-31T05:32:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f3359964f0 0x55f335998910 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f33592e750 0x55f335988a60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b3dffb640 0 client.16179 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f335925980 0x55f33598eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b5f3cd740 1 --2- 172.21.15.45:0/4049463702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f335925060 0x55f33598e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4049463702 >> 172.21.15.45:0/4049463702 conn(0x55f33591f980 msgr2=0x55f33591e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4049463702 shutdown_connections 2024-01-31T05:32:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b5f3cd740 1 -- 172.21.15.45:0/4049463702 wait complete. 2024-01-31T05:32:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.171+0000 7f5b5f3cd740 20 client.16179 trim_cache size 0 max 16384 2024-01-31T05:32:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:32:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:32:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:32:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:32:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.172+0000 7f5b5c52f640 1 -- 172.21.15.45:0/463493719 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5b400a14f0 msgr2=0x7f5b400a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.172+0000 7f5b5c52f640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5b400a14f0 0x7f5b400a39b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009230 tx=0x7f5b44005fd0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.172+0000 7f5b5c52f640 1 -- 172.21.15.45:0/463493719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5419bb60 msgr2=0x7f5b5419c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.172+0000 7f5b5c52f640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5419bb60 0x7f5b5419c010 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00eb10 tx=0x7f5b4c00c2d0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.172+0000 7f5b5c52f640 1 -- 172.21.15.45:0/463493719 shutdown_connections 2024-01-31T05:32:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.172+0000 7f5b5c52f640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5b400a14f0 0x7f5b400a39b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.172+0000 7f5b5c52f640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5419bb60 0x7f5b5419c010 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.172+0000 7f5b5c52f640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b54106070 0x7f5b541977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.172+0000 7f5b5c52f640 1 --2- 172.21.15.45:0/463493719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b540ff400 0x7f5b541972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.172+0000 7f5b5c52f640 1 -- 172.21.15.45:0/463493719 >> 172.21.15.45:0/463493719 conn(0x7f5b54076110 msgr2=0x7f5b540fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.172+0000 7f5b5c52f640 1 -- 172.21.15.45:0/463493719 shutdown_connections 2024-01-31T05:32:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:18.172+0000 7f5b5c52f640 1 -- 172.21.15.45:0/463493719 wait complete. 2024-01-31T05:32:18.794 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1029155 10.95 MB/sec execute 378 sec latency 19.831 ms 2024-01-31T05:32:19.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1030679 10.95 MB/sec execute 379 sec latency 14.486 ms 2024-01-31T05:32:19.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:19 smithi156 ceph-mon[35720]: pgmap v305: 97 pgs: 97 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 253 op/s 2024-01-31T05:32:20.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:19 smithi186 ceph-mon[35659]: pgmap v305: 97 pgs: 97 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 253 op/s 2024-01-31T05:32:20.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:19 smithi045 ceph-mon[31585]: pgmap v305: 97 pgs: 97 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 253 op/s 2024-01-31T05:32:20.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1032785 10.95 MB/sec execute 380 sec latency 18.170 ms 2024-01-31T05:32:21.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1034233 10.95 MB/sec execute 381 sec latency 17.055 ms 2024-01-31T05:32:21.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:21 smithi156 ceph-mon[35720]: pgmap v306: 97 pgs: 97 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 109 op/s 2024-01-31T05:32:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:21 smithi186 ceph-mon[35659]: pgmap v306: 97 pgs: 97 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 109 op/s 2024-01-31T05:32:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:21 smithi045 ceph-mon[31585]: pgmap v306: 97 pgs: 97 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 109 op/s 2024-01-31T05:32:22.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1035869 10.93 MB/sec execute 382 sec latency 216.459 ms 2024-01-31T05:32:23.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1037440 10.94 MB/sec execute 383 sec latency 258.047 ms 2024-01-31T05:32:24.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1038840 10.92 MB/sec execute 384 sec latency 17.681 ms 2024-01-31T05:32:24.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:24 smithi045 ceph-mon[31585]: pgmap v307: 97 pgs: 97 active+clean; 587 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 189 op/s 2024-01-31T05:32:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:24 smithi156 ceph-mon[35720]: pgmap v307: 97 pgs: 97 active+clean; 587 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 189 op/s 2024-01-31T05:32:25.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:24 smithi186 ceph-mon[35659]: pgmap v307: 97 pgs: 97 active+clean; 587 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 189 op/s 2024-01-31T05:32:25.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1040939 10.93 MB/sec execute 385 sec latency 20.430 ms 2024-01-31T05:32:26.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1042380 10.92 MB/sec execute 386 sec latency 17.092 ms 2024-01-31T05:32:26.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:26 smithi045 ceph-mon[31585]: pgmap v308: 97 pgs: 97 active+clean; 587 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 189 op/s 2024-01-31T05:32:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:26 smithi156 ceph-mon[35720]: pgmap v308: 97 pgs: 97 active+clean; 587 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 189 op/s 2024-01-31T05:32:27.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:26 smithi186 ceph-mon[35659]: pgmap v308: 97 pgs: 97 active+clean; 587 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 189 op/s 2024-01-31T05:32:27.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1044426 10.92 MB/sec execute 387 sec latency 97.385 ms 2024-01-31T05:32:28.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1045339 10.90 MB/sec execute 388 sec latency 177.689 ms 2024-01-31T05:32:28.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:28 smithi045 ceph-mon[31585]: pgmap v309: 97 pgs: 97 active+clean; 554 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-01-31T05:32:28.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:28 smithi186 ceph-mon[35659]: pgmap v309: 97 pgs: 97 active+clean; 554 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-01-31T05:32:28.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:28 smithi156 ceph-mon[35720]: pgmap v309: 97 pgs: 97 active+clean; 554 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-01-31T05:32:29.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1047944 10.91 MB/sec execute 389 sec latency 16.798 ms 2024-01-31T05:32:30.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1049037 10.90 MB/sec execute 390 sec latency 32.494 ms 2024-01-31T05:32:30.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:30 smithi045 ceph-mon[31585]: pgmap v310: 97 pgs: 97 active+clean; 554 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 162 op/s 2024-01-31T05:32:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:30 smithi156 ceph-mon[35720]: pgmap v310: 97 pgs: 97 active+clean; 554 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 162 op/s 2024-01-31T05:32:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:30 smithi186 ceph-mon[35659]: pgmap v310: 97 pgs: 97 active+clean; 554 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 162 op/s 2024-01-31T05:32:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.264+0000 7f150536b640 1 -- 172.21.15.45:0/4251356351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500105ca0 msgr2=0x7f1500106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.264+0000 7f150536b640 1 --2- 172.21.15.45:0/4251356351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500105ca0 0x7f1500106080 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009930 tx=0x7f14f002bc30 comp rx=0 tx=0).stop 2024-01-31T05:32:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.264+0000 7f150536b640 1 -- 172.21.15.45:0/4251356351 shutdown_connections 2024-01-31T05:32:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.264+0000 7f150536b640 1 --2- 172.21.15.45:0/4251356351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1500069eb0 0x7f150010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.264+0000 7f150536b640 1 --2- 172.21.15.45:0/4251356351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15000694f0 0x7f1500069970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.264+0000 7f150536b640 1 --2- 172.21.15.45:0/4251356351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500105ca0 0x7f1500106080 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.264+0000 7f150536b640 1 -- 172.21.15.45:0/4251356351 >> 172.21.15.45:0/4251356351 conn(0x7f15000769c0 msgr2=0x7f1500076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.264+0000 7f150536b640 1 -- 172.21.15.45:0/4251356351 shutdown_connections 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.264+0000 7f150536b640 1 -- 172.21.15.45:0/4251356351 wait complete. 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.264+0000 7f150536b640 1 Processor -- start 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f150536b640 1 -- start start 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f150536b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15000694f0 0x7f15001970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f150536b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500069eb0 0x7f1500197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f150536b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1500105ca0 0x7f150019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f150536b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150019bf40 con 0x7f1500069eb0 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f150536b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150019c0b0 con 0x7f1500105ca0 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f150536b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150019c270 con 0x7f15000694f0 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f14fe7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500069eb0 0x7f1500197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f14fe7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500069eb0 0x7f1500197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47184/0 (socket says 172.21.15.45:47184) 2024-01-31T05:32:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f14fe7fc640 1 -- 172.21.15.45:0/207102905 learned_addr learned my addr 172.21.15.45:0/207102905 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f14feffd640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15000694f0 0x7f15001970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f14ff7fe640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1500105ca0 0x7f150019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f14fe7fc640 1 -- 172.21.15.45:0/207102905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15000694f0 msgr2=0x7f15001970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f14fe7fc640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15000694f0 0x7f15001970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f14fe7fc640 1 -- 172.21.15.45:0/207102905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1500105ca0 msgr2=0x7f150019b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f14fe7fc640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1500105ca0 0x7f150019b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f14fe7fc640 1 -- 172.21.15.45:0/207102905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f0009560 con 0x7f1500069eb0 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f14fe7fc640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500069eb0 0x7f1500197610 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f14f400c920 tx=0x7f14f400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.265+0000 7f14e7fff640 1 -- 172.21.15.45:0/207102905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4007c70 con 0x7f1500069eb0 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.266+0000 7f150536b640 1 -- 172.21.15.45:0/207102905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150019c550 con 0x7f1500069eb0 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.266+0000 7f14e7fff640 1 -- 172.21.15.45:0/207102905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f4007dd0 con 0x7f1500069eb0 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.266+0000 7f14e7fff640 1 -- 172.21.15.45:0/207102905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f40187c0 con 0x7f1500069eb0 2024-01-31T05:32:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.266+0000 7f150536b640 1 -- 172.21.15.45:0/207102905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15001a3dd0 con 0x7f1500069eb0 2024-01-31T05:32:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f14e7fff640 1 -- 172.21.15.45:0/207102905 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f14f400f450 con 0x7f1500069eb0 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f1508209740 1 Processor -- start 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f1508209740 1 -- start start 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f14e7fff640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f14e80a14f0 0x7f14e80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f1508209740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 0x5593fa1ea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f1508209740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1ea980 0x5593fa1f3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f1508209740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1f3750 0x5593fa1f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f1508209740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa159040 con 0x5593fa1ea060 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f1508209740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa156730 con 0x5593fa1f3750 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f1508209740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa152020 con 0x5593fa1ea980 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f14ff7fe640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f14e80a14f0 0x7f14e80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f14e7fff640 1 -- 172.21.15.45:0/207102905 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f14f40ccff0 con 0x7f1500069eb0 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f14feffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 0x5593fa1ea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f14feffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 0x5593fa1ea440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47188/0 (socket says 172.21.15.45:47188) 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f14feffd640 1 -- 172.21.15.45:0/360924756 learned_addr learned my addr 172.21.15.45:0/360924756 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f14fe7fc640 1 --2- 172.21.15.45:0/360924756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1ea980 0x5593fa1f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.268+0000 7f14ff7fe640 1 --2- 172.21.15.45:0/360924756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1f3750 0x5593fa1f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.269+0000 7f14feffd640 1 -- 172.21.15.45:0/360924756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1ea980 msgr2=0x5593fa1f3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.269+0000 7f14feffd640 1 --2- 172.21.15.45:0/360924756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1ea980 0x5593fa1f3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.269+0000 7f14feffd640 1 -- 172.21.15.45:0/360924756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1f3750 msgr2=0x5593fa1f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.269+0000 7f14feffd640 1 --2- 172.21.15.45:0/360924756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1f3750 0x5593fa1f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.269+0000 7f14feffd640 1 -- 172.21.15.45:0/360924756 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593fa149e40 con 0x5593fa1ea060 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.269+0000 7f14ff7fe640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f14e80a14f0 0x7f14e80a39b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14f8005fd0 tx=0x7f14f80149d0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.269+0000 7f14feffd640 1 --2- 172.21.15.45:0/360924756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 0x5593fa1ea440 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f14f0036040 tx=0x7f14f0004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55802a8e4cd496fe server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.269+0000 7f14e57fa640 1 -- 172.21.15.45:0/360924756 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0004800 con 0x5593fa1ea060 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.269+0000 7f14e57fa640 1 -- 172.21.15.45:0/360924756 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f0004960 con 0x5593fa1ea060 2024-01-31T05:32:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.269+0000 7f14e57fa640 1 -- 172.21.15.45:0/360924756 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0005260 con 0x5593fa1ea060 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.270+0000 7f1508209740 1 -- 172.21.15.45:0/360924756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 msgr2=0x5593fa1ea440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.270+0000 7f1508209740 1 --2- 172.21.15.45:0/360924756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 0x5593fa1ea440 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f14f0036040 tx=0x7f14f0004e80 comp rx=0 tx=0).stop 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.270+0000 7f1508209740 1 -- 172.21.15.45:0/360924756 shutdown_connections 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.270+0000 7f1508209740 1 --2- 172.21.15.45:0/360924756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1f3750 0x5593fa1f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.270+0000 7f1508209740 1 --2- 172.21.15.45:0/360924756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1ea980 0x5593fa1f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.270+0000 7f1508209740 1 --2- 172.21.15.45:0/360924756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 0x5593fa1ea440 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.270+0000 7f1508209740 1 -- 172.21.15.45:0/360924756 >> 172.21.15.45:0/360924756 conn(0x5593fa1e4980 msgr2=0x5593fa1e4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.270+0000 7f1508209740 1 -- 172.21.15.45:0/360924756 shutdown_connections 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.270+0000 7f1508209740 1 -- 172.21.15.45:0/360924756 wait complete. 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.270+0000 7f1508209740 1 -- start start 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.270+0000 7f1508209740 1 Processor -- start 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f1508209740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea060 0x5593fa1f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f1508209740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea980 0x5593fa1f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f1508209740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 0x5593fa1f1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f1508209740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa10b1b0 con 0x5593fa1ea980 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f1508209740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa10ae50 con 0x5593fa1ea060 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f1508209740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa0d0090 con 0x5593fa1f3750 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14feffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea060 0x5593fa1f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14fe7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea980 0x5593fa1f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f1508209740 10 client.0 _ll_register_callbacks cb 0x5593fa0f5a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14ff7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 0x5593fa1f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14ff7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 0x5593fa1f1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:56602/0 (socket says 172.21.15.45:56602) 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14fe7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea980 0x5593fa1f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47190/0 (socket says 172.21.15.45:47190) 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14feffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea060 0x5593fa1f05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:60078/0 (socket says 172.21.15.45:60078) 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14ff7fe640 1 -- 172.21.15.45:0/378542721 learned_addr learned my addr 172.21.15.45:0/378542721 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14fe7fc640 1 -- 172.21.15.45:0/378542721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 msgr2=0x5593fa1f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14fe7fc640 1 --2- 172.21.15.45:0/378542721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 0x5593fa1f1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14fe7fc640 1 -- 172.21.15.45:0/378542721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea060 msgr2=0x5593fa1f05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14fe7fc640 1 --2- 172.21.15.45:0/378542721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea060 0x5593fa1f05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14fe7fc640 1 -- 172.21.15.45:0/378542721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150010e470 con 0x5593fa1ea980 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14feffd640 1 --2- 172.21.15.45:0/378542721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea060 0x5593fa1f05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14fe7fc640 1 --2- 172.21.15.45:0/378542721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea980 0x5593fa1f0b00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f14f40ca760 tx=0x7f14f40999c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63a5b51c99cc7f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.271+0000 7f14e2ff5640 1 -- 172.21.15.45:0/378542721 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f409a510 con 0x5593fa1ea980 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f14e2ff5640 1 -- 172.21.15.45:0/378542721 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f14f400ed80 con 0x5593fa1ea980 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f1508209740 1 -- 172.21.15.45:0/378542721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593fa14b010 con 0x5593fa1ea980 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f14e2ff5640 1 -- 172.21.15.45:0/378542721 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14f4099e10 con 0x5593fa1ea980 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f1508209740 10 client.16191 fetch_fsmap learned FSMap version 29 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f1508209740 1 -- 172.21.15.45:0/378542721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5593fa1506a0 con 0x5593fa1ea980 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f14e2ff5640 1 -- 172.21.15.45:0/378542721 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f407c020 con 0x5593fa1ea980 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f14e2ff5640 1 -- 172.21.15.45:0/378542721 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f14f4016020 con 0x5593fa1ea980 2024-01-31T05:32:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f1508209740 10 client.16191 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f1508209740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f1508209740 20 client.16191 populate_metadata read hostname 'smithi045' 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f1508209740 1 --2- 172.21.15.45:0/378542721 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5593fa2032e0 0x5593fa2056d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f1508209740 4 client.16191 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f1508209740 1 -- 172.21.15.45:0/378542721 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593fa14ad30 con 0x5593fa2032e0 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.272+0000 7f14feffd640 1 --2- 172.21.15.45:0/378542721 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5593fa2032e0 0x5593fa2056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.273+0000 7f14e2ff5640 10 client.16191 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.273+0000 7f14feffd640 1 --2- 172.21.15.45:0/378542721 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5593fa2032e0 0x5593fa2056d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7833c61bbd595ba3 server_cookie=1ad657b970d9a54d in_seq=0 out_seq=0 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.273+0000 7f14e2ff5640 1 -- 172.21.15.45:0/378542721 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5593fa14ad30 con 0x5593fa2032e0 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.273+0000 7f14e2ff5640 10 client.16191 handle_command_reply: tid=0 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.273+0000 7f1508209740 1 client.16191 shutdown 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.274+0000 7f1508209740 1 -- 172.21.15.45:0/378542721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea980 msgr2=0x5593fa1f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.274+0000 7f1508209740 1 --2- 172.21.15.45:0/378542721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea980 0x5593fa1f0b00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f14f40ca760 tx=0x7f14f40999c0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.274+0000 7f1508209740 1 -- 172.21.15.45:0/378542721 shutdown_connections 2024-01-31T05:32:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.274+0000 7f1508209740 1 --2- 172.21.15.45:0/378542721 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5593fa2032e0 0x5593fa2056d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.274+0000 7f1508209740 1 --2- 172.21.15.45:0/378542721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea980 0x5593fa1f0b00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.274+0000 7f1508209740 1 --2- 172.21.15.45:0/378542721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 0x5593fa1f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.274+0000 7f14e2ff5640 0 client.16191 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.274+0000 7f1508209740 1 --2- 172.21.15.45:0/378542721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea060 0x5593fa1f05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.274+0000 7f1508209740 1 -- 172.21.15.45:0/378542721 >> 172.21.15.45:0/378542721 conn(0x5593fa1e4980 msgr2=0x5593fa1e66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.274+0000 7f1508209740 1 -- 172.21.15.45:0/378542721 shutdown_connections 2024-01-31T05:32:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.274+0000 7f1508209740 1 -- 172.21.15.45:0/378542721 wait complete. 2024-01-31T05:32:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.274+0000 7f1508209740 20 client.16191 trim_cache size 0 max 16384 2024-01-31T05:32:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.288+0000 7f1508209740 1 Processor -- start 2024-01-31T05:32:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.288+0000 7f1508209740 1 -- start start 2024-01-31T05:32:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.288+0000 7f1508209740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 0x5593fa1f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.288+0000 7f1508209740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea980 0x5593fa1f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.288+0000 7f1508209740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 0x5593fa1ef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.288+0000 7f1508209740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa1e99d0 con 0x5593fa1ea060 2024-01-31T05:32:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.288+0000 7f1508209740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa1e5550 con 0x5593fa1ea980 2024-01-31T05:32:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.288+0000 7f1508209740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa1f35a0 con 0x5593fa1f3750 2024-01-31T05:32:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.288+0000 7f14feffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 0x5593fa1f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14ff7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 0x5593fa1ef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14feffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 0x5593fa1f2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47198/0 (socket says 172.21.15.45:47198) 2024-01-31T05:32:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14ff7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 0x5593fa1ef5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:56604/0 (socket says 172.21.15.45:56604) 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14fe7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea980 0x5593fa1f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14feffd640 1 -- 172.21.15.45:0/235657024 learned_addr learned my addr 172.21.15.45:0/235657024 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14feffd640 1 -- 172.21.15.45:0/235657024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 msgr2=0x5593fa1ef5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14feffd640 1 --2- 172.21.15.45:0/235657024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 0x5593fa1ef5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14feffd640 1 -- 172.21.15.45:0/235657024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea980 msgr2=0x5593fa1f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14feffd640 1 --2- 172.21.15.45:0/235657024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea980 0x5593fa1f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14feffd640 1 -- 172.21.15.45:0/235657024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593fa0ed030 con 0x5593fa1ea060 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14fe7fc640 1 --2- 172.21.15.45:0/235657024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea980 0x5593fa1f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14ff7fe640 1 --2- 172.21.15.45:0/235657024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 0x5593fa1ef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14feffd640 1 --2- 172.21.15.45:0/235657024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 0x5593fa1f2cf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f14f00044c0 tx=0x7f14f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400f39bb4ef2a95c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14e4ff9640 1 -- 172.21.15.45:0/235657024 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0004ad0 con 0x5593fa1ea060 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14e4ff9640 1 -- 172.21.15.45:0/235657024 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f0053840 con 0x5593fa1ea060 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.289+0000 7f14e4ff9640 1 -- 172.21.15.45:0/235657024 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f004a450 con 0x5593fa1ea060 2024-01-31T05:32:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.290+0000 7f1508209740 1 -- 172.21.15.45:0/235657024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 msgr2=0x5593fa1f2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.290+0000 7f1508209740 1 --2- 172.21.15.45:0/235657024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 0x5593fa1f2cf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f14f00044c0 tx=0x7f14f0000bc0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.290+0000 7f1508209740 1 -- 172.21.15.45:0/235657024 shutdown_connections 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.290+0000 7f1508209740 1 --2- 172.21.15.45:0/235657024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1f3750 0x5593fa1ef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.290+0000 7f1508209740 1 --2- 172.21.15.45:0/235657024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea980 0x5593fa1f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.290+0000 7f1508209740 1 --2- 172.21.15.45:0/235657024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1ea060 0x5593fa1f2cf0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.290+0000 7f1508209740 1 -- 172.21.15.45:0/235657024 >> 172.21.15.45:0/235657024 conn(0x5593fa1e4980 msgr2=0x5593fa1eee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.290+0000 7f1508209740 1 -- 172.21.15.45:0/235657024 shutdown_connections 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.290+0000 7f1508209740 1 -- 172.21.15.45:0/235657024 wait complete. 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.290+0000 7f1508209740 1 -- start start 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.290+0000 7f1508209740 1 Processor -- start 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f1508209740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea060 0x5593fa1f8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f1508209740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1ea980 0x5593fa1f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f1508209740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1f3750 0x5593fa2587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f1508209740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa1f8e10 con 0x5593fa1f3750 2024-01-31T05:32:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f1508209740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa1f8f80 con 0x5593fa1ea060 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f1508209740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa258d00 con 0x5593fa1ea980 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f1508209740 10 client.0 _ll_register_callbacks cb 0x7f14f0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1f3750 0x5593fa2587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1f3750 0x5593fa2587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47202/0 (socket says 172.21.15.45:47202) 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14fe7fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1ea980 0x5593fa1f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14ff7fe640 1 -- 172.21.15.45:0/4275617306 learned_addr learned my addr 172.21.15.45:0/4275617306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14feffd640 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea060 0x5593fa1f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14ff7fe640 1 -- 172.21.15.45:0/4275617306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1ea980 msgr2=0x5593fa1f86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14ff7fe640 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1ea980 0x5593fa1f86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14ff7fe640 1 -- 172.21.15.45:0/4275617306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea060 msgr2=0x5593fa1f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14ff7fe640 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea060 0x5593fa1f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14ff7fe640 1 -- 172.21.15.45:0/4275617306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593fa1506a0 con 0x5593fa1f3750 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14fe7fc640 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1ea980 0x5593fa1f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14ff7fe640 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1f3750 0x5593fa2587c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f14f8012f20 tx=0x7f14f8016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7788255b38b7ed7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14e2ff5640 1 -- 172.21.15.45:0/4275617306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8025040 con 0x5593fa1f3750 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f1508209740 1 -- 172.21.15.45:0/4275617306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14f0004960 con 0x5593fa1f3750 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.291+0000 7f14e2ff5640 1 -- 172.21.15.45:0/4275617306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f14f802b5f0 con 0x5593fa1f3750 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.292+0000 7f14e2ff5640 1 -- 172.21.15.45:0/4275617306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14f802b7b0 con 0x5593fa1f3750 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.292+0000 7f1508209740 10 client.16197 fetch_fsmap learned FSMap version 29 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.292+0000 7f1508209740 1 -- 172.21.15.45:0/4275617306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f14f801a070 con 0x5593fa1f3750 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.292+0000 7f14e2ff5640 1 -- 172.21.15.45:0/4275617306 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f8038020 con 0x5593fa1f3750 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.292+0000 7f14e2ff5640 1 -- 172.21.15.45:0/4275617306 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f14f801faa0 con 0x5593fa1f3750 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.292+0000 7f1508209740 10 client.16197 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.292+0000 7f1508209740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.292+0000 7f1508209740 20 client.16197 populate_metadata read hostname 'smithi045' 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.292+0000 7f1508209740 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5593fa25a4e0 0x5593fa25c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.292+0000 7f1508209740 4 client.16197 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.292+0000 7f1508209740 1 -- 172.21.15.45:0/4275617306 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5593f9e51600 con 0x5593fa25a4e0 2024-01-31T05:32:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.292+0000 7f14feffd640 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5593fa25a4e0 0x5593fa25c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.293+0000 7f14e2ff5640 10 client.16197 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.293+0000 7f14feffd640 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5593fa25a4e0 0x5593fa25c960 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5784d07b4314d692 server_cookie=cde992420b5ff42 in_seq=0 out_seq=0 2024-01-31T05:32:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.303+0000 7f14e2ff5640 1 -- 172.21.15.45:0/4275617306 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5593f9e51600 con 0x5593fa25a4e0 2024-01-31T05:32:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.303+0000 7f14e2ff5640 10 client.16197 handle_command_reply: tid=0 2024-01-31T05:32:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.303+0000 7f1508209740 1 client.16197 shutdown 2024-01-31T05:32:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f1508209740 1 -- 172.21.15.45:0/4275617306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1f3750 msgr2=0x5593fa2587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f1508209740 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1f3750 0x5593fa2587c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f14f8012f20 tx=0x7f14f8016040 comp rx=0 tx=0).stop 2024-01-31T05:32:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f1508209740 1 -- 172.21.15.45:0/4275617306 shutdown_connections 2024-01-31T05:32:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f1508209740 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5593fa25a4e0 0x5593fa25c960 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f1508209740 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593fa1f3750 0x5593fa2587c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f1508209740 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5593fa1ea980 0x5593fa1f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f14e2ff5640 0 client.16197 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f1508209740 1 --2- 172.21.15.45:0/4275617306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593fa1ea060 0x5593fa1f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f1508209740 1 -- 172.21.15.45:0/4275617306 >> 172.21.15.45:0/4275617306 conn(0x5593fa1e4980 msgr2=0x5593fa1e5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f1508209740 1 -- 172.21.15.45:0/4275617306 shutdown_connections 2024-01-31T05:32:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f1508209740 1 -- 172.21.15.45:0/4275617306 wait complete. 2024-01-31T05:32:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f1508209740 20 client.16197 trim_cache size 0 max 16384 2024-01-31T05:32:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:32:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:32:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:32:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:32:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f150536b640 1 -- 172.21.15.45:0/207102905 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f14e80a14f0 msgr2=0x7f14e80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f150536b640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f14e80a14f0 0x7f14e80a39b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14f8005fd0 tx=0x7f14f80149d0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f150536b640 1 -- 172.21.15.45:0/207102905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500069eb0 msgr2=0x7f1500197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.304+0000 7f150536b640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500069eb0 0x7f1500197610 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f14f400c920 tx=0x7f14f400cde0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.305+0000 7f150536b640 1 -- 172.21.15.45:0/207102905 shutdown_connections 2024-01-31T05:32:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.305+0000 7f150536b640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f14e80a14f0 0x7f14e80a39b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.305+0000 7f150536b640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500069eb0 0x7f1500197610 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.305+0000 7f150536b640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1500105ca0 0x7f150019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.305+0000 7f150536b640 1 --2- 172.21.15.45:0/207102905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15000694f0 0x7f15001970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.305+0000 7f150536b640 1 -- 172.21.15.45:0/207102905 >> 172.21.15.45:0/207102905 conn(0x7f15000769c0 msgr2=0x7f15000fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.305+0000 7f150536b640 1 -- 172.21.15.45:0/207102905 shutdown_connections 2024-01-31T05:32:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:31.305+0000 7f150536b640 1 -- 172.21.15.45:0/207102905 wait complete. 2024-01-31T05:32:31.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1051490 10.91 MB/sec execute 391 sec latency 16.397 ms 2024-01-31T05:32:32.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1052260 10.88 MB/sec execute 392 sec latency 83.826 ms 2024-01-31T05:32:32.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:32 smithi045 ceph-mon[31585]: pgmap v311: 97 pgs: 97 active+clean; 554 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 162 op/s 2024-01-31T05:32:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:32 smithi156 ceph-mon[35720]: pgmap v311: 97 pgs: 97 active+clean; 554 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 162 op/s 2024-01-31T05:32:33.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:32 smithi186 ceph-mon[35659]: pgmap v311: 97 pgs: 97 active+clean; 554 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 162 op/s 2024-01-31T05:32:33.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1054105 10.88 MB/sec execute 393 sec latency 47.592 ms 2024-01-31T05:32:34.795 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1055609 10.88 MB/sec execute 394 sec latency 13.444 ms 2024-01-31T05:32:34.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:34 smithi045 ceph-mon[31585]: pgmap v312: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 244 op/s 2024-01-31T05:32:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:34 smithi156 ceph-mon[35720]: pgmap v312: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 244 op/s 2024-01-31T05:32:35.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:34 smithi186 ceph-mon[35659]: pgmap v312: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 244 op/s 2024-01-31T05:32:35.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1057816 10.88 MB/sec execute 395 sec latency 27.147 ms 2024-01-31T05:32:36.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1059156 10.87 MB/sec execute 396 sec latency 27.109 ms 2024-01-31T05:32:36.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:36 smithi045 ceph-mon[31585]: pgmap v313: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-01-31T05:32:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:36 smithi156 ceph-mon[35720]: pgmap v313: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-01-31T05:32:37.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:36 smithi186 ceph-mon[35659]: pgmap v313: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-01-31T05:32:37.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1061496 10.87 MB/sec execute 397 sec latency 17.941 ms 2024-01-31T05:32:38.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1062506 10.86 MB/sec execute 398 sec latency 95.705 ms 2024-01-31T05:32:38.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:38 smithi045 ceph-mon[31585]: pgmap v314: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 249 op/s 2024-01-31T05:32:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:38 smithi156 ceph-mon[35720]: pgmap v314: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 249 op/s 2024-01-31T05:32:39.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:38 smithi186 ceph-mon[35659]: pgmap v314: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 249 op/s 2024-01-31T05:32:39.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1063842 10.85 MB/sec execute 399 sec latency 147.712 ms 2024-01-31T05:32:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:39 smithi156 ceph-mon[35720]: pgmap v315: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 163 op/s 2024-01-31T05:32:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:39 smithi186 ceph-mon[35659]: pgmap v315: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 163 op/s 2024-01-31T05:32:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:39 smithi045 ceph-mon[31585]: pgmap v315: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 163 op/s 2024-01-31T05:32:40.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1065836 10.85 MB/sec execute 400 sec latency 15.584 ms 2024-01-31T05:32:41.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1067392 10.84 MB/sec execute 401 sec latency 19.873 ms 2024-01-31T05:32:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:41 smithi156 ceph-mon[35720]: pgmap v316: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 163 op/s 2024-01-31T05:32:42.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:41 smithi186 ceph-mon[35659]: pgmap v316: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 163 op/s 2024-01-31T05:32:42.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:41 smithi045 ceph-mon[31585]: pgmap v316: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 163 op/s 2024-01-31T05:32:42.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1069452 10.85 MB/sec execute 402 sec latency 17.172 ms 2024-01-31T05:32:43.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1070876 10.84 MB/sec execute 403 sec latency 73.725 ms 2024-01-31T05:32:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.392+0000 7fb0290a7640 1 -- 172.21.15.45:0/1718757995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb024104b90 msgr2=0x7fb024104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.392+0000 7fb0290a7640 1 --2- 172.21.15.45:0/1718757995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb024104b90 0x7fb024104f70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb014009900 tx=0x7fb01402bc30 comp rx=0 tx=0).stop 2024-01-31T05:32:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.392+0000 7fb0290a7640 1 -- 172.21.15.45:0/1718757995 shutdown_connections 2024-01-31T05:32:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.392+0000 7fb0290a7640 1 --2- 172.21.15.45:0/1718757995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb024100800 0x7fb02410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.392+0000 7fb0290a7640 1 --2- 172.21.15.45:0/1718757995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0240ffe40 0x7fb0241002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.392+0000 7fb0290a7640 1 --2- 172.21.15.45:0/1718757995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb024104b90 0x7fb024104f70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.392+0000 7fb0290a7640 1 -- 172.21.15.45:0/1718757995 >> 172.21.15.45:0/1718757995 conn(0x7fb024076a40 msgr2=0x7fb024076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.392+0000 7fb0290a7640 1 -- 172.21.15.45:0/1718757995 shutdown_connections 2024-01-31T05:32:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.393+0000 7fb0290a7640 1 -- 172.21.15.45:0/1718757995 wait complete. 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.393+0000 7fb0290a7640 1 Processor -- start 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.393+0000 7fb0290a7640 1 -- start start 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.393+0000 7fb0290a7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0240ffe40 0x7fb0241972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.393+0000 7fb0290a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb024100800 0x7fb0241977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.393+0000 7fb0290a7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb024104b90 0x7fb02419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.393+0000 7fb0290a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02419c170 con 0x7fb024100800 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.393+0000 7fb0290a7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02419c2e0 con 0x7fb024104b90 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.393+0000 7fb0290a7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02419c4a0 con 0x7fb0240ffe40 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.393+0000 7fb01bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb024100800 0x7fb0241977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb01bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb024100800 0x7fb0241977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42988/0 (socket says 172.21.15.45:42988) 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb01bfff640 1 -- 172.21.15.45:0/100627431 learned_addr learned my addr 172.21.15.45:0/100627431 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb022d9d640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0240ffe40 0x7fb0241972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb02359e640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb024104b90 0x7fb02419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb01bfff640 1 -- 172.21.15.45:0/100627431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0240ffe40 msgr2=0x7fb0241972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb01bfff640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0240ffe40 0x7fb0241972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb01bfff640 1 -- 172.21.15.45:0/100627431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb024104b90 msgr2=0x7fb02419bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb01bfff640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb024104b90 0x7fb02419bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb01bfff640 1 -- 172.21.15.45:0/100627431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb014009560 con 0x7fb024100800 2024-01-31T05:32:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb01bfff640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb024100800 0x7fb0241977f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00e9f0 tx=0x7fb00c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb020d99640 1 -- 172.21.15.45:0/100627431 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c009720 con 0x7fb024100800 2024-01-31T05:32:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb020d99640 1 -- 172.21.15.45:0/100627431 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb00c00cd80 con 0x7fb024100800 2024-01-31T05:32:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb0290a7640 1 -- 172.21.15.45:0/100627431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02419c780 con 0x7fb024100800 2024-01-31T05:32:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb020d99640 1 -- 172.21.15.45:0/100627431 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c010430 con 0x7fb024100800 2024-01-31T05:32:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.394+0000 7fb0290a7640 1 -- 172.21.15.45:0/100627431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0241a4000 con 0x7fb024100800 2024-01-31T05:32:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.397+0000 7fb020d99640 1 -- 172.21.15.45:0/100627431 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fb00c005020 con 0x7fb024100800 2024-01-31T05:32:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.397+0000 7fb02be6b740 1 Processor -- start 2024-01-31T05:32:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.397+0000 7fb02be6b740 1 -- start start 2024-01-31T05:32:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.397+0000 7fb020d99640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb0100a14f0 0x7fb0100a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb02359e640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb0100a14f0 0x7fb0100a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb020d99640 1 -- 172.21.15.45:0/100627431 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb00c014070 con 0x7fb024100800 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb02be6b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 0x561b96283e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb02be6b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96284340 0x561b96291620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb02be6b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96291b60 0x561b96293f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb02be6b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b961f3040 con 0x561b96283a20 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb02be6b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b961f0730 con 0x561b96284340 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb02be6b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b961ec020 con 0x561b96291b60 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 0x561b96283e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 0x561b96283e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42998/0 (socket says 172.21.15.45:42998) 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb022d9d640 1 -- 172.21.15.45:0/126809058 learned_addr learned my addr 172.21.15.45:0/126809058 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb01bfff640 1 --2- 172.21.15.45:0/126809058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96284340 0x561b96291620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb02359e640 1 --2- 172.21.15.45:0/126809058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96291b60 0x561b96293f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb022d9d640 1 -- 172.21.15.45:0/126809058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96291b60 msgr2=0x561b96293f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb022d9d640 1 --2- 172.21.15.45:0/126809058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96291b60 0x561b96293f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb022d9d640 1 -- 172.21.15.45:0/126809058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96284340 msgr2=0x561b96291620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb022d9d640 1 --2- 172.21.15.45:0/126809058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96284340 0x561b96291620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb022d9d640 1 -- 172.21.15.45:0/126809058 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b961e3e40 con 0x561b96283a20 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb01bfff640 1 --2- 172.21.15.45:0/126809058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96284340 0x561b96291620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb02359e640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb0100a14f0 0x7fb0100a39b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009fd0 tx=0x7fb01c011bc0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.398+0000 7fb022d9d640 1 --2- 172.21.15.45:0/126809058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 0x561b96283e00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb014000c00 tx=0x7fb01402c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97608f982ba16a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.399+0000 7fb0197fa640 1 -- 172.21.15.45:0/126809058 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014004800 con 0x561b96283a20 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.399+0000 7fb0197fa640 1 -- 172.21.15.45:0/126809058 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb014004960 con 0x561b96283a20 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.399+0000 7fb0197fa640 1 -- 172.21.15.45:0/126809058 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014003850 con 0x561b96283a20 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.399+0000 7fb02be6b740 1 -- 172.21.15.45:0/126809058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 msgr2=0x561b96283e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.399+0000 7fb02be6b740 1 --2- 172.21.15.45:0/126809058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 0x561b96283e00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb014000c00 tx=0x7fb01402c670 comp rx=0 tx=0).stop 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.399+0000 7fb02be6b740 1 -- 172.21.15.45:0/126809058 shutdown_connections 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.399+0000 7fb02be6b740 1 --2- 172.21.15.45:0/126809058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96291b60 0x561b96293f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.399+0000 7fb02be6b740 1 --2- 172.21.15.45:0/126809058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96284340 0x561b96291620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.399+0000 7fb02be6b740 1 --2- 172.21.15.45:0/126809058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 0x561b96283e00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.399+0000 7fb02be6b740 1 -- 172.21.15.45:0/126809058 >> 172.21.15.45:0/126809058 conn(0x561b9627e980 msgr2=0x561b9627ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.399+0000 7fb02be6b740 1 -- 172.21.15.45:0/126809058 shutdown_connections 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.399+0000 7fb02be6b740 1 -- 172.21.15.45:0/126809058 wait complete. 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb02be6b740 1 -- start start 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb02be6b740 1 Processor -- start 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb02be6b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96283a20 0x561b9628ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb02be6b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 0x561b9628efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb02be6b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96291b60 0x561b9628f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb02be6b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b961a51b0 con 0x561b96284340 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb02be6b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b961a4e50 con 0x561b96283a20 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb02be6b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9616a090 con 0x561b96291b60 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb02be6b740 10 client.0 _ll_register_callbacks cb 0x561b9618fa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb01bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 0x561b9628efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb01bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 0x561b9628efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43000/0 (socket says 172.21.15.45:43000) 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb02359e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96291b60 0x561b9628f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb01bfff640 1 -- 172.21.15.45:0/1274975480 learned_addr learned my addr 172.21.15.45:0/1274975480 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb022d9d640 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96283a20 0x561b9628ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb01bfff640 1 -- 172.21.15.45:0/1274975480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96291b60 msgr2=0x561b9628f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb01bfff640 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96291b60 0x561b9628f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.400+0000 7fb01bfff640 1 -- 172.21.15.45:0/1274975480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96283a20 msgr2=0x561b9628ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb01bfff640 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96283a20 0x561b9628ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb01bfff640 1 -- 172.21.15.45:0/1274975480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02410d3d0 con 0x561b96284340 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb022d9d640 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96283a20 0x561b9628ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb02359e640 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96291b60 0x561b9628f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb01bfff640 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 0x561b9628efc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0caa40 tx=0x7fb00c005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225ee73cffc3b530 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb00a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1274975480 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0cc070 con 0x561b96284340 2024-01-31T05:32:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1274975480 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb00c0a2420 con 0x561b96284340 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb02be6b740 1 -- 172.21.15.45:0/1274975480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b961e5010 con 0x561b96284340 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1274975480 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb00c0052d0 con 0x561b96284340 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb02be6b740 10 client.16209 fetch_fsmap learned FSMap version 29 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb02be6b740 1 -- 172.21.15.45:0/1274975480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561b961ea6a0 con 0x561b96284340 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1274975480 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b961e5010 con 0x561b96284340 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1274975480 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb00c01f060 con 0x561b96284340 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb02be6b740 10 client.16209 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb02be6b740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb02be6b740 20 client.16209 populate_metadata read hostname 'smithi045' 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561b962a1880 0x561b962a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb02be6b740 4 client.16209 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.401+0000 7fb02be6b740 1 -- 172.21.15.45:0/1274975480 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b961e4d30 con 0x561b962a1880 2024-01-31T05:32:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.402+0000 7fb022d9d640 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561b962a1880 0x561b962a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.402+0000 7fb022d9d640 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561b962a1880 0x561b962a3c70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde0727f5d7bbb06 server_cookie=aa654066913bcd1a in_seq=0 out_seq=0 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.402+0000 7fb00a7fc640 10 client.16209 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1274975480 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b961e4d30 con 0x561b962a1880 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb00a7fc640 10 client.16209 handle_command_reply: tid=0 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb02be6b740 1 client.16209 shutdown 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb02be6b740 1 -- 172.21.15.45:0/1274975480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 msgr2=0x561b9628efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 0x561b9628efc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0caa40 tx=0x7fb00c005730 comp rx=0 tx=0).stop 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb02be6b740 1 -- 172.21.15.45:0/1274975480 shutdown_connections 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561b962a1880 0x561b962a3c70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 0x561b9628efc0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96291b60 0x561b9628f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb00a7fc640 0 client.16209 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1274975480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96283a20 0x561b9628ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb02be6b740 1 -- 172.21.15.45:0/1274975480 >> 172.21.15.45:0/1274975480 conn(0x561b9627e980 msgr2=0x561b962806d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb02be6b740 1 -- 172.21.15.45:0/1274975480 shutdown_connections 2024-01-31T05:32:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb02be6b740 1 -- 172.21.15.45:0/1274975480 wait complete. 2024-01-31T05:32:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.403+0000 7fb02be6b740 20 client.16209 trim_cache size 0 max 16384 2024-01-31T05:32:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.416+0000 7fb02be6b740 1 Processor -- start 2024-01-31T05:32:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.416+0000 7fb02be6b740 1 -- start start 2024-01-31T05:32:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.416+0000 7fb02be6b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96283a20 0x561b96291160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.416+0000 7fb02be6b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 0x561b9628f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.416+0000 7fb02be6b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96291b60 0x561b9628bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.416+0000 7fb02be6b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9627a0b0 con 0x561b96284340 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.416+0000 7fb02be6b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9627f9b0 con 0x561b96291b60 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.416+0000 7fb02be6b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9627e1a0 con 0x561b96283a20 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.416+0000 7fb01bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 0x561b9628f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb01bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 0x561b9628f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43006/0 (socket says 172.21.15.45:43006) 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb01bfff640 1 -- 172.21.15.45:0/1814530277 learned_addr learned my addr 172.21.15.45:0/1814530277 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb022d9d640 1 --2- 172.21.15.45:0/1814530277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96283a20 0x561b96291160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb02359e640 1 --2- 172.21.15.45:0/1814530277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96291b60 0x561b9628bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb01bfff640 1 -- 172.21.15.45:0/1814530277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96283a20 msgr2=0x561b96291160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb01bfff640 1 --2- 172.21.15.45:0/1814530277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96283a20 0x561b96291160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb01bfff640 1 -- 172.21.15.45:0/1814530277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96291b60 msgr2=0x561b9628bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb01bfff640 1 --2- 172.21.15.45:0/1814530277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96291b60 0x561b9628bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb01bfff640 1 -- 172.21.15.45:0/1814530277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b96187030 con 0x561b96284340 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb01bfff640 1 --2- 172.21.15.45:0/1814530277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 0x561b9628f320 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb00c096590 tx=0x7fb00c099e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660a0e34a42ea996 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb018ff9640 1 -- 172.21.15.45:0/1814530277 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0cc030 con 0x561b96284340 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb018ff9640 1 -- 172.21.15.45:0/1814530277 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb00c0cd040 con 0x561b96284340 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.417+0000 7fb018ff9640 1 -- 172.21.15.45:0/1814530277 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0a2420 con 0x561b96284340 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.418+0000 7fb02be6b740 1 -- 172.21.15.45:0/1814530277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 msgr2=0x561b9628f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.418+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1814530277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 0x561b9628f320 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb00c096590 tx=0x7fb00c099e60 comp rx=0 tx=0).stop 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.418+0000 7fb02be6b740 1 -- 172.21.15.45:0/1814530277 shutdown_connections 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.418+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1814530277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96284340 0x561b9628f320 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.418+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1814530277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96291b60 0x561b9628bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.418+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1814530277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96283a20 0x561b96291160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.418+0000 7fb02be6b740 1 -- 172.21.15.45:0/1814530277 >> 172.21.15.45:0/1814530277 conn(0x561b9627e980 msgr2=0x561b96289e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.418+0000 7fb02be6b740 1 -- 172.21.15.45:0/1814530277 shutdown_connections 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.418+0000 7fb02be6b740 1 -- 172.21.15.45:0/1814530277 wait complete. 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.418+0000 7fb02be6b740 1 -- start start 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.418+0000 7fb02be6b740 1 Processor -- start 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb02be6b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 0x561b96298af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb02be6b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96284340 0x561b96296e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb02be6b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96291b60 0x561b962973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb02be6b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b96297a50 con 0x561b96283a20 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb02be6b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b96297bc0 con 0x561b96291b60 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb02be6b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b962f7d60 con 0x561b96284340 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb02be6b740 10 client.0 _ll_register_callbacks cb 0x7fb00c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 0x561b96298af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 0x561b96298af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43022/0 (socket says 172.21.15.45:43022) 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb022d9d640 1 -- 172.21.15.45:0/1686549821 learned_addr learned my addr 172.21.15.45:0/1686549821 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb01bfff640 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96284340 0x561b96296e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb02359e640 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96291b60 0x561b962973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb022d9d640 1 -- 172.21.15.45:0/1686549821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96284340 msgr2=0x561b96296e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb022d9d640 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96284340 0x561b96296e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb022d9d640 1 -- 172.21.15.45:0/1686549821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96291b60 msgr2=0x561b962973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb022d9d640 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96291b60 0x561b962973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb022d9d640 1 -- 172.21.15.45:0/1686549821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b961ea6a0 con 0x561b96283a20 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb022d9d640 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 0x561b96298af0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb014004460 tx=0x7fb014004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d963bd3660ab04bb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb00a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1686549821 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014059070 con 0x561b96283a20 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1686549821 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb01404f070 con 0x561b96283a20 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.419+0000 7fb02be6b740 1 -- 172.21.15.45:0/1686549821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb014004960 con 0x561b96283a20 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1686549821 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb014004800 con 0x561b96283a20 2024-01-31T05:32:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb02be6b740 10 client.16215 fetch_fsmap learned FSMap version 29 2024-01-31T05:32:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb02be6b740 1 -- 172.21.15.45:0/1686549821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561b961aca50 con 0x561b96283a20 2024-01-31T05:32:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1686549821 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb014004960 con 0x561b96283a20 2024-01-31T05:32:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1686549821 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb01405c020 con 0x561b96283a20 2024-01-31T05:32:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb02be6b740 10 client.16215 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:32:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb02be6b740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:32:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb02be6b740 20 client.16215 populate_metadata read hostname 'smithi045' 2024-01-31T05:32:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561b962f9000 0x561b962fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb02be6b740 4 client.16215 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:32:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb02be6b740 1 -- 172.21.15.45:0/1686549821 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561b95eeb600 con 0x561b962f9000 2024-01-31T05:32:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb01bfff640 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561b962f9000 0x561b962fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb00a7fc640 10 client.16215 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.420+0000 7fb01bfff640 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561b962f9000 0x561b962fb3f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef2611359bd5b1f server_cookie=f3fd1b6b0c357a56 in_seq=0 out_seq=0 2024-01-31T05:32:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.433+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1686549821 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561b95eeb600 con 0x561b962f9000 2024-01-31T05:32:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.433+0000 7fb00a7fc640 10 client.16215 handle_command_reply: tid=0 2024-01-31T05:32:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.433+0000 7fb02be6b740 1 client.16215 shutdown 2024-01-31T05:32:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb02be6b740 1 -- 172.21.15.45:0/1686549821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 msgr2=0x561b96298af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 0x561b96298af0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb014004460 tx=0x7fb014004530 comp rx=0 tx=0).stop 2024-01-31T05:32:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb02be6b740 1 -- 172.21.15.45:0/1686549821 shutdown_connections 2024-01-31T05:32:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x561b962f9000 0x561b962fb3f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b96291b60 0x561b962973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b96284340 0x561b96296e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb00a7fc640 0 client.16215 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb02be6b740 1 --2- 172.21.15.45:0/1686549821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b96283a20 0x561b96298af0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb02be6b740 1 -- 172.21.15.45:0/1686549821 >> 172.21.15.45:0/1686549821 conn(0x561b9627e980 msgr2=0x561b9627d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb02be6b740 1 -- 172.21.15.45:0/1686549821 shutdown_connections 2024-01-31T05:32:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb02be6b740 1 -- 172.21.15.45:0/1686549821 wait complete. 2024-01-31T05:32:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb02be6b740 20 client.16215 trim_cache size 0 max 16384 2024-01-31T05:32:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:32:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:32:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:32:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:32:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb0290a7640 1 -- 172.21.15.45:0/100627431 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb0100a14f0 msgr2=0x7fb0100a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb0290a7640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb0100a14f0 0x7fb0100a39b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009fd0 tx=0x7fb01c011bc0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb0290a7640 1 -- 172.21.15.45:0/100627431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb024100800 msgr2=0x7fb0241977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb0290a7640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb024100800 0x7fb0241977f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00e9f0 tx=0x7fb00c00edf0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb0290a7640 1 -- 172.21.15.45:0/100627431 shutdown_connections 2024-01-31T05:32:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb0290a7640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb0100a14f0 0x7fb0100a39b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb0290a7640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb024100800 0x7fb0241977f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb0290a7640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb024104b90 0x7fb02419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb0290a7640 1 --2- 172.21.15.45:0/100627431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0240ffe40 0x7fb0241972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb0290a7640 1 -- 172.21.15.45:0/100627431 >> 172.21.15.45:0/100627431 conn(0x7fb024076a40 msgr2=0x7fb024068610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.434+0000 7fb0290a7640 1 -- 172.21.15.45:0/100627431 shutdown_connections 2024-01-31T05:32:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:44.435+0000 7fb0290a7640 1 -- 172.21.15.45:0/100627431 wait complete. 2024-01-31T05:32:44.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1072871 10.84 MB/sec execute 404 sec latency 82.235 ms 2024-01-31T05:32:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:44 smithi045 ceph-mon[31585]: pgmap v317: 97 pgs: 97 active+clean; 582 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 245 op/s 2024-01-31T05:32:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:44 smithi156 ceph-mon[35720]: pgmap v317: 97 pgs: 97 active+clean; 582 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 245 op/s 2024-01-31T05:32:45.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:44 smithi186 ceph-mon[35659]: pgmap v317: 97 pgs: 97 active+clean; 582 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 245 op/s 2024-01-31T05:32:45.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1074390 10.83 MB/sec execute 405 sec latency 21.648 ms 2024-01-31T05:32:46.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1076537 10.84 MB/sec execute 406 sec latency 20.264 ms 2024-01-31T05:32:46.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:46 smithi045 ceph-mon[31585]: pgmap v318: 97 pgs: 97 active+clean; 582 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 163 op/s 2024-01-31T05:32:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:46 smithi156 ceph-mon[35720]: pgmap v318: 97 pgs: 97 active+clean; 582 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 163 op/s 2024-01-31T05:32:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:46 smithi186 ceph-mon[35659]: pgmap v318: 97 pgs: 97 active+clean; 582 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 163 op/s 2024-01-31T05:32:47.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1078306 10.82 MB/sec execute 407 sec latency 23.927 ms 2024-01-31T05:32:48.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1080103 10.83 MB/sec execute 408 sec latency 91.489 ms 2024-01-31T05:32:48.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:48 smithi045 ceph-mon[31585]: pgmap v319: 97 pgs: 97 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.7 MiB/s wr, 249 op/s 2024-01-31T05:32:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:48 smithi156 ceph-mon[35720]: pgmap v319: 97 pgs: 97 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.7 MiB/s wr, 249 op/s 2024-01-31T05:32:49.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:48 smithi186 ceph-mon[35659]: pgmap v319: 97 pgs: 97 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.7 MiB/s wr, 249 op/s 2024-01-31T05:32:49.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1081530 10.82 MB/sec execute 409 sec latency 76.996 ms 2024-01-31T05:32:50.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1083556 10.82 MB/sec execute 410 sec latency 78.979 ms 2024-01-31T05:32:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:50 smithi045 ceph-mon[31585]: pgmap v320: 97 pgs: 97 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 167 op/s 2024-01-31T05:32:50.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:50 smithi156 ceph-mon[35720]: pgmap v320: 97 pgs: 97 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 167 op/s 2024-01-31T05:32:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:50 smithi186 ceph-mon[35659]: pgmap v320: 97 pgs: 97 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 167 op/s 2024-01-31T05:32:51.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1085197 10.81 MB/sec execute 411 sec latency 22.493 ms 2024-01-31T05:32:52.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1087173 10.82 MB/sec execute 412 sec latency 15.688 ms 2024-01-31T05:32:52.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:52 smithi045 ceph-mon[31585]: pgmap v321: 97 pgs: 97 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 167 op/s 2024-01-31T05:32:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:52 smithi156 ceph-mon[35720]: pgmap v321: 97 pgs: 97 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 167 op/s 2024-01-31T05:32:53.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:52 smithi186 ceph-mon[35659]: pgmap v321: 97 pgs: 97 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 167 op/s 2024-01-31T05:32:53.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1088491 10.81 MB/sec execute 413 sec latency 55.424 ms 2024-01-31T05:32:54.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1090618 10.81 MB/sec execute 414 sec latency 16.630 ms 2024-01-31T05:32:54.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:54 smithi045 ceph-mon[31585]: pgmap v322: 97 pgs: 97 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 248 op/s 2024-01-31T05:32:54.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:54 smithi156 ceph-mon[35720]: pgmap v322: 97 pgs: 97 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 248 op/s 2024-01-31T05:32:55.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:54 smithi186 ceph-mon[35659]: pgmap v322: 97 pgs: 97 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 248 op/s 2024-01-31T05:32:55.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1091935 10.80 MB/sec execute 415 sec latency 64.164 ms 2024-01-31T05:32:55.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:55 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:32:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:55 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:32:56.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:55 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:32:56.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1094063 10.80 MB/sec execute 416 sec latency 16.551 ms 2024-01-31T05:32:56.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:56 smithi045 ceph-mon[31585]: pgmap v323: 97 pgs: 97 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 167 op/s 2024-01-31T05:32:56.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:56 smithi156 ceph-mon[35720]: pgmap v323: 97 pgs: 97 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 167 op/s 2024-01-31T05:32:57.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:56 smithi186 ceph-mon[35659]: pgmap v323: 97 pgs: 97 active+clean; 601 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 167 op/s 2024-01-31T05:32:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.523+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3988618515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee4104b90 msgr2=0x7f7ee4104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.523+0000 7f7eeac6e640 1 --2- 172.21.15.45:0/3988618515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee4104b90 0x7f7ee4104f70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009900 tx=0x7f7edc02bc30 comp rx=0 tx=0).stop 2024-01-31T05:32:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.523+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3988618515 shutdown_connections 2024-01-31T05:32:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.523+0000 7f7eeac6e640 1 --2- 172.21.15.45:0/3988618515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee4100800 0x7f7ee410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.523+0000 7f7eeac6e640 1 --2- 172.21.15.45:0/3988618515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ee40ffe40 0x7f7ee41002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.523+0000 7f7eeac6e640 1 --2- 172.21.15.45:0/3988618515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee4104b90 0x7f7ee4104f70 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.523+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3988618515 >> 172.21.15.45:0/3988618515 conn(0x7f7ee4076a40 msgr2=0x7f7ee4076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.523+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3988618515 shutdown_connections 2024-01-31T05:32:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.523+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3988618515 wait complete. 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7eeac6e640 1 Processor -- start 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7eeac6e640 1 -- start start 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7eeac6e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee40ffe40 0x7f7ee4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7eeac6e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ee4100800 0x7f7ee4197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7eeac6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee4104b90 0x7f7ee419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7eeac6e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee419c0c0 con 0x7f7ee4104b90 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7eeac6e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee419c230 con 0x7f7ee4100800 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7eeac6e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee419c3f0 con 0x7f7ee40ffe40 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7ee920b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee4104b90 0x7f7ee419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7ee8a0a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee40ffe40 0x7f7ee4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7ee920b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee4104b90 0x7f7ee419bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37856/0 (socket says 172.21.15.45:37856) 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7ee920b640 1 -- 172.21.15.45:0/3877432348 learned_addr learned my addr 172.21.15.45:0/3877432348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7ee3fff640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ee4100800 0x7f7ee4197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7ee8a0a640 1 -- 172.21.15.45:0/3877432348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ee4100800 msgr2=0x7f7ee4197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ee4100800 0x7f7ee4197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7ee8a0a640 1 -- 172.21.15.45:0/3877432348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee4104b90 msgr2=0x7f7ee419bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee4104b90 0x7f7ee419bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7ee8a0a640 1 -- 172.21.15.45:0/3877432348 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc009560 con 0x7f7ee40ffe40 2024-01-31T05:32:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.524+0000 7f7ee920b640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee4104b90 0x7f7ee419bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.525+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee40ffe40 0x7f7ee4197200 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7edc036040 tx=0x7f7edc02c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.525+0000 7f7ee1ffb640 1 -- 172.21.15.45:0/3877432348 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc004250 con 0x7f7ee40ffe40 2024-01-31T05:32:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.525+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3877432348 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee419c670 con 0x7f7ee40ffe40 2024-01-31T05:32:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.525+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3877432348 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee41a3f50 con 0x7f7ee40ffe40 2024-01-31T05:32:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.525+0000 7f7ee1ffb640 1 -- 172.21.15.45:0/3877432348 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7edc0043b0 con 0x7f7ee40ffe40 2024-01-31T05:32:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.525+0000 7f7ee1ffb640 1 -- 172.21.15.45:0/3877432348 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc003ba0 con 0x7f7ee40ffe40 2024-01-31T05:32:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.527+0000 7f7ee1ffb640 1 -- 172.21.15.45:0/3877432348 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f7edc003d00 con 0x7f7ee40ffe40 2024-01-31T05:32:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7eeda30740 1 Processor -- start 2024-01-31T05:32:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7eeda30740 1 -- start start 2024-01-31T05:32:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee1ffb640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7ed00a14f0 0x7f7ed00a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7eeda30740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c59b00 0x55c465c59ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7eeda30740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c5a420 0x55c465c67620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee920b640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7ed00a14f0 0x7f7ed00a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee1ffb640 1 -- 172.21.15.45:0/3877432348 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7edc02de40 con 0x7f7ee40ffe40 2024-01-31T05:32:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7eeda30740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c67b60 0x55c465c69f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7eeda30740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c465bc9040 con 0x55c465c5a420 2024-01-31T05:32:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7eeda30740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c465bc6730 con 0x55c465c59b00 2024-01-31T05:32:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7eeda30740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c465bc2020 con 0x55c465c67b60 2024-01-31T05:32:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee8a0a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c59b00 0x55c465c59ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee8a0a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c59b00 0x55c465c59ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:46392/0 (socket says 172.21.15.45:46392) 2024-01-31T05:32:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee8a0a640 1 -- 172.21.15.45:0/2850587779 learned_addr learned my addr 172.21.15.45:0/2850587779 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee3fff640 1 --2- 172.21.15.45:0/2850587779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c5a420 0x55c465c67620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee920b640 1 --2- 172.21.15.45:0/2850587779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c67b60 0x55c465c69f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee8a0a640 1 -- 172.21.15.45:0/2850587779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c67b60 msgr2=0x55c465c69f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/2850587779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c67b60 0x55c465c69f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee8a0a640 1 -- 172.21.15.45:0/2850587779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c5a420 msgr2=0x55c465c67620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/2850587779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c5a420 0x55c465c67620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.528+0000 7f7ee8a0a640 1 -- 172.21.15.45:0/2850587779 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c465bb9e40 con 0x55c465c59b00 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.529+0000 7f7ee920b640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7ed00a14f0 0x7f7ed00a39b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009760 tx=0x7f7ecc013cd0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.529+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/2850587779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c59b00 0x55c465c59ee0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0c3c60 tx=0x7f7edc0b8300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adec7512c6450e70 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.529+0000 7f7edaffd640 1 -- 172.21.15.45:0/2850587779 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc0bc590 con 0x55c465c59b00 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.529+0000 7f7edaffd640 1 -- 172.21.15.45:0/2850587779 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7edc0bbe90 con 0x55c465c59b00 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.529+0000 7f7edaffd640 1 -- 172.21.15.45:0/2850587779 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc0eca90 con 0x55c465c59b00 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.529+0000 7f7eeda30740 1 -- 172.21.15.45:0/2850587779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c59b00 msgr2=0x55c465c59ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.529+0000 7f7eeda30740 1 --2- 172.21.15.45:0/2850587779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c59b00 0x55c465c59ee0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0c3c60 tx=0x7f7edc0b8300 comp rx=0 tx=0).stop 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 -- 172.21.15.45:0/2850587779 shutdown_connections 2024-01-31T05:32:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 --2- 172.21.15.45:0/2850587779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c5a420 0x55c465c67620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 --2- 172.21.15.45:0/2850587779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c67b60 0x55c465c69f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 --2- 172.21.15.45:0/2850587779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c59b00 0x55c465c59ee0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 -- 172.21.15.45:0/2850587779 >> 172.21.15.45:0/2850587779 conn(0x55c465c54980 msgr2=0x55c465c54d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 -- 172.21.15.45:0/2850587779 shutdown_connections 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 -- 172.21.15.45:0/2850587779 wait complete. 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 -- start start 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 Processor -- start 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c59b00 0x55c465c67620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c5a420 0x55c465c649b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c67b60 0x55c465c650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c465b7b1b0 con 0x55c465c67b60 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c465b7ae50 con 0x55c465c5a420 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c465b40090 con 0x55c465c59b00 2024-01-31T05:32:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.530+0000 7f7eeda30740 10 client.0 _ll_register_callbacks cb 0x55c465b65a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:32:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ee920b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c67b60 0x55c465c650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ee8a0a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c59b00 0x55c465c67620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ee920b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c67b60 0x55c465c650f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37876/0 (socket says 172.21.15.45:37876) 2024-01-31T05:32:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ee920b640 1 -- 172.21.15.45:0/1996522440 learned_addr learned my addr 172.21.15.45:0/1996522440 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ee3fff640 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c5a420 0x55c465c649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ee920b640 1 -- 172.21.15.45:0/1996522440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c59b00 msgr2=0x55c465c67620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ee920b640 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c59b00 0x55c465c67620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ee920b640 1 -- 172.21.15.45:0/1996522440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c5a420 msgr2=0x55c465c649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ee920b640 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c5a420 0x55c465c649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ee920b640 1 -- 172.21.15.45:0/1996522440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc0db040 con 0x55c465c67b60 2024-01-31T05:32:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ee3fff640 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c5a420 0x55c465c649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ee920b640 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c67b60 0x55c465c650f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00fca0 tx=0x7f7ecc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e478d33fe03134 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ec3fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:32:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ec3fff640 1 -- 172.21.15.45:0/1996522440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc018040 con 0x55c465c67b60 2024-01-31T05:32:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ec3fff640 1 -- 172.21.15.45:0/1996522440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7ecc010e40 con 0x55c465c67b60 2024-01-31T05:32:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7eeda30740 1 -- 172.21.15.45:0/1996522440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c465bbb010 con 0x55c465c67b60 2024-01-31T05:32:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7ec3fff640 1 -- 172.21.15.45:0/1996522440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ecc020630 con 0x55c465c67b60 2024-01-31T05:32:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7eeda30740 10 client.16227 fetch_fsmap learned FSMap version 29 2024-01-31T05:32:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.531+0000 7f7eeda30740 1 -- 172.21.15.45:0/1996522440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c465bc06a0 con 0x55c465c67b60 2024-01-31T05:32:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.532+0000 7f7ec3fff640 1 -- 172.21.15.45:0/1996522440 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c465bbb010 con 0x55c465c67b60 2024-01-31T05:32:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.532+0000 7f7ec3fff640 1 -- 172.21.15.45:0/1996522440 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7ecc02a880 con 0x55c465c67b60 2024-01-31T05:32:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.532+0000 7f7eeda30740 10 client.16227 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:32:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.532+0000 7f7eeda30740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.532+0000 7f7eeda30740 20 client.16227 populate_metadata read hostname 'smithi045' 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.532+0000 7f7eeda30740 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c465c77690 0x55c465c79ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.532+0000 7f7eeda30740 4 client.16227 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.532+0000 7f7eeda30740 1 -- 172.21.15.45:0/1996522440 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c465bbad30 con 0x55c465c77690 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.532+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c465c77690 0x55c465c79ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.532+0000 7f7ec3fff640 10 client.16227 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.532+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c465c77690 0x55c465c79ab0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=970cac8825f61f71 server_cookie=a5c34a445976fae0 in_seq=0 out_seq=0 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.533+0000 7f7ec3fff640 1 -- 172.21.15.45:0/1996522440 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c465bbad30 con 0x55c465c77690 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.533+0000 7f7ec3fff640 10 client.16227 handle_command_reply: tid=0 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.533+0000 7f7eeda30740 1 client.16227 shutdown 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.533+0000 7f7eeda30740 1 -- 172.21.15.45:0/1996522440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c67b60 msgr2=0x55c465c650f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.533+0000 7f7eeda30740 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c67b60 0x55c465c650f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00fca0 tx=0x7f7ecc004730 comp rx=0 tx=0).stop 2024-01-31T05:32:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.533+0000 7f7eeda30740 1 -- 172.21.15.45:0/1996522440 shutdown_connections 2024-01-31T05:32:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.533+0000 7f7eeda30740 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c465c77690 0x55c465c79ab0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.533+0000 7f7eeda30740 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c67b60 0x55c465c650f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.533+0000 7f7eeda30740 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c5a420 0x55c465c649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.533+0000 7f7ec3fff640 0 client.16227 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.533+0000 7f7eeda30740 1 --2- 172.21.15.45:0/1996522440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c59b00 0x55c465c67620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.533+0000 7f7eeda30740 1 -- 172.21.15.45:0/1996522440 >> 172.21.15.45:0/1996522440 conn(0x55c465c54980 msgr2=0x55c465c566d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.534+0000 7f7eeda30740 1 -- 172.21.15.45:0/1996522440 shutdown_connections 2024-01-31T05:32:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.534+0000 7f7eeda30740 1 -- 172.21.15.45:0/1996522440 wait complete. 2024-01-31T05:32:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.534+0000 7f7eeda30740 20 client.16227 trim_cache size 0 max 16384 2024-01-31T05:32:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7eeda30740 1 Processor -- start 2024-01-31T05:32:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7eeda30740 1 -- start start 2024-01-31T05:32:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7eeda30740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c59b00 0x55c465c670e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7eeda30740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c5a420 0x55c465c67620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7eeda30740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c67b60 0x55c465c6e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7eeda30740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c465c603d0 con 0x55c465c59b00 2024-01-31T05:32:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7eeda30740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c465bbad30 con 0x55c465c5a420 2024-01-31T05:32:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7eeda30740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c465bbb010 con 0x55c465c67b60 2024-01-31T05:32:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7ee8a0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c59b00 0x55c465c670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7ee8a0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c59b00 0x55c465c670e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37892/0 (socket says 172.21.15.45:37892) 2024-01-31T05:32:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7ee8a0a640 1 -- 172.21.15.45:0/2125494870 learned_addr learned my addr 172.21.15.45:0/2125494870 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7ee3fff640 1 --2- 172.21.15.45:0/2125494870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c5a420 0x55c465c67620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.547+0000 7f7ee920b640 1 --2- 172.21.15.45:0/2125494870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c67b60 0x55c465c6e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.548+0000 7f7ee8a0a640 1 -- 172.21.15.45:0/2125494870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c67b60 msgr2=0x55c465c6e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.548+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/2125494870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c67b60 0x55c465c6e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.548+0000 7f7ee8a0a640 1 -- 172.21.15.45:0/2125494870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c5a420 msgr2=0x55c465c67620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.548+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/2125494870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c5a420 0x55c465c67620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.548+0000 7f7ee8a0a640 1 -- 172.21.15.45:0/2125494870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c465b5d030 con 0x55c465c59b00 2024-01-31T05:32:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.548+0000 7f7ee3fff640 1 --2- 172.21.15.45:0/2125494870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c5a420 0x55c465c67620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.548+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/2125494870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c59b00 0x55c465c670e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0b8390 tx=0x7f7edc0b81d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930e1e42bb942fc server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.548+0000 7f7eda7fc640 1 -- 172.21.15.45:0/2125494870 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc0bffe0 con 0x55c465c59b00 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.548+0000 7f7eda7fc640 1 -- 172.21.15.45:0/2125494870 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7edc0c4190 con 0x55c465c59b00 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.548+0000 7f7eda7fc640 1 -- 172.21.15.45:0/2125494870 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc0bb5e0 con 0x55c465c59b00 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 -- 172.21.15.45:0/2125494870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c59b00 msgr2=0x55c465c670e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 --2- 172.21.15.45:0/2125494870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c59b00 0x55c465c670e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0b8390 tx=0x7f7edc0b81d0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 -- 172.21.15.45:0/2125494870 shutdown_connections 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 --2- 172.21.15.45:0/2125494870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c67b60 0x55c465c6e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 --2- 172.21.15.45:0/2125494870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c5a420 0x55c465c67620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 --2- 172.21.15.45:0/2125494870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c59b00 0x55c465c670e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 -- 172.21.15.45:0/2125494870 >> 172.21.15.45:0/2125494870 conn(0x55c465c54980 msgr2=0x55c465c730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 -- 172.21.15.45:0/2125494870 shutdown_connections 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 -- 172.21.15.45:0/2125494870 wait complete. 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 -- start start 2024-01-31T05:32:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 Processor -- start 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c59b00 0x55c465c6c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.549+0000 7f7eeda30740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c5a420 0x55c465c6ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7eeda30740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c67b60 0x55c465cccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7eeda30740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c465c6d420 con 0x55c465c5a420 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7eeda30740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c465ccd0d0 con 0x55c465c67b60 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7eeda30740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c465ccd290 con 0x55c465c59b00 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7eeda30740 10 client.0 _ll_register_callbacks cb 0x7f7edc0035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ee3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c5a420 0x55c465c6ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ee8a0a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c59b00 0x55c465c6c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ee3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c5a420 0x55c465c6ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37908/0 (socket says 172.21.15.45:37908) 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ee3fff640 1 -- 172.21.15.45:0/1629041841 learned_addr learned my addr 172.21.15.45:0/1629041841 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ee920b640 1 --2- 172.21.15.45:0/1629041841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c67b60 0x55c465cccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ee3fff640 1 -- 172.21.15.45:0/1629041841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c59b00 msgr2=0x55c465c6c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ee3fff640 1 --2- 172.21.15.45:0/1629041841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c59b00 0x55c465c6c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ee3fff640 1 -- 172.21.15.45:0/1629041841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c67b60 msgr2=0x55c465cccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ee3fff640 1 --2- 172.21.15.45:0/1629041841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c67b60 0x55c465cccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ee3fff640 1 -- 172.21.15.45:0/1629041841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed4011750 con 0x55c465c5a420 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ee3fff640 1 --2- 172.21.15.45:0/1629041841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c5a420 0x55c465c6ccb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4104450 tx=0x7f7ed400faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d4b21daea3f48b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ec3fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ec3fff640 1 -- 172.21.15.45:0/1629041841 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4016040 con 0x55c465c5a420 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7ec3fff640 1 -- 172.21.15.45:0/1629041841 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7ed4004b20 con 0x55c465c5a420 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.550+0000 7f7eeda30740 1 -- 172.21.15.45:0/1629041841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7edc0bbe90 con 0x55c465c5a420 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.551+0000 7f7ec3fff640 1 -- 172.21.15.45:0/1629041841 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ed4004ce0 con 0x55c465c5a420 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.551+0000 7f7eeda30740 10 client.16233 fetch_fsmap learned FSMap version 29 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.551+0000 7f7eeda30740 1 -- 172.21.15.45:0/1629041841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c465bb9e40 con 0x55c465c5a420 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.551+0000 7f7ec3fff640 1 -- 172.21.15.45:0/1629041841 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7edc0bbe90 con 0x55c465c5a420 2024-01-31T05:32:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.551+0000 7f7ec3fff640 1 -- 172.21.15.45:0/1629041841 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7ed401da90 con 0x55c465c5a420 2024-01-31T05:32:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.551+0000 7f7eeda30740 10 client.16233 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:32:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.551+0000 7f7eeda30740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:32:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.551+0000 7f7eeda30740 20 client.16233 populate_metadata read hostname 'smithi045' 2024-01-31T05:32:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.551+0000 7f7eeda30740 1 --2- 172.21.15.45:0/1629041841 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c465ccec30 0x55c465cd1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:32:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.551+0000 7f7eeda30740 4 client.16233 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:32:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.551+0000 7f7eeda30740 1 -- 172.21.15.45:0/1629041841 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c4658c1600 con 0x55c465ccec30 2024-01-31T05:32:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.551+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/1629041841 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c465ccec30 0x55c465cd1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:32:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.552+0000 7f7ec3fff640 10 client.16233 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.552+0000 7f7ee8a0a640 1 --2- 172.21.15.45:0/1629041841 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c465ccec30 0x55c465cd1050 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59eb3eeb5f341568 server_cookie=5b2ef6d5e3e43ffd in_seq=0 out_seq=0 2024-01-31T05:32:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7ec3fff640 1 -- 172.21.15.45:0/1629041841 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c4658c1600 con 0x55c465ccec30 2024-01-31T05:32:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7ec3fff640 10 client.16233 handle_command_reply: tid=0 2024-01-31T05:32:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7eeda30740 1 client.16233 shutdown 2024-01-31T05:32:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7eeda30740 1 -- 172.21.15.45:0/1629041841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c5a420 msgr2=0x55c465c6ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7eeda30740 1 --2- 172.21.15.45:0/1629041841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c5a420 0x55c465c6ccb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4104450 tx=0x7f7ed400faf0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7eeda30740 1 -- 172.21.15.45:0/1629041841 shutdown_connections 2024-01-31T05:32:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7eeda30740 1 --2- 172.21.15.45:0/1629041841 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55c465ccec30 0x55c465cd1050 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7eeda30740 1 --2- 172.21.15.45:0/1629041841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c465c5a420 0x55c465c6ccb0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7eeda30740 1 --2- 172.21.15.45:0/1629041841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c465c67b60 0x55c465cccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7ec3fff640 0 client.16233 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:32:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7eeda30740 1 --2- 172.21.15.45:0/1629041841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c465c59b00 0x55c465c6c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7eeda30740 1 -- 172.21.15.45:0/1629041841 >> 172.21.15.45:0/1629041841 conn(0x55c465c54980 msgr2=0x55c465c64240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.564+0000 7f7eeda30740 1 -- 172.21.15.45:0/1629041841 shutdown_connections 2024-01-31T05:32:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeda30740 1 -- 172.21.15.45:0/1629041841 wait complete. 2024-01-31T05:32:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeda30740 20 client.16233 trim_cache size 0 max 16384 2024-01-31T05:32:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:32:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:32:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:32:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:32:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3877432348 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7ed00a14f0 msgr2=0x7f7ed00a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeac6e640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7ed00a14f0 0x7f7ed00a39b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009760 tx=0x7f7ecc013cd0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3877432348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee40ffe40 msgr2=0x7f7ee4197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:32:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeac6e640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee40ffe40 0x7f7ee4197200 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7edc036040 tx=0x7f7edc02c680 comp rx=0 tx=0).stop 2024-01-31T05:32:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3877432348 shutdown_connections 2024-01-31T05:32:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeac6e640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7ed00a14f0 0x7f7ed00a39b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeac6e640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee4104b90 0x7f7ee419bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeac6e640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ee4100800 0x7f7ee4197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeac6e640 1 --2- 172.21.15.45:0/3877432348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ee40ffe40 0x7f7ee4197200 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:32:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3877432348 >> 172.21.15.45:0/3877432348 conn(0x7f7ee4076a40 msgr2=0x7f7ee4069710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:32:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3877432348 shutdown_connections 2024-01-31T05:32:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:32:57.565+0000 7f7eeac6e640 1 -- 172.21.15.45:0/3877432348 wait complete. 2024-01-31T05:32:57.796 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1095208 10.79 MB/sec execute 417 sec latency 24.261 ms 2024-01-31T05:32:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: Unable to set osd_memory_target on smithi045 to 185902796: error parsing value: Value '185902796' is below minimum 939524096 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: pgmap v324: 97 pgs: 97 active+clean; 610 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 246 op/s 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:57 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-01-31T05:32:58.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: Unable to set osd_memory_target on smithi045 to 185902796: error parsing value: Value '185902796' is below minimum 939524096 2024-01-31T05:32:58.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: pgmap v324: 97 pgs: 97 active+clean; 610 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 246 op/s 2024-01-31T05:32:58.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:57 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-01-31T05:32:58.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: Unable to set osd_memory_target on smithi045 to 185902796: error parsing value: Value '185902796' is below minimum 939524096 2024-01-31T05:32:58.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: pgmap v324: 97 pgs: 97 active+clean; 610 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 246 op/s 2024-01-31T05:32:58.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:57 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:58.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1097402 10.79 MB/sec execute 418 sec latency 58.744 ms 2024-01-31T05:32:59.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:58 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-01-31T05:32:59.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:58 smithi186 ceph-mon[35659]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-01-31T05:32:59.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:58 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:59.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:58 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:59.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:58 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:59.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:58 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:59.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:58 smithi186 ceph-mon[35659]: Adjusting osd_memory_target on smithi186 to 174.1M 2024-01-31T05:32:59.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:58 smithi186 ceph-mon[35659]: Unable to set osd_memory_target on smithi186 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-01-31T05:32:59.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:58 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:32:59.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:58 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:32:59.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:58 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:59.128 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:58 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-01-31T05:32:59.128 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:58 smithi156 ceph-mon[35720]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-01-31T05:32:59.128 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:58 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:59.128 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:58 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:59.128 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:58 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:59.129 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:58 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:59.129 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:58 smithi156 ceph-mon[35720]: Adjusting osd_memory_target on smithi186 to 174.1M 2024-01-31T05:32:59.129 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:58 smithi156 ceph-mon[35720]: Unable to set osd_memory_target on smithi186 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-01-31T05:32:59.129 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:58 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:32:59.129 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:58 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:32:59.129 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:58 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:58 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-01-31T05:32:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:58 smithi045 ceph-mon[31585]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-01-31T05:32:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:58 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:58 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:58 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:58 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-31T05:32:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:58 smithi045 ceph-mon[31585]: Adjusting osd_memory_target on smithi186 to 174.1M 2024-01-31T05:32:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:58 smithi045 ceph-mon[31585]: Unable to set osd_memory_target on smithi186 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-01-31T05:32:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:58 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:32:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:58 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:32:59.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:58 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:32:59.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1098392 10.78 MB/sec execute 419 sec latency 35.524 ms 2024-01-31T05:33:00.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:32:59 smithi186 ceph-mon[35659]: pgmap v325: 97 pgs: 97 active+clean; 610 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-01-31T05:33:00.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:32:59 smithi045 ceph-mon[31585]: pgmap v325: 97 pgs: 97 active+clean; 610 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-01-31T05:33:00.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:32:59 smithi156 ceph-mon[35720]: pgmap v325: 97 pgs: 97 active+clean; 610 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-01-31T05:33:00.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1100939 10.78 MB/sec execute 420 sec latency 59.098 ms 2024-01-31T05:33:01.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1101514 10.77 MB/sec execute 421 sec latency 70.320 ms 2024-01-31T05:33:01.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:01 smithi156 ceph-mon[35720]: pgmap v326: 97 pgs: 97 active+clean; 610 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-01-31T05:33:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:01 smithi186 ceph-mon[35659]: pgmap v326: 97 pgs: 97 active+clean; 610 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-01-31T05:33:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:01 smithi045 ceph-mon[31585]: pgmap v326: 97 pgs: 97 active+clean; 610 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-01-31T05:33:02.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1103510 10.76 MB/sec execute 422 sec latency 23.871 ms 2024-01-31T05:33:03.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1104953 10.76 MB/sec execute 423 sec latency 85.583 ms 2024-01-31T05:33:04.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1106672 10.75 MB/sec execute 424 sec latency 22.545 ms 2024-01-31T05:33:04.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:04 smithi045 ceph-mon[31585]: pgmap v327: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 277 op/s 2024-01-31T05:33:04.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:04 smithi156 ceph-mon[35720]: pgmap v327: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 277 op/s 2024-01-31T05:33:05.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:04 smithi186 ceph-mon[35659]: pgmap v327: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 277 op/s 2024-01-31T05:33:05.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1108503 10.76 MB/sec execute 425 sec latency 13.315 ms 2024-01-31T05:33:06.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1109875 10.75 MB/sec execute 426 sec latency 181.303 ms 2024-01-31T05:33:06.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:06 smithi045 ceph-mon[31585]: pgmap v328: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 196 op/s 2024-01-31T05:33:06.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:06 smithi156 ceph-mon[35720]: pgmap v328: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 196 op/s 2024-01-31T05:33:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:06 smithi186 ceph-mon[35659]: pgmap v328: 97 pgs: 97 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 196 op/s 2024-01-31T05:33:07.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1111891 10.75 MB/sec execute 427 sec latency 185.623 ms 2024-01-31T05:33:08.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1113042 10.73 MB/sec execute 428 sec latency 64.193 ms 2024-01-31T05:33:08.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:08 smithi045 ceph-mon[31585]: pgmap v329: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 280 op/s 2024-01-31T05:33:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:08 smithi156 ceph-mon[35720]: pgmap v329: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 280 op/s 2024-01-31T05:33:09.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:08 smithi186 ceph-mon[35659]: pgmap v329: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 280 op/s 2024-01-31T05:33:09.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1115279 10.74 MB/sec execute 429 sec latency 99.713 ms 2024-01-31T05:33:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.676+0000 7fd38bbbb640 1 -- 172.21.15.45:0/3832002682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd384105ff0 msgr2=0x7fd38410dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.676+0000 7fd38bbbb640 1 --2- 172.21.15.45:0/3832002682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd384105ff0 0x7fd38410dbc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd378009860 tx=0x7fd37802bb70 comp rx=0 tx=0).stop 2024-01-31T05:33:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.676+0000 7fd38bbbb640 1 -- 172.21.15.45:0/3832002682 shutdown_connections 2024-01-31T05:33:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.676+0000 7fd38bbbb640 1 --2- 172.21.15.45:0/3832002682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3841018d0 0x7fd384105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.676+0000 7fd38bbbb640 1 --2- 172.21.15.45:0/3832002682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd384105ff0 0x7fd38410dbc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.676+0000 7fd38bbbb640 1 --2- 172.21.15.45:0/3832002682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd384100f20 0x7fd384101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.676+0000 7fd38bbbb640 1 -- 172.21.15.45:0/3832002682 >> 172.21.15.45:0/3832002682 conn(0x7fd3840fc7c0 msgr2=0x7fd3840febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd38bbbb640 1 -- 172.21.15.45:0/3832002682 shutdown_connections 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd38bbbb640 1 -- 172.21.15.45:0/3832002682 wait complete. 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd38bbbb640 1 Processor -- start 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd38bbbb640 1 -- start start 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd38bbbb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd384100f20 0x7fd384197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd38bbbb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd384105ff0 0x7fd3841976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd38bbbb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd38419ba30 0x7fd38419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd38bbbb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38419c420 con 0x7fd384100f20 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd38bbbb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38419c590 con 0x7fd384105ff0 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd38bbbb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38419c700 con 0x7fd38419ba30 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd389957640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd384100f20 0x7fd384197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd389957640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd384100f20 0x7fd384197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49554/0 (socket says 172.21.15.45:49554) 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd389957640 1 -- 172.21.15.45:0/1302163108 learned_addr learned my addr 172.21.15.45:0/1302163108 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd38a158640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd38419ba30 0x7fd38419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.677+0000 7fd389156640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd384105ff0 0x7fd3841976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.678+0000 7fd389957640 1 -- 172.21.15.45:0/1302163108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd38419ba30 msgr2=0x7fd38419bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.678+0000 7fd389957640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd38419ba30 0x7fd38419bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.678+0000 7fd389957640 1 -- 172.21.15.45:0/1302163108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd384105ff0 msgr2=0x7fd3841976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.678+0000 7fd389957640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd384105ff0 0x7fd3841976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.678+0000 7fd389957640 1 -- 172.21.15.45:0/1302163108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c009560 con 0x7fd384100f20 2024-01-31T05:33:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.678+0000 7fd389156640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd384105ff0 0x7fd3841976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:33:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.678+0000 7fd389957640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd384100f20 0x7fd384197170 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00e9c0 tx=0x7fd37c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.678+0000 7fd382ffd640 1 -- 172.21.15.45:0/1302163108 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c00cde0 con 0x7fd384100f20 2024-01-31T05:33:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.678+0000 7fd38bbbb640 1 -- 172.21.15.45:0/1302163108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd374009840 con 0x7fd384100f20 2024-01-31T05:33:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.678+0000 7fd382ffd640 1 -- 172.21.15.45:0/1302163108 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd37c0035f0 con 0x7fd384100f20 2024-01-31T05:33:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.678+0000 7fd382ffd640 1 -- 172.21.15.45:0/1302163108 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0044e0 con 0x7fd384100f20 2024-01-31T05:33:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.678+0000 7fd38bbbb640 1 -- 172.21.15.45:0/1302163108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd378009490 con 0x7fd384100f20 2024-01-31T05:33:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.681+0000 7fd38e97f740 1 Processor -- start 2024-01-31T05:33:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd38e97f740 1 -- start start 2024-01-31T05:33:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd382ffd640 1 -- 172.21.15.45:0/1302163108 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fd37c003760 con 0x7fd384100f20 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd38e97f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd7793f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd38e97f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd77a1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd38e97f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77a1b60 0x560dd77a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd38e97f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd7703040 con 0x560dd7793b80 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd38e97f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd7700730 con 0x560dd77a1b60 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd38e97f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd76fc020 con 0x560dd77944a0 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd389957640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd77a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd38a158640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd7793f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd389957640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd77a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:48510/0 (socket says 172.21.15.45:48510) 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd389957640 1 -- 172.21.15.45:0/829683479 learned_addr learned my addr 172.21.15.45:0/829683479 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd38a158640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd7793f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49568/0 (socket says 172.21.15.45:49568) 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd389156640 1 --2- 172.21.15.45:0/829683479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77a1b60 0x560dd77a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd389957640 1 -- 172.21.15.45:0/829683479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77a1b60 msgr2=0x560dd77a3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd389957640 1 --2- 172.21.15.45:0/829683479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77a1b60 0x560dd77a3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd389957640 1 -- 172.21.15.45:0/829683479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 msgr2=0x560dd7793f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd389957640 1 --2- 172.21.15.45:0/829683479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd7793f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd389957640 1 -- 172.21.15.45:0/829683479 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dd76f3e40 con 0x560dd77944a0 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.682+0000 7fd382ffd640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd36c0a1570 0x7fd36c0a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.683+0000 7fd38a158640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd36c0a1570 0x7fd36c0a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.683+0000 7fd382ffd640 1 -- 172.21.15.45:0/1302163108 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd37c014070 con 0x7fd384100f20 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.683+0000 7fd389957640 1 --2- 172.21.15.45:0/829683479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd77a1620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0caa00 tx=0x7fd37c0d56f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30830d9515d5e121 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.683+0000 7fd36bfff640 1 -- 172.21.15.45:0/829683479 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c099f10 con 0x560dd77944a0 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.683+0000 7fd36bfff640 1 -- 172.21.15.45:0/829683479 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd37c099c80 con 0x560dd77944a0 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.683+0000 7fd36bfff640 1 -- 172.21.15.45:0/829683479 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c09a840 con 0x560dd77944a0 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.684+0000 7fd38e97f740 1 -- 172.21.15.45:0/829683479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 msgr2=0x560dd77a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.684+0000 7fd38a158640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd36c0a1570 0x7fd36c0a3a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd378009830 tx=0x7fd37802cea0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.684+0000 7fd38e97f740 1 --2- 172.21.15.45:0/829683479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd77a1620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0caa00 tx=0x7fd37c0d56f0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.684+0000 7fd38e97f740 1 -- 172.21.15.45:0/829683479 shutdown_connections 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.684+0000 7fd38e97f740 1 --2- 172.21.15.45:0/829683479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77a1b60 0x560dd77a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.684+0000 7fd38e97f740 1 --2- 172.21.15.45:0/829683479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd77a1620 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.684+0000 7fd38e97f740 1 --2- 172.21.15.45:0/829683479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd7793f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.684+0000 7fd38e97f740 1 -- 172.21.15.45:0/829683479 >> 172.21.15.45:0/829683479 conn(0x560dd778e980 msgr2=0x560dd778ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.684+0000 7fd38e97f740 1 -- 172.21.15.45:0/829683479 shutdown_connections 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.684+0000 7fd38e97f740 1 -- 172.21.15.45:0/829683479 wait complete. 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.684+0000 7fd38e97f740 1 -- start start 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.684+0000 7fd38e97f740 1 Processor -- start 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd38e97f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd779ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd38e97f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd779efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd38e97f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77a1b60 0x560dd779f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd38e97f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd76b51b0 con 0x560dd7793b80 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd38e97f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd76b4e50 con 0x560dd77a1b60 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd38e97f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd767a090 con 0x560dd77944a0 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd38e97f740 10 client.0 _ll_register_callbacks cb 0x560dd769fa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd389957640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd779ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd389156640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd779efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd389957640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd779ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49584/0 (socket says 172.21.15.45:49584) 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd38a158640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77a1b60 0x560dd779f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd389957640 1 -- 172.21.15.45:0/1962022376 learned_addr learned my addr 172.21.15.45:0/1962022376 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd389957640 1 -- 172.21.15.45:0/1962022376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 msgr2=0x560dd779efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd389957640 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd779efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd389957640 1 -- 172.21.15.45:0/1962022376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77a1b60 msgr2=0x560dd779f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd389957640 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77a1b60 0x560dd779f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd389957640 1 -- 172.21.15.45:0/1962022376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd378009490 con 0x560dd7793b80 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd38a158640 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77a1b60 0x560dd779f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.685+0000 7fd389156640 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd779efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd389957640 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd779ea80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0a1fa0 tx=0x7fd37c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84179d3444476011 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd3697fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd3697fa640 1 -- 172.21.15.45:0/1962022376 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c099f10 con 0x560dd7793b80 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd3697fa640 1 -- 172.21.15.45:0/1962022376 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd37c09a840 con 0x560dd7793b80 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd38e97f740 1 -- 172.21.15.45:0/1962022376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560dd76f5010 con 0x560dd7793b80 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd3697fa640 1 -- 172.21.15.45:0/1962022376 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd37c0a24d0 con 0x560dd7793b80 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd38e97f740 10 client.16245 fetch_fsmap learned FSMap version 29 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd38e97f740 1 -- 172.21.15.45:0/1962022376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560dd76fa6a0 con 0x560dd7793b80 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd3697fa640 1 -- 172.21.15.45:0/1962022376 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560dd76f5010 con 0x560dd7793b80 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd3697fa640 1 -- 172.21.15.45:0/1962022376 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd37c01f060 con 0x560dd7793b80 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd38e97f740 10 client.16245 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:33:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd38e97f740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd38e97f740 20 client.16245 populate_metadata read hostname 'smithi045' 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd38e97f740 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560dd77b1480 0x560dd77b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd38e97f740 4 client.16245 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.686+0000 7fd38e97f740 1 -- 172.21.15.45:0/1962022376 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dd76f4d30 con 0x560dd77b1480 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.687+0000 7fd389156640 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560dd77b1480 0x560dd77b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.687+0000 7fd389156640 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560dd77b1480 0x560dd77b3870 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cbb6f6fb835ec18 server_cookie=91f66e7c4bc806c1 in_seq=0 out_seq=0 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.687+0000 7fd3697fa640 10 client.16245 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.687+0000 7fd3697fa640 1 -- 172.21.15.45:0/1962022376 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560dd76f4d30 con 0x560dd77b1480 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.687+0000 7fd3697fa640 10 client.16245 handle_command_reply: tid=0 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd38e97f740 1 client.16245 shutdown 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd38e97f740 1 -- 172.21.15.45:0/1962022376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 msgr2=0x560dd779ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd38e97f740 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd779ea80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0a1fa0 tx=0x7fd37c003d40 comp rx=0 tx=0).stop 2024-01-31T05:33:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd38e97f740 1 -- 172.21.15.45:0/1962022376 shutdown_connections 2024-01-31T05:33:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd38e97f740 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560dd77b1480 0x560dd77b3870 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd38e97f740 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77a1b60 0x560dd779f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd38e97f740 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd779efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd38e97f740 1 --2- 172.21.15.45:0/1962022376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd779ea80 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd38e97f740 1 -- 172.21.15.45:0/1962022376 >> 172.21.15.45:0/1962022376 conn(0x560dd778e980 msgr2=0x560dd77906d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd3697fa640 0 client.16245 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd38e97f740 1 -- 172.21.15.45:0/1962022376 shutdown_connections 2024-01-31T05:33:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd38e97f740 1 -- 172.21.15.45:0/1962022376 wait complete. 2024-01-31T05:33:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.688+0000 7fd38e97f740 20 client.16245 trim_cache size 0 max 16384 2024-01-31T05:33:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.702+0000 7fd38e97f740 1 Processor -- start 2024-01-31T05:33:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd38e97f740 1 -- start start 2024-01-31T05:33:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd38e97f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd77a8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd38e97f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77944a0 0x560dd77a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd38e97f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77a1b60 0x560dd7799d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd38e97f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd778a0b0 con 0x560dd7793b80 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd38e97f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd76f5040 con 0x560dd77944a0 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd38e97f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd7795010 con 0x560dd77a1b60 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd389957640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd77a8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd389957640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd77a8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49592/0 (socket says 172.21.15.45:49592) 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd389957640 1 -- 172.21.15.45:0/929291875 learned_addr learned my addr 172.21.15.45:0/929291875 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd38a158640 1 --2- 172.21.15.45:0/929291875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77a1b60 0x560dd7799d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd389156640 1 --2- 172.21.15.45:0/929291875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77944a0 0x560dd77a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd389957640 1 -- 172.21.15.45:0/929291875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77a1b60 msgr2=0x560dd7799d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd389957640 1 --2- 172.21.15.45:0/929291875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77a1b60 0x560dd7799d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd389957640 1 -- 172.21.15.45:0/929291875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77944a0 msgr2=0x560dd77a8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd389957640 1 --2- 172.21.15.45:0/929291875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77944a0 0x560dd77a8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd389957640 1 -- 172.21.15.45:0/929291875 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dd7697030 con 0x560dd7793b80 2024-01-31T05:33:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd389156640 1 --2- 172.21.15.45:0/929291875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77944a0 0x560dd77a8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:33:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.703+0000 7fd389957640 1 --2- 172.21.15.45:0/929291875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd77a8820 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd37c096bb0 tx=0x7fd37c003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8d539f32a6408d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.704+0000 7fd36b7fe640 1 -- 172.21.15.45:0/929291875 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c099aa0 con 0x560dd7793b80 2024-01-31T05:33:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.704+0000 7fd36b7fe640 1 -- 172.21.15.45:0/929291875 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd37c0d5bf0 con 0x560dd7793b80 2024-01-31T05:33:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.704+0000 7fd36b7fe640 1 -- 172.21.15.45:0/929291875 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c09e290 con 0x560dd7793b80 2024-01-31T05:33:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.704+0000 7fd38e97f740 1 -- 172.21.15.45:0/929291875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 msgr2=0x560dd77a8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.704+0000 7fd38e97f740 1 --2- 172.21.15.45:0/929291875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd77a8820 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd37c096bb0 tx=0x7fd37c003db0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.704+0000 7fd38e97f740 1 -- 172.21.15.45:0/929291875 shutdown_connections 2024-01-31T05:33:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.704+0000 7fd38e97f740 1 --2- 172.21.15.45:0/929291875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77a1b60 0x560dd7799d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.704+0000 7fd38e97f740 1 --2- 172.21.15.45:0/929291875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd77944a0 0x560dd77a8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.704+0000 7fd38e97f740 1 --2- 172.21.15.45:0/929291875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd7793b80 0x560dd77a8820 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.704+0000 7fd38e97f740 1 -- 172.21.15.45:0/929291875 >> 172.21.15.45:0/929291875 conn(0x560dd778e980 msgr2=0x560dd779f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.704+0000 7fd38e97f740 1 -- 172.21.15.45:0/929291875 shutdown_connections 2024-01-31T05:33:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.704+0000 7fd38e97f740 1 -- 172.21.15.45:0/929291875 wait complete. 2024-01-31T05:33:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd38e97f740 1 -- start start 2024-01-31T05:33:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd38e97f740 1 Processor -- start 2024-01-31T05:33:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd38e97f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd7793b80 0x560dd77fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd38e97f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd77fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd38e97f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd77a1b60 0x560dd77fc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd38e97f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd78069d0 con 0x560dd77a1b60 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd38e97f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd7806b40 con 0x560dd7793b80 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd38e97f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd7806d00 con 0x560dd77944a0 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd38e97f740 10 client.0 _ll_register_callbacks cb 0x7fd37c0d0b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd38a158640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd77a1b60 0x560dd77fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd38a158640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd77a1b60 0x560dd77fc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49600/0 (socket says 172.21.15.45:49600) 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd38a158640 1 -- 172.21.15.45:0/1765561520 learned_addr learned my addr 172.21.15.45:0/1765561520 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd389156640 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd77fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.705+0000 7fd389957640 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd7793b80 0x560dd77fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd38a158640 1 -- 172.21.15.45:0/1765561520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 msgr2=0x560dd77fbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd38a158640 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd77fbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd38a158640 1 -- 172.21.15.45:0/1765561520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd7793b80 msgr2=0x560dd77fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd38a158640 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd7793b80 0x560dd77fb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd38a158640 1 -- 172.21.15.45:0/1765561520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560dd76fa6a0 con 0x560dd77a1b60 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd38a158640 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd77a1b60 0x560dd77fc330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd378009860 tx=0x7fd37802bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e386d532c416339e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd3697fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd3697fa640 1 -- 172.21.15.45:0/1765561520 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37802c8a0 con 0x560dd77a1b60 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd3697fa640 1 -- 172.21.15.45:0/1765561520 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd37802ca00 con 0x560dd77a1b60 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd38e97f740 1 -- 172.21.15.45:0/1765561520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560dd73fb600 con 0x560dd77a1b60 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd3697fa640 1 -- 172.21.15.45:0/1765561520 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd37802cbc0 con 0x560dd77a1b60 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd38e97f740 10 client.16251 fetch_fsmap learned FSMap version 29 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.706+0000 7fd38e97f740 1 -- 172.21.15.45:0/1765561520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560dd76bca50 con 0x560dd77a1b60 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.707+0000 7fd3697fa640 1 -- 172.21.15.45:0/1765561520 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd378062080 con 0x560dd77a1b60 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.707+0000 7fd3697fa640 1 -- 172.21.15.45:0/1765561520 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd378032a70 con 0x560dd77a1b60 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.707+0000 7fd38e97f740 10 client.16251 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.707+0000 7fd38e97f740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:33:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.707+0000 7fd38e97f740 20 client.16251 populate_metadata read hostname 'smithi045' 2024-01-31T05:33:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.707+0000 7fd38e97f740 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560dd7808560 0x560dd780a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.707+0000 7fd38e97f740 4 client.16251 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:33:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.707+0000 7fd38e97f740 1 -- 172.21.15.45:0/1765561520 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560dd744ec50 con 0x560dd7808560 2024-01-31T05:33:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.708+0000 7fd389957640 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560dd7808560 0x560dd780a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.708+0000 7fd389957640 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560dd7808560 0x560dd780a980 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f2e97a0db0f992d server_cookie=f7ebf0c5e66cbbad in_seq=0 out_seq=0 2024-01-31T05:33:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.708+0000 7fd3697fa640 10 client.16251 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.720+0000 7fd3697fa640 1 -- 172.21.15.45:0/1765561520 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560dd744ec50 con 0x560dd7808560 2024-01-31T05:33:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.720+0000 7fd3697fa640 10 client.16251 handle_command_reply: tid=0 2024-01-31T05:33:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.720+0000 7fd38e97f740 1 client.16251 shutdown 2024-01-31T05:33:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38e97f740 1 -- 172.21.15.45:0/1765561520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd77a1b60 msgr2=0x560dd77fc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38e97f740 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd77a1b60 0x560dd77fc330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd378009860 tx=0x7fd37802bb70 comp rx=0 tx=0).stop 2024-01-31T05:33:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38e97f740 1 -- 172.21.15.45:0/1765561520 shutdown_connections 2024-01-31T05:33:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38e97f740 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x560dd7808560 0x560dd780a980 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38e97f740 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dd77a1b60 0x560dd77fc330 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38e97f740 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560dd77944a0 0x560dd77fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd3697fa640 0 client.16251 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38e97f740 1 --2- 172.21.15.45:0/1765561520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560dd7793b80 0x560dd77fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38e97f740 1 -- 172.21.15.45:0/1765561520 >> 172.21.15.45:0/1765561520 conn(0x560dd778e980 msgr2=0x560dd7799d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38e97f740 1 -- 172.21.15.45:0/1765561520 shutdown_connections 2024-01-31T05:33:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38e97f740 1 -- 172.21.15.45:0/1765561520 wait complete. 2024-01-31T05:33:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38e97f740 20 client.16251 trim_cache size 0 max 16384 2024-01-31T05:33:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:33:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:33:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:33:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:33:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38bbbb640 1 -- 172.21.15.45:0/1302163108 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd36c0a1570 msgr2=0x7fd36c0a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38bbbb640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd36c0a1570 0x7fd36c0a3a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd378009830 tx=0x7fd37802cea0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38bbbb640 1 -- 172.21.15.45:0/1302163108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd384100f20 msgr2=0x7fd384197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38bbbb640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd384100f20 0x7fd384197170 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00e9c0 tx=0x7fd37c00ee80 comp rx=0 tx=0).stop 2024-01-31T05:33:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38bbbb640 1 -- 172.21.15.45:0/1302163108 shutdown_connections 2024-01-31T05:33:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38bbbb640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd36c0a1570 0x7fd36c0a3a30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38bbbb640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd38419ba30 0x7fd38419bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38bbbb640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd384105ff0 0x7fd3841976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38bbbb640 1 --2- 172.21.15.45:0/1302163108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd384100f20 0x7fd384197170 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.721+0000 7fd38bbbb640 1 -- 172.21.15.45:0/1302163108 >> 172.21.15.45:0/1302163108 conn(0x7fd3840fc7c0 msgr2=0x7fd3840fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.722+0000 7fd38bbbb640 1 -- 172.21.15.45:0/1302163108 shutdown_connections 2024-01-31T05:33:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:10.722+0000 7fd38bbbb640 1 -- 172.21.15.45:0/1302163108 wait complete. 2024-01-31T05:33:10.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1116270 10.72 MB/sec execute 430 sec latency 18.627 ms 2024-01-31T05:33:10.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:10 smithi045 ceph-mon[31585]: pgmap v330: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-01-31T05:33:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:10 smithi156 ceph-mon[35720]: pgmap v330: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-01-31T05:33:11.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:10 smithi186 ceph-mon[35659]: pgmap v330: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-01-31T05:33:11.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1118513 10.73 MB/sec execute 431 sec latency 60.931 ms 2024-01-31T05:33:11.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:11 smithi045 ceph-mon[31585]: osdmap e103: 12 total, 12 up, 12 in 2024-01-31T05:33:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:11 smithi156 ceph-mon[35720]: osdmap e103: 12 total, 12 up, 12 in 2024-01-31T05:33:12.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:11 smithi186 ceph-mon[35659]: osdmap e103: 12 total, 12 up, 12 in 2024-01-31T05:33:12.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1119618 10.72 MB/sec execute 432 sec latency 23.652 ms 2024-01-31T05:33:12.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:12 smithi045 ceph-mon[31585]: pgmap v332: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 241 op/s 2024-01-31T05:33:12.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:12 smithi156 ceph-mon[35720]: pgmap v332: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 241 op/s 2024-01-31T05:33:13.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:12 smithi186 ceph-mon[35659]: pgmap v332: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 241 op/s 2024-01-31T05:33:13.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1122070 10.72 MB/sec execute 433 sec latency 25.374 ms 2024-01-31T05:33:13.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:13 smithi156 ceph-mon[35720]: pgmap v333: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 200 op/s 2024-01-31T05:33:14.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:13 smithi186 ceph-mon[35659]: pgmap v333: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 200 op/s 2024-01-31T05:33:14.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:13 smithi045 ceph-mon[31585]: pgmap v333: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 200 op/s 2024-01-31T05:33:14.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1122975 10.71 MB/sec execute 434 sec latency 63.157 ms 2024-01-31T05:33:15.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1124998 10.71 MB/sec execute 435 sec latency 24.630 ms 2024-01-31T05:33:15.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:15 smithi156 ceph-mon[35720]: pgmap v334: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 200 op/s 2024-01-31T05:33:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:15 smithi186 ceph-mon[35659]: pgmap v334: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 200 op/s 2024-01-31T05:33:16.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:15 smithi045 ceph-mon[31585]: pgmap v334: 97 pgs: 97 active+clean; 570 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 200 op/s 2024-01-31T05:33:16.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1126429 10.71 MB/sec execute 436 sec latency 56.847 ms 2024-01-31T05:33:17.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1128542 10.70 MB/sec execute 437 sec latency 20.894 ms 2024-01-31T05:33:18.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1129934 10.70 MB/sec execute 438 sec latency 25.875 ms 2024-01-31T05:33:18.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:18 smithi045 ceph-mon[31585]: pgmap v335: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-01-31T05:33:18.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:18 smithi045 ceph-mon[31585]: osdmap e104: 12 total, 12 up, 12 in 2024-01-31T05:33:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:18 smithi156 ceph-mon[35720]: pgmap v335: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-01-31T05:33:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:18 smithi156 ceph-mon[35720]: osdmap e104: 12 total, 12 up, 12 in 2024-01-31T05:33:19.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:18 smithi186 ceph-mon[35659]: pgmap v335: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-01-31T05:33:19.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:18 smithi186 ceph-mon[35659]: osdmap e104: 12 total, 12 up, 12 in 2024-01-31T05:33:19.797 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1131762 10.69 MB/sec execute 439 sec latency 58.414 ms 2024-01-31T05:33:20.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1133411 10.70 MB/sec execute 440 sec latency 15.566 ms 2024-01-31T05:33:20.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:20 smithi045 ceph-mon[31585]: pgmap v337: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.5 MiB/s wr, 230 op/s 2024-01-31T05:33:20.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:20 smithi156 ceph-mon[35720]: pgmap v337: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.5 MiB/s wr, 230 op/s 2024-01-31T05:33:21.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:20 smithi186 ceph-mon[35659]: pgmap v337: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.5 MiB/s wr, 230 op/s 2024-01-31T05:33:21.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1135291 10.68 MB/sec execute 441 sec latency 17.790 ms 2024-01-31T05:33:22.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1137044 10.69 MB/sec execute 442 sec latency 21.069 ms 2024-01-31T05:33:22.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:22 smithi045 ceph-mon[31585]: pgmap v338: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-01-31T05:33:22.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:22 smithi156 ceph-mon[35720]: pgmap v338: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-01-31T05:33:23.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:22 smithi186 ceph-mon[35659]: pgmap v338: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-01-31T05:33:23.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1138479 10.68 MB/sec execute 443 sec latency 212.159 ms 2024-01-31T05:33:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.812+0000 7fe258998640 1 -- 172.21.15.45:0/3117610213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe254068d10 msgr2=0x7fe25410db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.812+0000 7fe258998640 1 --2- 172.21.15.45:0/3117610213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe254068d10 0x7fe25410db80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009900 tx=0x7fe24c02bcd0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.812+0000 7fe258998640 1 -- 172.21.15.45:0/3117610213 shutdown_connections 2024-01-31T05:33:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.812+0000 7fe258998640 1 --2- 172.21.15.45:0/3117610213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe254068d10 0x7fe25410db80 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.812+0000 7fe258998640 1 --2- 172.21.15.45:0/3117610213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe254068370 0x7fe2540687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.812+0000 7fe258998640 1 --2- 172.21.15.45:0/3117610213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe254106de0 0x7fe2541071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.812+0000 7fe258998640 1 -- 172.21.15.45:0/3117610213 >> 172.21.15.45:0/3117610213 conn(0x7fe254076a40 msgr2=0x7fe254076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.812+0000 7fe258998640 1 -- 172.21.15.45:0/3117610213 shutdown_connections 2024-01-31T05:33:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.812+0000 7fe258998640 1 -- 172.21.15.45:0/3117610213 wait complete. 2024-01-31T05:33:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe258998640 1 Processor -- start 2024-01-31T05:33:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe258998640 1 -- start start 2024-01-31T05:33:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe258998640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe254068370 0x7fe254197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe258998640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe254068d10 0x7fe2541975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe258998640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe254106de0 0x7fe25419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe258998640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25419bf20 con 0x7fe254068370 2024-01-31T05:33:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe258998640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25419c090 con 0x7fe254106de0 2024-01-31T05:33:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe258998640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25419c250 con 0x7fe254068d10 2024-01-31T05:33:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe25259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe254068370 0x7fe254197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe25259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe254068370 0x7fe254197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54110/0 (socket says 172.21.15.45:54110) 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe25259c640 1 -- 172.21.15.45:0/2444504978 learned_addr learned my addr 172.21.15.45:0/2444504978 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe251d9b640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe254068d10 0x7fe2541975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.813+0000 7fe252d9d640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe254106de0 0x7fe25419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.814+0000 7fe25259c640 1 -- 172.21.15.45:0/2444504978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe254068d10 msgr2=0x7fe2541975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.814+0000 7fe25259c640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe254068d10 0x7fe2541975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.814+0000 7fe25259c640 1 -- 172.21.15.45:0/2444504978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe254106de0 msgr2=0x7fe25419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.814+0000 7fe25259c640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe254106de0 0x7fe25419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.814+0000 7fe25259c640 1 -- 172.21.15.45:0/2444504978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c009560 con 0x7fe254068370 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.814+0000 7fe25259c640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe254068370 0x7fe254197060 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe24400ebc0 tx=0x7fe24400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.814+0000 7fe24b7fe640 1 -- 172.21.15.45:0/2444504978 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2440099b0 con 0x7fe254068370 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.814+0000 7fe24b7fe640 1 -- 172.21.15.45:0/2444504978 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe24400eec0 con 0x7fe254068370 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.814+0000 7fe258998640 1 -- 172.21.15.45:0/2444504978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25419c530 con 0x7fe254068370 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.814+0000 7fe24b7fe640 1 -- 172.21.15.45:0/2444504978 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe244018aa0 con 0x7fe254068370 2024-01-31T05:33:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.814+0000 7fe258998640 1 -- 172.21.15.45:0/2444504978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2541a3db0 con 0x7fe254068370 2024-01-31T05:33:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.816+0000 7fe24b7fe640 1 -- 172.21.15.45:0/2444504978 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fe244018c00 con 0x7fe254068370 2024-01-31T05:33:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe25b835740 1 Processor -- start 2024-01-31T05:33:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe25b835740 1 -- start start 2024-01-31T05:33:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe25b835740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x556577764440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe25b835740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556577764980 0x55657776d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe25b835740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55657776d750 0x55657776fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe25b835740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565776d3040 con 0x55657776d750 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe25b835740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565776d0730 con 0x556577764980 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe25b835740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565776cc020 con 0x556577764060 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe24b7fe640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe2400a14f0 0x7fe2400a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55657776d750 0x55657776fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe252d9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x556577764440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe252d9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x556577764440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:60740/0 (socket says 172.21.15.45:60740) 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe252d9d640 1 -- 172.21.15.45:0/3922449498 learned_addr learned my addr 172.21.15.45:0/3922449498 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55657776d750 0x55657776fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54120/0 (socket says 172.21.15.45:54120) 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe252d9d640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe2400a14f0 0x7fe2400a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe24b7fe640 1 -- 172.21.15.45:0/2444504978 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe244014070 con 0x7fe254068370 2024-01-31T05:33:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.817+0000 7fe25259c640 1 --2- 172.21.15.45:0/3922449498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556577764980 0x55657776d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.818+0000 7fe252d9d640 1 -- 172.21.15.45:0/3922449498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556577764980 msgr2=0x55657776d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.818+0000 7fe252d9d640 1 --2- 172.21.15.45:0/3922449498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556577764980 0x55657776d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.818+0000 7fe252d9d640 1 -- 172.21.15.45:0/3922449498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55657776d750 msgr2=0x55657776fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.818+0000 7fe252d9d640 1 --2- 172.21.15.45:0/3922449498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55657776d750 0x55657776fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.818+0000 7fe252d9d640 1 -- 172.21.15.45:0/3922449498 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565776c3e40 con 0x556577764060 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.818+0000 7fe251d9b640 1 --2- 172.21.15.45:0/3922449498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55657776d750 0x55657776fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.818+0000 7fe252d9d640 1 --2- 172.21.15.45:0/3922449498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x556577764440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe24c036040 tx=0x7fe24c0041f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4258a365893d767e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.818+0000 7fe252d9d640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe2400a14f0 0x7fe2400a39b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe24c006010 tx=0x7fe24c043000 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.818+0000 7fe248ff9640 1 -- 172.21.15.45:0/3922449498 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c048070 con 0x556577764060 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.818+0000 7fe248ff9640 1 -- 172.21.15.45:0/3922449498 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe24c0054c0 con 0x556577764060 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.818+0000 7fe248ff9640 1 -- 172.21.15.45:0/3922449498 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c0035f0 con 0x556577764060 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.819+0000 7fe25b835740 1 -- 172.21.15.45:0/3922449498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 msgr2=0x556577764440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.819+0000 7fe25b835740 1 --2- 172.21.15.45:0/3922449498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x556577764440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe24c036040 tx=0x7fe24c0041f0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.819+0000 7fe25b835740 1 -- 172.21.15.45:0/3922449498 shutdown_connections 2024-01-31T05:33:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.819+0000 7fe25b835740 1 --2- 172.21.15.45:0/3922449498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55657776d750 0x55657776fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.819+0000 7fe25b835740 1 --2- 172.21.15.45:0/3922449498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556577764980 0x55657776d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.819+0000 7fe25b835740 1 --2- 172.21.15.45:0/3922449498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x556577764440 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.819+0000 7fe25b835740 1 -- 172.21.15.45:0/3922449498 >> 172.21.15.45:0/3922449498 conn(0x55657775e980 msgr2=0x55657775ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.819+0000 7fe25b835740 1 -- 172.21.15.45:0/3922449498 shutdown_connections 2024-01-31T05:33:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.819+0000 7fe25b835740 1 -- 172.21.15.45:0/3922449498 wait complete. 2024-01-31T05:33:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.819+0000 7fe25b835740 1 -- start start 2024-01-31T05:33:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.819+0000 7fe25b835740 1 Processor -- start 2024-01-31T05:33:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.819+0000 7fe25b835740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x55657776a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe25b835740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x55657776abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe25b835740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x55657776b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe25b835740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565776851b0 con 0x556577764980 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe25b835740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556577684e50 con 0x55657776d750 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe25b835740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657764a090 con 0x556577764060 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe25b835740 10 client.0 _ll_register_callbacks cb 0x55657766fa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x55657776abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x55657776abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54128/0 (socket says 172.21.15.45:54128) 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe25259c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x55657776a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe251d9b640 1 -- 172.21.15.45:0/1423293055 learned_addr learned my addr 172.21.15.45:0/1423293055 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe252d9d640 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x55657776b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe251d9b640 1 -- 172.21.15.45:0/1423293055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 msgr2=0x55657776a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe251d9b640 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x55657776a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe251d9b640 1 -- 172.21.15.45:0/1423293055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 msgr2=0x55657776b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe251d9b640 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x55657776b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe251d9b640 1 -- 172.21.15.45:0/1423293055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25410e3c0 con 0x556577764980 2024-01-31T05:33:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe252d9d640 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x55657776b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:33:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe25259c640 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x55657776a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:33:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe251d9b640 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x55657776abf0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe23c012730 tx=0x7fe23c012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc7aaf9fe37c6a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:33:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe239ffb640 1 -- 172.21.15.45:0/1423293055 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c017070 con 0x556577764980 2024-01-31T05:33:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe239ffb640 1 -- 172.21.15.45:0/1423293055 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe23c004030 con 0x556577764980 2024-01-31T05:33:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe25b835740 1 -- 172.21.15.45:0/1423293055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565776c5010 con 0x556577764980 2024-01-31T05:33:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe239ffb640 1 -- 172.21.15.45:0/1423293055 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe23c0041f0 con 0x556577764980 2024-01-31T05:33:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe25b835740 10 client.16263 fetch_fsmap learned FSMap version 29 2024-01-31T05:33:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.820+0000 7fe25b835740 1 -- 172.21.15.45:0/1423293055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5565776ca6a0 con 0x556577764980 2024-01-31T05:33:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.821+0000 7fe239ffb640 1 -- 172.21.15.45:0/1423293055 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565776c5010 con 0x556577764980 2024-01-31T05:33:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.821+0000 7fe239ffb640 1 -- 172.21.15.45:0/1423293055 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe23c012d20 con 0x556577764980 2024-01-31T05:33:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.821+0000 7fe25b835740 10 client.16263 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.821+0000 7fe25b835740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.821+0000 7fe25b835740 20 client.16263 populate_metadata read hostname 'smithi045' 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.821+0000 7fe25b835740 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55657777d4f0 0x55657777f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.821+0000 7fe25b835740 4 client.16263 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.821+0000 7fe25b835740 1 -- 172.21.15.45:0/1423293055 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565776c4d30 con 0x55657777d4f0 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.821+0000 7fe25259c640 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55657777d4f0 0x55657777f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe25259c640 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55657777d4f0 0x55657777f930 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b100d341165a168a server_cookie=37fae38410f30ce0 in_seq=0 out_seq=0 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe239ffb640 10 client.16263 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe239ffb640 1 -- 172.21.15.45:0/1423293055 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565776c4d30 con 0x55657777d4f0 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe239ffb640 10 client.16263 handle_command_reply: tid=0 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe25b835740 1 client.16263 shutdown 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe25b835740 1 -- 172.21.15.45:0/1423293055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 msgr2=0x55657776abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe25b835740 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x55657776abf0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe23c012730 tx=0x7fe23c012bf0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe25b835740 1 -- 172.21.15.45:0/1423293055 shutdown_connections 2024-01-31T05:33:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe25b835740 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55657777d4f0 0x55657777f930 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe25b835740 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x55657776abf0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe25b835740 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x55657776b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe239ffb640 0 client.16263 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe25b835740 1 --2- 172.21.15.45:0/1423293055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x55657776a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe25b835740 1 -- 172.21.15.45:0/1423293055 >> 172.21.15.45:0/1423293055 conn(0x55657775e980 msgr2=0x5565777606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe25b835740 1 -- 172.21.15.45:0/1423293055 shutdown_connections 2024-01-31T05:33:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.822+0000 7fe25b835740 1 -- 172.21.15.45:0/1423293055 wait complete. 2024-01-31T05:33:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.823+0000 7fe25b835740 20 client.16263 trim_cache size 0 max 16384 2024-01-31T05:33:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.837+0000 7fe25b835740 1 Processor -- start 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.837+0000 7fe25b835740 1 -- start start 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe25b835740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x55657776cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe25b835740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x55657776af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe25b835740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x556577767970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe25b835740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556577766100 con 0x556577764980 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe25b835740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565776c4d30 con 0x55657776d750 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe25b835740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565776c5010 con 0x556577764060 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x55657776af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x55657776af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54144/0 (socket says 172.21.15.45:54144) 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe251d9b640 1 -- 172.21.15.45:0/1863148694 learned_addr learned my addr 172.21.15.45:0/1863148694 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe252d9d640 1 --2- 172.21.15.45:0/1863148694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x556577767970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe25259c640 1 --2- 172.21.15.45:0/1863148694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x55657776cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe251d9b640 1 -- 172.21.15.45:0/1863148694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 msgr2=0x55657776cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe251d9b640 1 --2- 172.21.15.45:0/1863148694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x55657776cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe251d9b640 1 -- 172.21.15.45:0/1863148694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 msgr2=0x556577767970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe251d9b640 1 --2- 172.21.15.45:0/1863148694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x556577767970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe251d9b640 1 -- 172.21.15.45:0/1863148694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556577667030 con 0x556577764980 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe25259c640 1 --2- 172.21.15.45:0/1863148694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x55657776cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe251d9b640 1 --2- 172.21.15.45:0/1863148694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x55657776af50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0098d0 tx=0x7fe23c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b50c15a384a97ed server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe23bfff640 1 -- 172.21.15.45:0/1863148694 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c017070 con 0x556577764980 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe23bfff640 1 -- 172.21.15.45:0/1863148694 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe23c018040 con 0x556577764980 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.838+0000 7fe23bfff640 1 -- 172.21.15.45:0/1863148694 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0052d0 con 0x556577764980 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.839+0000 7fe25b835740 1 -- 172.21.15.45:0/1863148694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 msgr2=0x55657776af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.839+0000 7fe25b835740 1 --2- 172.21.15.45:0/1863148694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x55657776af50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0098d0 tx=0x7fe23c005850 comp rx=0 tx=0).stop 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.839+0000 7fe25b835740 1 -- 172.21.15.45:0/1863148694 shutdown_connections 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.839+0000 7fe25b835740 1 --2- 172.21.15.45:0/1863148694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x55657776af50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.839+0000 7fe25b835740 1 --2- 172.21.15.45:0/1863148694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x556577767970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.839+0000 7fe25b835740 1 --2- 172.21.15.45:0/1863148694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x55657776cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.839+0000 7fe25b835740 1 -- 172.21.15.45:0/1863148694 >> 172.21.15.45:0/1863148694 conn(0x55657775e980 msgr2=0x556577768fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.839+0000 7fe25b835740 1 -- 172.21.15.45:0/1863148694 shutdown_connections 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.839+0000 7fe25b835740 1 -- 172.21.15.45:0/1863148694 wait complete. 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.839+0000 7fe25b835740 1 -- start start 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe25b835740 1 Processor -- start 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe25b835740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x556577773190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe25b835740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x5565777736d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe25b835740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x5565777d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe25b835740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556577773fa0 con 0x556577764980 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe25b835740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565777d3fa0 con 0x55657776d750 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe25b835740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565777d40e0 con 0x556577764060 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe25b835740 10 client.0 _ll_register_callbacks cb 0x7fe2440093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:33:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x5565777736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x5565777736d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54150/0 (socket says 172.21.15.45:54150) 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe251d9b640 1 -- 172.21.15.45:0/4113433701 learned_addr learned my addr 172.21.15.45:0/4113433701 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe25259c640 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x556577773190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe252d9d640 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x5565777d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe251d9b640 1 -- 172.21.15.45:0/4113433701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 msgr2=0x556577773190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe251d9b640 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x556577773190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe251d9b640 1 -- 172.21.15.45:0/4113433701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 msgr2=0x5565777d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe251d9b640 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x5565777d3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe251d9b640 1 -- 172.21.15.45:0/4113433701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565776ca6a0 con 0x556577764980 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe251d9b640 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x5565777736d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00f890 tx=0x7fe23c003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a841c8c44ea423 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.840+0000 7fe239ffb640 1 -- 172.21.15.45:0/4113433701 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c017040 con 0x556577764980 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe239ffb640 1 -- 172.21.15.45:0/4113433701 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe23c005a70 con 0x556577764980 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe25b835740 1 -- 172.21.15.45:0/4113433701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe24c0054c0 con 0x556577764980 2024-01-31T05:33:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe239ffb640 1 -- 172.21.15.45:0/4113433701 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe23c0035f0 con 0x556577764980 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe25b835740 10 client.16269 fetch_fsmap learned FSMap version 29 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe25b835740 1 -- 172.21.15.45:0/4113433701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55657768ca50 con 0x556577764980 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe239ffb640 1 -- 172.21.15.45:0/4113433701 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe24c0054c0 con 0x556577764980 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe239ffb640 1 -- 172.21.15.45:0/4113433701 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe23c025b20 con 0x556577764980 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe25b835740 10 client.16269 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe25b835740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe25b835740 20 client.16269 populate_metadata read hostname 'smithi045' 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe25b835740 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5565777d54e0 0x5565777d7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe25b835740 4 client.16269 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe25b835740 1 -- 172.21.15.45:0/4113433701 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5565773cb600 con 0x5565777d54e0 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.841+0000 7fe25259c640 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5565777d54e0 0x5565777d7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.842+0000 7fe239ffb640 10 client.16269 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.842+0000 7fe25259c640 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5565777d54e0 0x5565777d7900 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2ed78ae1d68a09c server_cookie=72e7b6a6c973ff62 in_seq=0 out_seq=0 2024-01-31T05:33:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe239ffb640 1 -- 172.21.15.45:0/4113433701 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5565773cb600 con 0x5565777d54e0 2024-01-31T05:33:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe239ffb640 10 client.16269 handle_command_reply: tid=0 2024-01-31T05:33:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe25b835740 1 client.16269 shutdown 2024-01-31T05:33:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe25b835740 1 -- 172.21.15.45:0/4113433701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 msgr2=0x5565777736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe25b835740 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x5565777736d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00f890 tx=0x7fe23c003ee0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe25b835740 1 -- 172.21.15.45:0/4113433701 shutdown_connections 2024-01-31T05:33:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe25b835740 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5565777d54e0 0x5565777d7900 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe25b835740 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556577764980 0x5565777736d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe25b835740 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55657776d750 0x5565777d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe25b835740 1 --2- 172.21.15.45:0/4113433701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556577764060 0x556577773190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe239ffb640 0 client.16269 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe25b835740 1 -- 172.21.15.45:0/4113433701 >> 172.21.15.45:0/4113433701 conn(0x55657775e980 msgr2=0x55657775da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe25b835740 1 -- 172.21.15.45:0/4113433701 shutdown_connections 2024-01-31T05:33:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe25b835740 1 -- 172.21.15.45:0/4113433701 wait complete. 2024-01-31T05:33:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe25b835740 20 client.16269 trim_cache size 0 max 16384 2024-01-31T05:33:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:33:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:33:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:33:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:33:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe258998640 1 -- 172.21.15.45:0/2444504978 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe2400a14f0 msgr2=0x7fe2400a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe258998640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe2400a14f0 0x7fe2400a39b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe24c006010 tx=0x7fe24c043000 comp rx=0 tx=0).stop 2024-01-31T05:33:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe258998640 1 -- 172.21.15.45:0/2444504978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe254068370 msgr2=0x7fe254197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.853+0000 7fe258998640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe254068370 0x7fe254197060 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe24400ebc0 tx=0x7fe24400c6a0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.854+0000 7fe258998640 1 -- 172.21.15.45:0/2444504978 shutdown_connections 2024-01-31T05:33:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.854+0000 7fe258998640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fe2400a14f0 0x7fe2400a39b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.854+0000 7fe258998640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe254106de0 0x7fe25419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.854+0000 7fe258998640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe254068d10 0x7fe2541975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.854+0000 7fe258998640 1 --2- 172.21.15.45:0/2444504978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe254068370 0x7fe254197060 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.854+0000 7fe258998640 1 -- 172.21.15.45:0/2444504978 >> 172.21.15.45:0/2444504978 conn(0x7fe254076a40 msgr2=0x7fe2540fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.854+0000 7fe258998640 1 -- 172.21.15.45:0/2444504978 shutdown_connections 2024-01-31T05:33:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:23.854+0000 7fe258998640 1 -- 172.21.15.45:0/2444504978 wait complete. 2024-01-31T05:33:24.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1140318 10.68 MB/sec execute 444 sec latency 85.244 ms 2024-01-31T05:33:24.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:24 smithi045 ceph-mon[31585]: pgmap v339: 97 pgs: 97 active+clean; 592 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 202 op/s 2024-01-31T05:33:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:24 smithi156 ceph-mon[35720]: pgmap v339: 97 pgs: 97 active+clean; 592 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 202 op/s 2024-01-31T05:33:25.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:24 smithi186 ceph-mon[35659]: pgmap v339: 97 pgs: 97 active+clean; 592 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 202 op/s 2024-01-31T05:33:25.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1141810 10.67 MB/sec execute 445 sec latency 19.915 ms 2024-01-31T05:33:26.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1143866 10.68 MB/sec execute 446 sec latency 14.728 ms 2024-01-31T05:33:26.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:26 smithi045 ceph-mon[31585]: pgmap v340: 97 pgs: 97 active+clean; 592 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 202 op/s 2024-01-31T05:33:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:26 smithi156 ceph-mon[35720]: pgmap v340: 97 pgs: 97 active+clean; 592 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 202 op/s 2024-01-31T05:33:27.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:26 smithi186 ceph-mon[35659]: pgmap v340: 97 pgs: 97 active+clean; 592 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 202 op/s 2024-01-31T05:33:27.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1145335 10.67 MB/sec execute 447 sec latency 16.276 ms 2024-01-31T05:33:28.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1147396 10.67 MB/sec execute 448 sec latency 70.450 ms 2024-01-31T05:33:28.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:28 smithi045 ceph-mon[31585]: pgmap v341: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 199 op/s 2024-01-31T05:33:28.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:28 smithi186 ceph-mon[35659]: pgmap v341: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 199 op/s 2024-01-31T05:33:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:28 smithi156 ceph-mon[35720]: pgmap v341: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 199 op/s 2024-01-31T05:33:29.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1148551 10.66 MB/sec execute 449 sec latency 106.227 ms 2024-01-31T05:33:30.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1150837 10.67 MB/sec execute 450 sec latency 21.183 ms 2024-01-31T05:33:30.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:30 smithi045 ceph-mon[31585]: pgmap v342: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 166 op/s 2024-01-31T05:33:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:30 smithi156 ceph-mon[35720]: pgmap v342: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 166 op/s 2024-01-31T05:33:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:30 smithi186 ceph-mon[35659]: pgmap v342: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 166 op/s 2024-01-31T05:33:31.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1151903 10.65 MB/sec execute 451 sec latency 19.355 ms 2024-01-31T05:33:32.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1154378 10.66 MB/sec execute 452 sec latency 14.941 ms 2024-01-31T05:33:32.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:32 smithi045 ceph-mon[31585]: pgmap v343: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 165 op/s 2024-01-31T05:33:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:32 smithi156 ceph-mon[35720]: pgmap v343: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 165 op/s 2024-01-31T05:33:33.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:32 smithi186 ceph-mon[35659]: pgmap v343: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 165 op/s 2024-01-31T05:33:33.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1155298 10.64 MB/sec execute 453 sec latency 47.140 ms 2024-01-31T05:33:34.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1157370 10.65 MB/sec execute 454 sec latency 80.162 ms 2024-01-31T05:33:34.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:34 smithi045 ceph-mon[31585]: pgmap v344: 97 pgs: 97 active+clean; 563 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2024-01-31T05:33:34.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:34 smithi156 ceph-mon[35720]: pgmap v344: 97 pgs: 97 active+clean; 563 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2024-01-31T05:33:35.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:34 smithi186 ceph-mon[35659]: pgmap v344: 97 pgs: 97 active+clean; 563 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2024-01-31T05:33:35.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1158637 10.64 MB/sec execute 455 sec latency 30.433 ms 2024-01-31T05:33:36.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1160924 10.64 MB/sec execute 456 sec latency 22.561 ms 2024-01-31T05:33:36.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:36 smithi045 ceph-mon[31585]: pgmap v345: 97 pgs: 97 active+clean; 563 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 173 op/s 2024-01-31T05:33:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.942+0000 7f9725101640 1 -- 172.21.15.45:0/1804036042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972010d4b0 msgr2=0x7f972010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.942+0000 7f9725101640 1 --2- 172.21.15.45:0/1804036042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972010d4b0 0x7f972010f920 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9710009ca0 tx=0x7f971002c180 comp rx=0 tx=0).stop 2024-01-31T05:33:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.942+0000 7f9725101640 1 -- 172.21.15.45:0/1804036042 shutdown_connections 2024-01-31T05:33:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.942+0000 7f9725101640 1 --2- 172.21.15.45:0/1804036042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972010d4b0 0x7f972010f920 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.942+0000 7f9725101640 1 --2- 172.21.15.45:0/1804036042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97200688b0 0x7f9720068cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.942+0000 7f9725101640 1 --2- 172.21.15.45:0/1804036042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9720101c70 0x7f9720068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.942+0000 7f9725101640 1 -- 172.21.15.45:0/1804036042 >> 172.21.15.45:0/1804036042 conn(0x7f9720076a40 msgr2=0x7f9720076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.942+0000 7f9725101640 1 -- 172.21.15.45:0/1804036042 shutdown_connections 2024-01-31T05:33:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.942+0000 7f9725101640 1 -- 172.21.15.45:0/1804036042 wait complete. 2024-01-31T05:33:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.943+0000 7f9725101640 1 Processor -- start 2024-01-31T05:33:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.943+0000 7f9725101640 1 -- start start 2024-01-31T05:33:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.943+0000 7f9725101640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97200688b0 0x7f97201992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.943+0000 7f9725101640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9720101c70 0x7f9720199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.943+0000 7f9725101640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f972010d4b0 0x7f972019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.943+0000 7f9725101640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972019e1b0 con 0x7f97200688b0 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.943+0000 7f9725101640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972019e320 con 0x7f9720101c70 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.943+0000 7f9725101640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972019e4e0 con 0x7f972010d4b0 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f971ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97200688b0 0x7f97201992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f971ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97200688b0 0x7f97201992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39794/0 (socket says 172.21.15.45:39794) 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f971ed9d640 1 -- 172.21.15.45:0/2600193130 learned_addr learned my addr 172.21.15.45:0/2600193130 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f971e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9720101c70 0x7f9720199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f971f59e640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f972010d4b0 0x7f972019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f971ed9d640 1 -- 172.21.15.45:0/2600193130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f972010d4b0 msgr2=0x7f972019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f971ed9d640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f972010d4b0 0x7f972019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f971ed9d640 1 -- 172.21.15.45:0/2600193130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9720101c70 msgr2=0x7f9720199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f971ed9d640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9720101c70 0x7f9720199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f971ed9d640 1 -- 172.21.15.45:0/2600193130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9710009900 con 0x7f97200688b0 2024-01-31T05:33:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f971e59c640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9720101c70 0x7f9720199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:33:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f971ed9d640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97200688b0 0x7f97201992f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f971800e9f0 tx=0x7f971800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f9707fff640 1 -- 172.21.15.45:0/2600193130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718009720 con 0x7f97200688b0 2024-01-31T05:33:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f9725101640 1 -- 172.21.15.45:0/2600193130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972019e7c0 con 0x7f97200688b0 2024-01-31T05:33:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f9707fff640 1 -- 172.21.15.45:0/2600193130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f971800cd80 con 0x7f97200688b0 2024-01-31T05:33:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f9707fff640 1 -- 172.21.15.45:0/2600193130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718010430 con 0x7f97200688b0 2024-01-31T05:33:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.944+0000 7f9725101640 1 -- 172.21.15.45:0/2600193130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97201a6040 con 0x7f97200688b0 2024-01-31T05:33:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.946+0000 7f9707fff640 1 -- 172.21.15.45:0/2600193130 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f9718005020 con 0x7f97200688b0 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f9727f9b740 1 Processor -- start 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f9727f9b740 1 -- start start 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f9707fff640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f97080a14f0 0x7f97080a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f9727f9b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504060 0x55ba3e504440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f9727f9b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504980 0x55ba3e50d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f9727f9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e50d750 0x55ba3e50fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f9727f9b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e473040 con 0x55ba3e504980 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f9727f9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e470730 con 0x55ba3e50d750 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f9727f9b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e46c020 con 0x55ba3e504060 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971f59e640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f97080a14f0 0x7f97080a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504980 0x55ba3e50d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971ed9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504060 0x55ba3e504440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504980 0x55ba3e50d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39798/0 (socket says 172.21.15.45:39798) 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f9707fff640 1 -- 172.21.15.45:0/2600193130 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9718014070 con 0x7f97200688b0 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971e59c640 1 -- 172.21.15.45:0/3804685391 learned_addr learned my addr 172.21.15.45:0/3804685391 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971f59e640 1 --2- 172.21.15.45:0/3804685391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e50d750 0x55ba3e50fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971e59c640 1 -- 172.21.15.45:0/3804685391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504060 msgr2=0x55ba3e504440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971e59c640 1 --2- 172.21.15.45:0/3804685391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504060 0x55ba3e504440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971e59c640 1 -- 172.21.15.45:0/3804685391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e50d750 msgr2=0x55ba3e50fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971e59c640 1 --2- 172.21.15.45:0/3804685391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e50d750 0x55ba3e50fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971e59c640 1 -- 172.21.15.45:0/3804685391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba3e463e40 con 0x55ba3e504980 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971ed9d640 1 --2- 172.21.15.45:0/3804685391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504060 0x55ba3e504440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.947+0000 7f971f59e640 1 --2- 172.21.15.45:0/3804685391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e50d750 0x55ba3e50fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.948+0000 7f971f59e640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f97080a14f0 0x7f97080a39b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9710004af0 tx=0x7f9710034e00 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.948+0000 7f971e59c640 1 --2- 172.21.15.45:0/3804685391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504980 0x55ba3e50d210 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f97140114f0 tx=0x7f97140119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404663fc99846504 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.948+0000 7f97057fa640 1 -- 172.21.15.45:0/3804685391 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714011250 con 0x55ba3e504980 2024-01-31T05:33:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.948+0000 7f97057fa640 1 -- 172.21.15.45:0/3804685391 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97140044f0 con 0x55ba3e504980 2024-01-31T05:33:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.948+0000 7f97057fa640 1 -- 172.21.15.45:0/3804685391 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971400f5c0 con 0x55ba3e504980 2024-01-31T05:33:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.948+0000 7f9727f9b740 1 -- 172.21.15.45:0/3804685391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504980 msgr2=0x55ba3e50d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.948+0000 7f9727f9b740 1 --2- 172.21.15.45:0/3804685391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504980 0x55ba3e50d210 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f97140114f0 tx=0x7f97140119b0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 -- 172.21.15.45:0/3804685391 shutdown_connections 2024-01-31T05:33:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 --2- 172.21.15.45:0/3804685391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504980 0x55ba3e50d210 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 --2- 172.21.15.45:0/3804685391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e50d750 0x55ba3e50fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 --2- 172.21.15.45:0/3804685391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504060 0x55ba3e504440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 -- 172.21.15.45:0/3804685391 >> 172.21.15.45:0/3804685391 conn(0x55ba3e4fe980 msgr2=0x55ba3e4fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 -- 172.21.15.45:0/3804685391 shutdown_connections 2024-01-31T05:33:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 -- 172.21.15.45:0/3804685391 wait complete. 2024-01-31T05:33:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 -- start start 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 Processor -- start 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e504060 0x55ba3e50a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504980 0x55ba3e50abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e50d750 0x55ba3e50b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e4251b0 con 0x55ba3e50d750 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e424e50 con 0x55ba3e504060 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e3ea090 con 0x55ba3e504980 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f9727f9b740 10 client.0 _ll_register_callbacks cb 0x55ba3e40fa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f971f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e50d750 0x55ba3e50b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f971f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e50d750 0x55ba3e50b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39802/0 (socket says 172.21.15.45:39802) 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f971f59e640 1 -- 172.21.15.45:0/442554144 learned_addr learned my addr 172.21.15.45:0/442554144 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f971e59c640 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504980 0x55ba3e50abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.949+0000 7f971ed9d640 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e504060 0x55ba3e50a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f971f59e640 1 -- 172.21.15.45:0/442554144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504980 msgr2=0x55ba3e50abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f971f59e640 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504980 0x55ba3e50abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f971f59e640 1 -- 172.21.15.45:0/442554144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e504060 msgr2=0x55ba3e50a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f971f59e640 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e504060 0x55ba3e50a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f971f59e640 1 -- 172.21.15.45:0/442554144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9714009cc0 con 0x55ba3e50d750 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f971ed9d640 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e504060 0x55ba3e50a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f971f59e640 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e50d750 0x55ba3e50b0f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9710036000 tx=0x7f971002c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a076e9a53bf6483 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f9702ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f9702ff5640 1 -- 172.21.15.45:0/442554144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9710032e30 con 0x55ba3e50d750 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f9702ff5640 1 -- 172.21.15.45:0/442554144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9710034890 con 0x55ba3e50d750 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f9727f9b740 1 -- 172.21.15.45:0/442554144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba3e465010 con 0x55ba3e50d750 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f9702ff5640 1 -- 172.21.15.45:0/442554144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9710003e70 con 0x55ba3e50d750 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f9727f9b740 10 client.16281 fetch_fsmap learned FSMap version 29 2024-01-31T05:33:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f9727f9b740 1 -- 172.21.15.45:0/442554144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ba3e46a6a0 con 0x55ba3e50d750 2024-01-31T05:33:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.950+0000 7f9702ff5640 1 -- 172.21.15.45:0/442554144 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba3e465010 con 0x55ba3e50d750 2024-01-31T05:33:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.951+0000 7f9702ff5640 1 -- 172.21.15.45:0/442554144 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9710055060 con 0x55ba3e50d750 2024-01-31T05:33:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.951+0000 7f9727f9b740 10 client.16281 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:33:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.951+0000 7f9727f9b740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:33:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.951+0000 7f9727f9b740 20 client.16281 populate_metadata read hostname 'smithi045' 2024-01-31T05:33:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.951+0000 7f9727f9b740 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba3e51d540 0x55ba3e51f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.951+0000 7f9727f9b740 4 client.16281 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:33:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.951+0000 7f9727f9b740 1 -- 172.21.15.45:0/442554144 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba3e464d30 con 0x55ba3e51d540 2024-01-31T05:33:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.951+0000 7f971ed9d640 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba3e51d540 0x55ba3e51f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.951+0000 7f971ed9d640 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba3e51d540 0x55ba3e51f930 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df224a7d0c15367 server_cookie=f20e31e33ba51d7 in_seq=0 out_seq=0 2024-01-31T05:33:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.951+0000 7f9702ff5640 10 client.16281 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9702ff5640 1 -- 172.21.15.45:0/442554144 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba3e464d30 con 0x55ba3e51d540 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9702ff5640 10 client.16281 handle_command_reply: tid=0 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9727f9b740 1 client.16281 shutdown 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9727f9b740 1 -- 172.21.15.45:0/442554144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e50d750 msgr2=0x55ba3e50b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9727f9b740 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e50d750 0x55ba3e50b0f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9710036000 tx=0x7f971002c180 comp rx=0 tx=0).stop 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9727f9b740 1 -- 172.21.15.45:0/442554144 shutdown_connections 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9727f9b740 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba3e51d540 0x55ba3e51f930 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9727f9b740 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e50d750 0x55ba3e50b0f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9727f9b740 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504980 0x55ba3e50abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9702ff5640 0 client.16281 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9727f9b740 1 --2- 172.21.15.45:0/442554144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e504060 0x55ba3e50a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9727f9b740 1 -- 172.21.15.45:0/442554144 >> 172.21.15.45:0/442554144 conn(0x55ba3e4fe980 msgr2=0x55ba3e5006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9727f9b740 1 -- 172.21.15.45:0/442554144 shutdown_connections 2024-01-31T05:33:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9727f9b740 1 -- 172.21.15.45:0/442554144 wait complete. 2024-01-31T05:33:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.952+0000 7f9727f9b740 20 client.16281 trim_cache size 0 max 16384 2024-01-31T05:33:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f9727f9b740 1 Processor -- start 2024-01-31T05:33:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f9727f9b740 1 -- start start 2024-01-31T05:33:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f9727f9b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 0x55ba3e50cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f9727f9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e504980 0x55ba3e514440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f9727f9b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e50d750 0x55ba3e514980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f9727f9b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e519120 con 0x55ba3e504060 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f9727f9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e505240 con 0x55ba3e504980 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f9727f9b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e503710 con 0x55ba3e50d750 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f971ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 0x55ba3e50cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f971ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 0x55ba3e50cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39818/0 (socket says 172.21.15.45:39818) 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f971ed9d640 1 -- 172.21.15.45:0/2168558871 learned_addr learned my addr 172.21.15.45:0/2168558871 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f971f59e640 1 --2- 172.21.15.45:0/2168558871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e50d750 0x55ba3e514980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f971e59c640 1 --2- 172.21.15.45:0/2168558871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e504980 0x55ba3e514440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f971ed9d640 1 -- 172.21.15.45:0/2168558871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e50d750 msgr2=0x55ba3e514980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f971ed9d640 1 --2- 172.21.15.45:0/2168558871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e50d750 0x55ba3e514980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f971ed9d640 1 -- 172.21.15.45:0/2168558871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e504980 msgr2=0x55ba3e514440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f971ed9d640 1 --2- 172.21.15.45:0/2168558871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e504980 0x55ba3e514440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.966+0000 7f971ed9d640 1 -- 172.21.15.45:0/2168558871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba3e407030 con 0x55ba3e504060 2024-01-31T05:33:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.967+0000 7f971ed9d640 1 --2- 172.21.15.45:0/2168558871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 0x55ba3e50cda0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9718096580 tx=0x7f9718014f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dead744120c10b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.967+0000 7f9704ff9640 1 -- 172.21.15.45:0/2168558871 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97180cd040 con 0x55ba3e504060 2024-01-31T05:33:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.967+0000 7f9704ff9640 1 -- 172.21.15.45:0/2168558871 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9718005670 con 0x55ba3e504060 2024-01-31T05:33:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.967+0000 7f9704ff9640 1 -- 172.21.15.45:0/2168558871 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97180a2410 con 0x55ba3e504060 2024-01-31T05:33:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.967+0000 7f9727f9b740 1 -- 172.21.15.45:0/2168558871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 msgr2=0x55ba3e50cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.967+0000 7f9727f9b740 1 --2- 172.21.15.45:0/2168558871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 0x55ba3e50cda0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9718096580 tx=0x7f9718014f40 comp rx=0 tx=0).stop 2024-01-31T05:33:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.967+0000 7f9727f9b740 1 -- 172.21.15.45:0/2168558871 shutdown_connections 2024-01-31T05:33:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.967+0000 7f9727f9b740 1 --2- 172.21.15.45:0/2168558871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e50d750 0x55ba3e514980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.967+0000 7f9727f9b740 1 --2- 172.21.15.45:0/2168558871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e504980 0x55ba3e514440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.967+0000 7f9727f9b740 1 --2- 172.21.15.45:0/2168558871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 0x55ba3e50cda0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.967+0000 7f9727f9b740 1 -- 172.21.15.45:0/2168558871 >> 172.21.15.45:0/2168558871 conn(0x55ba3e4fe980 msgr2=0x55ba3e50af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.967+0000 7f9727f9b740 1 -- 172.21.15.45:0/2168558871 shutdown_connections 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f9727f9b740 1 -- 172.21.15.45:0/2168558871 wait complete. 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f9727f9b740 1 -- start start 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f9727f9b740 1 Processor -- start 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f9727f9b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 0x55ba3e56d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f9727f9b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504980 0x55ba3e56d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f9727f9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e50d750 0x55ba3e5679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f9727f9b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e567f80 con 0x55ba3e504060 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f9727f9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e5680f0 con 0x55ba3e50d750 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f9727f9b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e5682b0 con 0x55ba3e504980 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f9727f9b740 10 client.0 _ll_register_callbacks cb 0x7f9714009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f971ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 0x55ba3e56d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f971ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 0x55ba3e56d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39820/0 (socket says 172.21.15.45:39820) 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f971ed9d640 1 -- 172.21.15.45:0/186342878 learned_addr learned my addr 172.21.15.45:0/186342878 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f971f59e640 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e50d750 0x55ba3e5679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.968+0000 7f971e59c640 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504980 0x55ba3e56d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f971ed9d640 1 -- 172.21.15.45:0/186342878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504980 msgr2=0x55ba3e56d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f971ed9d640 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504980 0x55ba3e56d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f971ed9d640 1 -- 172.21.15.45:0/186342878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e50d750 msgr2=0x55ba3e5679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f971ed9d640 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e50d750 0x55ba3e5679c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f971ed9d640 1 -- 172.21.15.45:0/186342878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972019e7c0 con 0x55ba3e504060 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f971ed9d640 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 0x55ba3e56d340 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f97180049d0 tx=0x7f9718096440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b291300a699006e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9702ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9702ff5640 1 -- 172.21.15.45:0/186342878 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971809aa10 con 0x55ba3e504060 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9702ff5640 1 -- 172.21.15.45:0/186342878 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f971809b3b0 con 0x55ba3e504060 2024-01-31T05:33:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9727f9b740 1 -- 172.21.15.45:0/186342878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97140044f0 con 0x55ba3e504060 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9702ff5640 1 -- 172.21.15.45:0/186342878 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97180a2410 con 0x55ba3e504060 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9727f9b740 10 client.16287 fetch_fsmap learned FSMap version 29 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9727f9b740 1 -- 172.21.15.45:0/186342878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ba3e42ca50 con 0x55ba3e504060 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9702ff5640 1 -- 172.21.15.45:0/186342878 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97180a7020 con 0x55ba3e504060 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9702ff5640 1 -- 172.21.15.45:0/186342878 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f971801f060 con 0x55ba3e504060 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9727f9b740 10 client.16287 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9727f9b740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9727f9b740 20 client.16287 populate_metadata read hostname 'smithi045' 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9727f9b740 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba3e574e10 0x55ba3e577230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9727f9b740 4 client.16287 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.969+0000 7f9727f9b740 1 -- 172.21.15.45:0/186342878 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ba3e16b600 con 0x55ba3e574e10 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.970+0000 7f971e59c640 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba3e574e10 0x55ba3e577230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.970+0000 7f971e59c640 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba3e574e10 0x55ba3e577230 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3641b71b4fb4870 server_cookie=bf3fbec4e70e421 in_seq=0 out_seq=0 2024-01-31T05:33:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.970+0000 7f9702ff5640 10 client.16287 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:36.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:36 smithi156 ceph-mon[35720]: pgmap v345: 97 pgs: 97 active+clean; 563 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 173 op/s 2024-01-31T05:33:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9702ff5640 1 -- 172.21.15.45:0/186342878 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ba3e16b600 con 0x55ba3e574e10 2024-01-31T05:33:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9702ff5640 10 client.16287 handle_command_reply: tid=0 2024-01-31T05:33:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9727f9b740 1 client.16287 shutdown 2024-01-31T05:33:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9727f9b740 1 -- 172.21.15.45:0/186342878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 msgr2=0x55ba3e56d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9727f9b740 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 0x55ba3e56d340 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f97180049d0 tx=0x7f9718096440 comp rx=0 tx=0).stop 2024-01-31T05:33:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9727f9b740 1 -- 172.21.15.45:0/186342878 shutdown_connections 2024-01-31T05:33:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9727f9b740 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba3e574e10 0x55ba3e577230 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9727f9b740 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba3e50d750 0x55ba3e5679c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9727f9b740 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba3e504980 0x55ba3e56d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9727f9b740 1 --2- 172.21.15.45:0/186342878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba3e504060 0x55ba3e56d340 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9702ff5640 0 client.16287 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9727f9b740 1 -- 172.21.15.45:0/186342878 >> 172.21.15.45:0/186342878 conn(0x55ba3e4fe980 msgr2=0x55ba3e4ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9727f9b740 1 -- 172.21.15.45:0/186342878 shutdown_connections 2024-01-31T05:33:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9727f9b740 1 -- 172.21.15.45:0/186342878 wait complete. 2024-01-31T05:33:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.982+0000 7f9727f9b740 20 client.16287 trim_cache size 0 max 16384 2024-01-31T05:33:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:33:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:33:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:33:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:33:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.983+0000 7f9725101640 1 -- 172.21.15.45:0/2600193130 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f97080a14f0 msgr2=0x7f97080a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.983+0000 7f9725101640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f97080a14f0 0x7f97080a39b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9710004af0 tx=0x7f9710034e00 comp rx=0 tx=0).stop 2024-01-31T05:33:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.983+0000 7f9725101640 1 -- 172.21.15.45:0/2600193130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97200688b0 msgr2=0x7f97201992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.983+0000 7f9725101640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97200688b0 0x7f97201992f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f971800e9f0 tx=0x7f971800edf0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.983+0000 7f9725101640 1 -- 172.21.15.45:0/2600193130 shutdown_connections 2024-01-31T05:33:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.983+0000 7f9725101640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f97080a14f0 0x7f97080a39b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.983+0000 7f9725101640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f972010d4b0 0x7f972019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.983+0000 7f9725101640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9720101c70 0x7f9720199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.983+0000 7f9725101640 1 --2- 172.21.15.45:0/2600193130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97200688b0 0x7f97201992f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.983+0000 7f9725101640 1 -- 172.21.15.45:0/2600193130 >> 172.21.15.45:0/2600193130 conn(0x7f9720076a40 msgr2=0x7f97200fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.983+0000 7f9725101640 1 -- 172.21.15.45:0/2600193130 shutdown_connections 2024-01-31T05:33:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:36.983+0000 7f9725101640 1 -- 172.21.15.45:0/2600193130 wait complete. 2024-01-31T05:33:37.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:36 smithi186 ceph-mon[35659]: pgmap v345: 97 pgs: 97 active+clean; 563 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 173 op/s 2024-01-31T05:33:37.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1162255 10.63 MB/sec execute 457 sec latency 15.784 ms 2024-01-31T05:33:38.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1164649 10.64 MB/sec execute 458 sec latency 27.456 ms 2024-01-31T05:33:38.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:38 smithi045 ceph-mon[31585]: pgmap v346: 97 pgs: 97 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 249 op/s 2024-01-31T05:33:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:38 smithi156 ceph-mon[35720]: pgmap v346: 97 pgs: 97 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 249 op/s 2024-01-31T05:33:39.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:38 smithi186 ceph-mon[35659]: pgmap v346: 97 pgs: 97 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 249 op/s 2024-01-31T05:33:39.798 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1165594 10.63 MB/sec execute 459 sec latency 62.205 ms 2024-01-31T05:33:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:39 smithi156 ceph-mon[35720]: pgmap v347: 97 pgs: 97 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 163 op/s 2024-01-31T05:33:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:39 smithi186 ceph-mon[35659]: pgmap v347: 97 pgs: 97 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 163 op/s 2024-01-31T05:33:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:39 smithi045 ceph-mon[31585]: pgmap v347: 97 pgs: 97 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 163 op/s 2024-01-31T05:33:40.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1167472 10.62 MB/sec execute 460 sec latency 45.107 ms 2024-01-31T05:33:41.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1168998 10.62 MB/sec execute 461 sec latency 48.117 ms 2024-01-31T05:33:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:41 smithi156 ceph-mon[35720]: pgmap v348: 97 pgs: 97 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 163 op/s 2024-01-31T05:33:42.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:41 smithi186 ceph-mon[35659]: pgmap v348: 97 pgs: 97 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 163 op/s 2024-01-31T05:33:42.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:41 smithi045 ceph-mon[31585]: pgmap v348: 97 pgs: 97 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 163 op/s 2024-01-31T05:33:42.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1171150 10.62 MB/sec execute 462 sec latency 20.624 ms 2024-01-31T05:33:43.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1172682 10.62 MB/sec execute 463 sec latency 17.159 ms 2024-01-31T05:33:44.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1174577 10.61 MB/sec execute 464 sec latency 93.480 ms 2024-01-31T05:33:44.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:44 smithi045 ceph-mon[31585]: pgmap v349: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 251 op/s 2024-01-31T05:33:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:44 smithi156 ceph-mon[35720]: pgmap v349: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 251 op/s 2024-01-31T05:33:45.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:44 smithi186 ceph-mon[35659]: pgmap v349: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 251 op/s 2024-01-31T05:33:45.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1175910 10.61 MB/sec execute 465 sec latency 131.196 ms 2024-01-31T05:33:46.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1177636 10.60 MB/sec execute 466 sec latency 17.277 ms 2024-01-31T05:33:46.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:46 smithi045 ceph-mon[31585]: pgmap v350: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 164 op/s 2024-01-31T05:33:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:46 smithi156 ceph-mon[35720]: pgmap v350: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 164 op/s 2024-01-31T05:33:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:46 smithi186 ceph-mon[35659]: pgmap v350: 97 pgs: 97 active+clean; 583 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 164 op/s 2024-01-31T05:33:47.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1179471 10.61 MB/sec execute 467 sec latency 14.677 ms 2024-01-31T05:33:48.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1181116 10.60 MB/sec execute 468 sec latency 18.145 ms 2024-01-31T05:33:48.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:48 smithi045 ceph-mon[31585]: pgmap v351: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 246 op/s 2024-01-31T05:33:48.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:48 smithi156 ceph-mon[35720]: pgmap v351: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 246 op/s 2024-01-31T05:33:49.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:48 smithi186 ceph-mon[35659]: pgmap v351: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 246 op/s 2024-01-31T05:33:49.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1182903 10.60 MB/sec execute 469 sec latency 98.564 ms 2024-01-31T05:33:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.070+0000 7fb2322b1640 1 -- 172.21.15.45:0/3212458242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb22c100f30 msgr2=0x7fb22c101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.070+0000 7fb2322b1640 1 --2- 172.21.15.45:0/3212458242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb22c100f30 0x7fb22c101310 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009930 tx=0x7fb21c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:33:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.070+0000 7fb2322b1640 1 -- 172.21.15.45:0/3212458242 shutdown_connections 2024-01-31T05:33:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.070+0000 7fb2322b1640 1 --2- 172.21.15.45:0/3212458242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22c106000 0x7fb22c10dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.070+0000 7fb2322b1640 1 --2- 172.21.15.45:0/3212458242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb22c1018e0 0x7fb22c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.070+0000 7fb2322b1640 1 --2- 172.21.15.45:0/3212458242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb22c100f30 0x7fb22c101310 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.070+0000 7fb2322b1640 1 -- 172.21.15.45:0/3212458242 >> 172.21.15.45:0/3212458242 conn(0x7fb22c0fc7d0 msgr2=0x7fb22c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.070+0000 7fb2322b1640 1 -- 172.21.15.45:0/3212458242 shutdown_connections 2024-01-31T05:33:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.070+0000 7fb2322b1640 1 -- 172.21.15.45:0/3212458242 wait complete. 2024-01-31T05:33:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb2322b1640 1 Processor -- start 2024-01-31T05:33:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb2322b1640 1 -- start start 2024-01-31T05:33:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb2322b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb22c100f30 0x7fb22c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb2322b1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22c106000 0x7fb22c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb2322b1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb22c19bb00 0x7fb22c19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb2322b1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c19c4f0 con 0x7fb22c19bb00 2024-01-31T05:33:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb2322b1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c19c660 con 0x7fb22c100f30 2024-01-31T05:33:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb2322b1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c19c7d0 con 0x7fb22c106000 2024-01-31T05:33:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb23084e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb22c19bb00 0x7fb22c19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb22b7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22c106000 0x7fb22c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb23084e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb22c19bb00 0x7fb22c19bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49424/0 (socket says 172.21.15.45:49424) 2024-01-31T05:33:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb23084e640 1 -- 172.21.15.45:0/2017899011 learned_addr learned my addr 172.21.15.45:0/2017899011 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb22bfff640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb22c100f30 0x7fb22c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb23084e640 1 -- 172.21.15.45:0/2017899011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22c106000 msgr2=0x7fb22c197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb23084e640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22c106000 0x7fb22c197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb23084e640 1 -- 172.21.15.45:0/2017899011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb22c100f30 msgr2=0x7fb22c197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.071+0000 7fb23084e640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb22c100f30 0x7fb22c197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.072+0000 7fb23084e640 1 -- 172.21.15.45:0/2017899011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c009560 con 0x7fb22c19bb00 2024-01-31T05:33:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.072+0000 7fb22b7fe640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22c106000 0x7fb22c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:33:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.072+0000 7fb22bfff640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb22c100f30 0x7fb22c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:33:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.072+0000 7fb23084e640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb22c19bb00 0x7fb22c19bfb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb224009fd0 tx=0x7fb22400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.072+0000 7fb2297fa640 1 -- 172.21.15.45:0/2017899011 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22400bea0 con 0x7fb22c19bb00 2024-01-31T05:33:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.072+0000 7fb2297fa640 1 -- 172.21.15.45:0/2017899011 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb224009940 con 0x7fb22c19bb00 2024-01-31T05:33:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.072+0000 7fb2322b1640 1 -- 172.21.15.45:0/2017899011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22c1a3db0 con 0x7fb22c19bb00 2024-01-31T05:33:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.072+0000 7fb2297fa640 1 -- 172.21.15.45:0/2017899011 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2240106c0 con 0x7fb22c19bb00 2024-01-31T05:33:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.072+0000 7fb2322b1640 1 -- 172.21.15.45:0/2017899011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22c1a4210 con 0x7fb22c19bb00 2024-01-31T05:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.074+0000 7fb2297fa640 1 -- 172.21.15.45:0/2017899011 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fb2240041b0 con 0x7fb22c19bb00 2024-01-31T05:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb235075740 1 Processor -- start 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb2297fa640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb2140a14f0 0x7fb2140a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb235075740 1 -- start start 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb22b7fe640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb2140a14f0 0x7fb2140a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb2297fa640 1 -- 172.21.15.45:0/2017899011 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb2240ccb80 con 0x7fb22c19bb00 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb235075740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc82a20 0x55a9abc82e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb235075740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc83340 0x55a9abc90620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb235075740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc90b60 0x55a9abc92f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb235075740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9abbf2040 con 0x55a9abc83340 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb235075740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9abbef730 con 0x55a9abc82a20 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb235075740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9abbeb020 con 0x55a9abc90b60 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb22b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc83340 0x55a9abc90620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb22b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc83340 0x55a9abc90620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49438/0 (socket says 172.21.15.45:49438) 2024-01-31T05:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb22b7fe640 1 -- 172.21.15.45:0/1455898092 learned_addr learned my addr 172.21.15.45:0/1455898092 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb23084e640 1 --2- 172.21.15.45:0/1455898092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc90b60 0x55a9abc92f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb22bfff640 1 --2- 172.21.15.45:0/1455898092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc82a20 0x55a9abc82e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb23084e640 1 -- 172.21.15.45:0/1455898092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc82a20 msgr2=0x55a9abc82e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb23084e640 1 --2- 172.21.15.45:0/1455898092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc82a20 0x55a9abc82e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb22b7fe640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb2140a14f0 0x7fb2140a39b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb22000afd0 tx=0x7fb220013c80 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb23084e640 1 -- 172.21.15.45:0/1455898092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc83340 msgr2=0x55a9abc90620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb23084e640 1 --2- 172.21.15.45:0/1455898092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc83340 0x55a9abc90620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.075+0000 7fb23084e640 1 -- 172.21.15.45:0/1455898092 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9abbe2e40 con 0x55a9abc90b60 2024-01-31T05:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.076+0000 7fb23084e640 1 --2- 172.21.15.45:0/1455898092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc90b60 0x55a9abc92f50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb2240ca270 tx=0x7fb2240993f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e86a925637b40f65 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.076+0000 7fb2127fc640 1 -- 172.21.15.45:0/1455898092 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2240a1cb0 con 0x55a9abc90b60 2024-01-31T05:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.076+0000 7fb2127fc640 1 -- 172.21.15.45:0/1455898092 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb224097d70 con 0x55a9abc90b60 2024-01-31T05:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.076+0000 7fb2127fc640 1 -- 172.21.15.45:0/1455898092 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2240a1a80 con 0x55a9abc90b60 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.076+0000 7fb235075740 1 -- 172.21.15.45:0/1455898092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc90b60 msgr2=0x55a9abc92f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.076+0000 7fb235075740 1 --2- 172.21.15.45:0/1455898092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc90b60 0x55a9abc92f50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb2240ca270 tx=0x7fb2240993f0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.076+0000 7fb235075740 1 -- 172.21.15.45:0/1455898092 shutdown_connections 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.076+0000 7fb235075740 1 --2- 172.21.15.45:0/1455898092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc83340 0x55a9abc90620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.076+0000 7fb235075740 1 --2- 172.21.15.45:0/1455898092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc90b60 0x55a9abc92f50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.076+0000 7fb235075740 1 --2- 172.21.15.45:0/1455898092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc82a20 0x55a9abc82e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.076+0000 7fb235075740 1 -- 172.21.15.45:0/1455898092 >> 172.21.15.45:0/1455898092 conn(0x55a9abc7d980 msgr2=0x55a9abc7dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb235075740 1 -- 172.21.15.45:0/1455898092 shutdown_connections 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb235075740 1 -- 172.21.15.45:0/1455898092 wait complete. 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb235075740 1 -- start start 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb235075740 1 Processor -- start 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb235075740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 0x55a9abc8db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb235075740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 0x55a9abc8e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb235075740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc90b60 0x55a9abc8e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb235075740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9abba41b0 con 0x55a9abc82a20 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb235075740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9abba3e50 con 0x55a9abc90b60 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb235075740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9abb69090 con 0x55a9abc83340 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb235075740 10 client.0 _ll_register_callbacks cb 0x55a9abb8ea90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb22bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 0x55a9abc8db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb22bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 0x55a9abc8db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49444/0 (socket says 172.21.15.45:49444) 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb22bfff640 1 -- 172.21.15.45:0/1335687936 learned_addr learned my addr 172.21.15.45:0/1335687936 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.077+0000 7fb22b7fe640 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 0x55a9abc8e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb23084e640 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc90b60 0x55a9abc8e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb22bfff640 1 -- 172.21.15.45:0/1335687936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 msgr2=0x55a9abc8e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb22bfff640 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 0x55a9abc8e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb22bfff640 1 -- 172.21.15.45:0/1335687936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc90b60 msgr2=0x55a9abc8e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb22bfff640 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc90b60 0x55a9abc8e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb22bfff640 1 -- 172.21.15.45:0/1335687936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22c1a4210 con 0x55a9abc82a20 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb22bfff640 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 0x55a9abc8db70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009900 tx=0x7fb21c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299d573b0b1f160a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb20b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb20b7fe640 1 -- 172.21.15.45:0/1335687936 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c004a70 con 0x55a9abc82a20 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb20b7fe640 1 -- 172.21.15.45:0/1335687936 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb21c046070 con 0x55a9abc82a20 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb235075740 1 -- 172.21.15.45:0/1335687936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9abbe4010 con 0x55a9abc82a20 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb20b7fe640 1 -- 172.21.15.45:0/1335687936 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb21c005000 con 0x55a9abc82a20 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb235075740 10 client.16299 fetch_fsmap learned FSMap version 29 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb235075740 1 -- 172.21.15.45:0/1335687936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a9abbe96a0 con 0x55a9abc82a20 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb20b7fe640 1 -- 172.21.15.45:0/1335687936 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9abbe4010 con 0x55a9abc82a20 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.078+0000 7fb20b7fe640 1 -- 172.21.15.45:0/1335687936 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb21c03c040 con 0x55a9abc82a20 2024-01-31T05:33:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.079+0000 7fb235075740 10 client.16299 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.079+0000 7fb235075740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.079+0000 7fb235075740 20 client.16299 populate_metadata read hostname 'smithi045' 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.079+0000 7fb235075740 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a9abca0910 0x55a9abca2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.079+0000 7fb235075740 4 client.16299 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.079+0000 7fb235075740 1 -- 172.21.15.45:0/1335687936 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9abbe3d30 con 0x55a9abca0910 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.079+0000 7fb22b7fe640 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a9abca0910 0x55a9abca2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.079+0000 7fb22b7fe640 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a9abca0910 0x55a9abca2d30 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c7fdda72b6327ce server_cookie=de2b155944646cc0 in_seq=0 out_seq=0 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.079+0000 7fb20b7fe640 10 client.16299 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb20b7fe640 1 -- 172.21.15.45:0/1335687936 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9abbe3d30 con 0x55a9abca0910 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb20b7fe640 10 client.16299 handle_command_reply: tid=0 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb235075740 1 client.16299 shutdown 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb235075740 1 -- 172.21.15.45:0/1335687936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 msgr2=0x55a9abc8db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb235075740 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 0x55a9abc8db70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009900 tx=0x7fb21c004270 comp rx=0 tx=0).stop 2024-01-31T05:33:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb235075740 1 -- 172.21.15.45:0/1335687936 shutdown_connections 2024-01-31T05:33:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb235075740 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a9abca0910 0x55a9abca2d30 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb235075740 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc90b60 0x55a9abc8e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb235075740 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 0x55a9abc8e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb235075740 1 --2- 172.21.15.45:0/1335687936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 0x55a9abc8db70 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb20b7fe640 0 client.16299 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb235075740 1 -- 172.21.15.45:0/1335687936 >> 172.21.15.45:0/1335687936 conn(0x55a9abc7d980 msgr2=0x55a9abc7f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb235075740 1 -- 172.21.15.45:0/1335687936 shutdown_connections 2024-01-31T05:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb235075740 1 -- 172.21.15.45:0/1335687936 wait complete. 2024-01-31T05:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.080+0000 7fb235075740 20 client.16299 trim_cache size 0 max 16384 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb235075740 1 Processor -- start 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb235075740 1 -- start start 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb235075740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc82a20 0x55a9abc902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb235075740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 0x55a9abc8e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb235075740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc90b60 0x55a9abc8adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb235075740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9abc7d4b0 con 0x55a9abc90b60 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb235075740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9abc79240 con 0x55a9abc82a20 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb235075740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9abc8c0c0 con 0x55a9abc83340 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb23084e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc90b60 0x55a9abc8adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb23084e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc90b60 0x55a9abc8adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49456/0 (socket says 172.21.15.45:49456) 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb23084e640 1 -- 172.21.15.45:0/2027356989 learned_addr learned my addr 172.21.15.45:0/2027356989 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb22b7fe640 1 --2- 172.21.15.45:0/2027356989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 0x55a9abc8e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.093+0000 7fb22bfff640 1 --2- 172.21.15.45:0/2027356989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc82a20 0x55a9abc902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb23084e640 1 -- 172.21.15.45:0/2027356989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 msgr2=0x55a9abc8e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb23084e640 1 --2- 172.21.15.45:0/2027356989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 0x55a9abc8e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb23084e640 1 -- 172.21.15.45:0/2027356989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc82a20 msgr2=0x55a9abc902a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb23084e640 1 --2- 172.21.15.45:0/2027356989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc82a20 0x55a9abc902a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb23084e640 1 -- 172.21.15.45:0/2027356989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9abb86030 con 0x55a9abc90b60 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb23084e640 1 --2- 172.21.15.45:0/2027356989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc90b60 0x55a9abc8adf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb224099680 tx=0x7fb2240045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ed04be29f19635 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb211ffb640 1 -- 172.21.15.45:0/2027356989 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2240b1070 con 0x55a9abc90b60 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb211ffb640 1 -- 172.21.15.45:0/2027356989 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2240a1cb0 con 0x55a9abc90b60 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb211ffb640 1 -- 172.21.15.45:0/2027356989 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2240a1a80 con 0x55a9abc90b60 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb235075740 1 -- 172.21.15.45:0/2027356989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc90b60 msgr2=0x55a9abc8adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb235075740 1 --2- 172.21.15.45:0/2027356989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc90b60 0x55a9abc8adf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb224099680 tx=0x7fb2240045b0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb235075740 1 -- 172.21.15.45:0/2027356989 shutdown_connections 2024-01-31T05:33:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb235075740 1 --2- 172.21.15.45:0/2027356989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc90b60 0x55a9abc8adf0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb235075740 1 --2- 172.21.15.45:0/2027356989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 0x55a9abc8e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.094+0000 7fb235075740 1 --2- 172.21.15.45:0/2027356989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc82a20 0x55a9abc902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb235075740 1 -- 172.21.15.45:0/2027356989 >> 172.21.15.45:0/2027356989 conn(0x55a9abc7d980 msgr2=0x55a9abc8c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb235075740 1 -- 172.21.15.45:0/2027356989 shutdown_connections 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb235075740 1 -- 172.21.15.45:0/2027356989 wait complete. 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb235075740 1 -- start start 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb235075740 1 Processor -- start 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb235075740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 0x55a9abc96110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb235075740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 0x55a9abc96650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb235075740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc90b60 0x55a9abc96b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb235075740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9abc98060 con 0x55a9abc82a20 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb235075740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9abcf6e00 con 0x55a9abc90b60 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb235075740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9abcf6f40 con 0x55a9abc83340 2024-01-31T05:33:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb235075740 10 client.0 _ll_register_callbacks cb 0x7fb224004000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:33:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb22bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 0x55a9abc96110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb22bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 0x55a9abc96110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49472/0 (socket says 172.21.15.45:49472) 2024-01-31T05:33:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb22bfff640 1 -- 172.21.15.45:0/2409674476 learned_addr learned my addr 172.21.15.45:0/2409674476 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:33:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb23084e640 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc90b60 0x55a9abc96b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.095+0000 7fb22b7fe640 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 0x55a9abc96650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb22bfff640 1 -- 172.21.15.45:0/2409674476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 msgr2=0x55a9abc96650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb22bfff640 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 0x55a9abc96650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb22bfff640 1 -- 172.21.15.45:0/2409674476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc90b60 msgr2=0x55a9abc96b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:33:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb22bfff640 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc90b60 0x55a9abc96b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb22bfff640 1 -- 172.21.15.45:0/2409674476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9abbe2e40 con 0x55a9abc82a20 2024-01-31T05:33:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb22bfff640 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 0x55a9abc96110 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009900 tx=0x7fb21c005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b9e472f3a490c8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:33:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb20b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:33:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb20b7fe640 1 -- 172.21.15.45:0/2409674476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c02cc40 con 0x55a9abc82a20 2024-01-31T05:33:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb20b7fe640 1 -- 172.21.15.45:0/2409674476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb21c046040 con 0x55a9abc82a20 2024-01-31T05:33:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb235075740 1 -- 172.21.15.45:0/2409674476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb224097d70 con 0x55a9abc82a20 2024-01-31T05:33:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb20b7fe640 1 -- 172.21.15.45:0/2409674476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb21c054480 con 0x55a9abc82a20 2024-01-31T05:33:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb235075740 10 client.16305 fetch_fsmap learned FSMap version 29 2024-01-31T05:33:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb235075740 1 -- 172.21.15.45:0/2409674476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a9abbaba50 con 0x55a9abc82a20 2024-01-31T05:33:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb20b7fe640 1 -- 172.21.15.45:0/2409674476 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb224097d70 con 0x55a9abc82a20 2024-01-31T05:33:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb20b7fe640 1 -- 172.21.15.45:0/2409674476 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb21c03c040 con 0x55a9abc82a20 2024-01-31T05:33:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb235075740 10 client.16305 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:33:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb235075740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:33:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.096+0000 7fb235075740 20 client.16305 populate_metadata read hostname 'smithi045' 2024-01-31T05:33:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.097+0000 7fb235075740 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a9abcf8540 0x55a9abcfa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:33:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.097+0000 7fb235075740 4 client.16305 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:33:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.097+0000 7fb235075740 1 -- 172.21.15.45:0/2409674476 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a9ab8ea600 con 0x55a9abcf8540 2024-01-31T05:33:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.097+0000 7fb22b7fe640 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a9abcf8540 0x55a9abcfa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:33:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.097+0000 7fb20b7fe640 10 client.16305 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.097+0000 7fb22b7fe640 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a9abcf8540 0x55a9abcfa960 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=442fb0611d0beb94 server_cookie=234d38bcb85b837f in_seq=0 out_seq=0 2024-01-31T05:33:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.108+0000 7fb20b7fe640 1 -- 172.21.15.45:0/2409674476 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a9ab8ea600 con 0x55a9abcf8540 2024-01-31T05:33:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.108+0000 7fb20b7fe640 10 client.16305 handle_command_reply: tid=0 2024-01-31T05:33:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.108+0000 7fb235075740 1 client.16305 shutdown 2024-01-31T05:33:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb235075740 1 -- 172.21.15.45:0/2409674476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 msgr2=0x55a9abc96110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb235075740 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 0x55a9abc96110 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009900 tx=0x7fb21c005230 comp rx=0 tx=0).stop 2024-01-31T05:33:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb235075740 1 -- 172.21.15.45:0/2409674476 shutdown_connections 2024-01-31T05:33:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb235075740 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a9abcf8540 0x55a9abcfa960 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb235075740 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9abc90b60 0x55a9abc96b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb235075740 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9abc83340 0x55a9abc96650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb20b7fe640 0 client.16305 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:33:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb235075740 1 --2- 172.21.15.45:0/2409674476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9abc82a20 0x55a9abc96110 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb235075740 1 -- 172.21.15.45:0/2409674476 >> 172.21.15.45:0/2409674476 conn(0x55a9abc7d980 msgr2=0x55a9abc7c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb235075740 1 -- 172.21.15.45:0/2409674476 shutdown_connections 2024-01-31T05:33:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb235075740 1 -- 172.21.15.45:0/2409674476 wait complete. 2024-01-31T05:33:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb235075740 20 client.16305 trim_cache size 0 max 16384 2024-01-31T05:33:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:33:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:33:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:33:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:33:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb2322b1640 1 -- 172.21.15.45:0/2017899011 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb2140a14f0 msgr2=0x7fb2140a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb2322b1640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb2140a14f0 0x7fb2140a39b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb22000afd0 tx=0x7fb220013c80 comp rx=0 tx=0).stop 2024-01-31T05:33:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb2322b1640 1 -- 172.21.15.45:0/2017899011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb22c19bb00 msgr2=0x7fb22c19bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:33:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.109+0000 7fb2322b1640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb22c19bb00 0x7fb22c19bfb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb224009fd0 tx=0x7fb22400b260 comp rx=0 tx=0).stop 2024-01-31T05:33:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.110+0000 7fb2322b1640 1 -- 172.21.15.45:0/2017899011 shutdown_connections 2024-01-31T05:33:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.110+0000 7fb2322b1640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb2140a14f0 0x7fb2140a39b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.110+0000 7fb2322b1640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb22c19bb00 0x7fb22c19bfb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.110+0000 7fb2322b1640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22c106000 0x7fb22c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.110+0000 7fb2322b1640 1 --2- 172.21.15.45:0/2017899011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb22c100f30 0x7fb22c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:33:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.110+0000 7fb2322b1640 1 -- 172.21.15.45:0/2017899011 >> 172.21.15.45:0/2017899011 conn(0x7fb22c0fc7d0 msgr2=0x7fb22c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:33:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.110+0000 7fb2322b1640 1 -- 172.21.15.45:0/2017899011 shutdown_connections 2024-01-31T05:33:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:33:50.110+0000 7fb2322b1640 1 -- 172.21.15.45:0/2017899011 wait complete. 2024-01-31T05:33:50.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1184196 10.59 MB/sec execute 470 sec latency 86.370 ms 2024-01-31T05:33:50.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:50 smithi045 ceph-mon[31585]: pgmap v352: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-01-31T05:33:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:50 smithi156 ceph-mon[35720]: pgmap v352: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-01-31T05:33:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:50 smithi186 ceph-mon[35659]: pgmap v352: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-01-31T05:33:51.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1186375 10.60 MB/sec execute 471 sec latency 17.014 ms 2024-01-31T05:33:52.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1187528 10.58 MB/sec execute 472 sec latency 63.386 ms 2024-01-31T05:33:52.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:52 smithi045 ceph-mon[31585]: pgmap v353: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-01-31T05:33:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:52 smithi156 ceph-mon[35720]: pgmap v353: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-01-31T05:33:53.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:52 smithi186 ceph-mon[35659]: pgmap v353: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-01-31T05:33:53.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1189908 10.59 MB/sec execute 473 sec latency 13.442 ms 2024-01-31T05:33:54.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1191098 10.58 MB/sec execute 474 sec latency 33.119 ms 2024-01-31T05:33:54.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:54 smithi045 ceph-mon[31585]: pgmap v354: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 251 op/s 2024-01-31T05:33:54.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:54 smithi156 ceph-mon[35720]: pgmap v354: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 251 op/s 2024-01-31T05:33:55.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:54 smithi186 ceph-mon[35659]: pgmap v354: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 251 op/s 2024-01-31T05:33:55.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1193364 10.59 MB/sec execute 475 sec latency 57.256 ms 2024-01-31T05:33:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:55 smithi156 ceph-mon[35720]: pgmap v355: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 162 op/s 2024-01-31T05:33:56.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:55 smithi186 ceph-mon[35659]: pgmap v355: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 162 op/s 2024-01-31T05:33:56.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:55 smithi045 ceph-mon[31585]: pgmap v355: 97 pgs: 97 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 162 op/s 2024-01-31T05:33:56.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1194315 10.57 MB/sec execute 476 sec latency 19.655 ms 2024-01-31T05:33:57.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1196908 10.58 MB/sec execute 477 sec latency 20.385 ms 2024-01-31T05:33:58.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1197909 10.57 MB/sec execute 478 sec latency 26.902 ms 2024-01-31T05:33:58.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:58 smithi045 ceph-mon[31585]: pgmap v356: 97 pgs: 97 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.0 MiB/s wr, 246 op/s 2024-01-31T05:33:58.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:58 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:33:58.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:58 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:33:58.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:58 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:33:58.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:58 smithi156 ceph-mon[35720]: pgmap v356: 97 pgs: 97 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.0 MiB/s wr, 246 op/s 2024-01-31T05:33:58.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:58 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:33:58.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:58 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:33:58.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:58 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:33:59.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:58 smithi186 ceph-mon[35659]: pgmap v356: 97 pgs: 97 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.0 MiB/s wr, 246 op/s 2024-01-31T05:33:59.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:58 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:33:59.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:58 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:33:59.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:58 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:33:59.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1200480 10.58 MB/sec execute 479 sec latency 15.639 ms 2024-01-31T05:33:59.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:59 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:33:59.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:59 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:33:59.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:33:59 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:33:59.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:59 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:33:59.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:59 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:33:59.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:33:59 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:34:00.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:59 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:34:00.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:59 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:34:00.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:33:59 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:34:00.799 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1201416 10.56 MB/sec execute 480 sec latency 80.985 ms 2024-01-31T05:34:00.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:00 smithi045 ceph-mon[31585]: pgmap v357: 97 pgs: 97 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 164 op/s 2024-01-31T05:34:00.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:00 smithi156 ceph-mon[35720]: pgmap v357: 97 pgs: 97 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 164 op/s 2024-01-31T05:34:01.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:00 smithi186 ceph-mon[35659]: pgmap v357: 97 pgs: 97 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 164 op/s 2024-01-31T05:34:01.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1202971 10.56 MB/sec execute 481 sec latency 30.971 ms 2024-01-31T05:34:02.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1204444 10.56 MB/sec execute 482 sec latency 21.751 ms 2024-01-31T05:34:02.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:02 smithi045 ceph-mon[31585]: pgmap v358: 97 pgs: 97 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 164 op/s 2024-01-31T05:34:02.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:02 smithi156 ceph-mon[35720]: pgmap v358: 97 pgs: 97 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 164 op/s 2024-01-31T05:34:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:02 smithi186 ceph-mon[35659]: pgmap v358: 97 pgs: 97 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 164 op/s 2024-01-31T05:34:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.198+0000 7f6a50d16640 1 -- 172.21.15.45:0/692431109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a4c100f10 msgr2=0x7f6a4c1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.198+0000 7f6a50d16640 1 --2- 172.21.15.45:0/692431109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a4c100f10 0x7f6a4c1012f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009930 tx=0x7f6a3c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:34:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.198+0000 7f6a50d16640 1 -- 172.21.15.45:0/692431109 shutdown_connections 2024-01-31T05:34:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.198+0000 7f6a50d16640 1 --2- 172.21.15.45:0/692431109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a4c105fe0 0x7f6a4c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.198+0000 7f6a50d16640 1 --2- 172.21.15.45:0/692431109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4c1018c0 0x7f6a4c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.198+0000 7f6a50d16640 1 --2- 172.21.15.45:0/692431109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a4c100f10 0x7f6a4c1012f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.198+0000 7f6a50d16640 1 -- 172.21.15.45:0/692431109 >> 172.21.15.45:0/692431109 conn(0x7f6a4c0fc7b0 msgr2=0x7f6a4c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.198+0000 7f6a50d16640 1 -- 172.21.15.45:0/692431109 shutdown_connections 2024-01-31T05:34:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.198+0000 7f6a50d16640 1 -- 172.21.15.45:0/692431109 wait complete. 2024-01-31T05:34:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a50d16640 1 Processor -- start 2024-01-31T05:34:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a50d16640 1 -- start start 2024-01-31T05:34:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a50d16640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4c100f10 0x7f6a4c072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a50d16640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a4c105fe0 0x7f6a4c073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a50d16640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a4c073a70 0x7f6a4c1a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a50d16640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c1a5a00 con 0x7f6a4c105fe0 2024-01-31T05:34:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a50d16640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c1a5b70 con 0x7f6a4c100f10 2024-01-31T05:34:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a50d16640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c1a5d30 con 0x7f6a4c073a70 2024-01-31T05:34:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a4c105fe0 0x7f6a4c073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a4c105fe0 0x7f6a4c073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54684/0 (socket says 172.21.15.45:54684) 2024-01-31T05:34:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a41d9b640 1 -- 172.21.15.45:0/1460669512 learned_addr learned my addr 172.21.15.45:0/1460669512 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a4ad9d640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a4c073a70 0x7f6a4c1a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.200+0000 7f6a4a59c640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4c100f10 0x7f6a4c072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.201+0000 7f6a41d9b640 1 -- 172.21.15.45:0/1460669512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a4c073a70 msgr2=0x7f6a4c1a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.201+0000 7f6a41d9b640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a4c073a70 0x7f6a4c1a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.201+0000 7f6a41d9b640 1 -- 172.21.15.45:0/1460669512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4c100f10 msgr2=0x7f6a4c072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.201+0000 7f6a41d9b640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4c100f10 0x7f6a4c072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.201+0000 7f6a41d9b640 1 -- 172.21.15.45:0/1460669512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c009560 con 0x7f6a4c105fe0 2024-01-31T05:34:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.201+0000 7f6a41d9b640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a4c105fe0 0x7f6a4c073530 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400e9c0 tx=0x7f6a3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.201+0000 7f6a43fff640 1 -- 172.21.15.45:0/1460669512 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34009700 con 0x7f6a4c105fe0 2024-01-31T05:34:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.201+0000 7f6a43fff640 1 -- 172.21.15.45:0/1460669512 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a3400cd80 con 0x7f6a4c105fe0 2024-01-31T05:34:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.201+0000 7f6a50d16640 1 -- 172.21.15.45:0/1460669512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4c1a5fb0 con 0x7f6a4c105fe0 2024-01-31T05:34:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.201+0000 7f6a43fff640 1 -- 172.21.15.45:0/1460669512 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34010430 con 0x7f6a4c105fe0 2024-01-31T05:34:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.201+0000 7f6a50d16640 1 -- 172.21.15.45:0/1460669512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4c1a6490 con 0x7f6a4c105fe0 2024-01-31T05:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a43fff640 1 -- 172.21.15.45:0/1460669512 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f6a34004f90 con 0x7f6a4c105fe0 2024-01-31T05:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a53bb1740 1 Processor -- start 2024-01-31T05:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a53bb1740 1 -- start start 2024-01-31T05:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a43fff640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f6a380a1520 0x7f6a380a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a53bb1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee170a060 0x559ee170a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a53bb1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee1713210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a53bb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee1713750 0x559ee1715b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a4ad9d640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f6a380a1520 0x7f6a380a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a53bb1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee1679040 con 0x559ee170a060 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a53bb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee1676730 con 0x559ee1713750 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a53bb1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee1672020 con 0x559ee170a980 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a43fff640 1 -- 172.21.15.45:0/1460669512 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6a34014070 con 0x7f6a4c105fe0 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee1713210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a4a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee170a060 0x559ee170a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee1713210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:38772/0 (socket says 172.21.15.45:38772) 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a41d9b640 1 -- 172.21.15.45:0/3232304503 learned_addr learned my addr 172.21.15.45:0/3232304503 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.203+0000 7f6a4ad9d640 1 --2- 172.21.15.45:0/3232304503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee1713750 0x559ee1715b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.205+0000 7f6a41d9b640 1 -- 172.21.15.45:0/3232304503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee1713750 msgr2=0x559ee1715b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.205+0000 7f6a41d9b640 1 --2- 172.21.15.45:0/3232304503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee1713750 0x559ee1715b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.205+0000 7f6a41d9b640 1 -- 172.21.15.45:0/3232304503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee170a060 msgr2=0x559ee170a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.205+0000 7f6a41d9b640 1 --2- 172.21.15.45:0/3232304503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee170a060 0x559ee170a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.205+0000 7f6a41d9b640 1 -- 172.21.15.45:0/3232304503 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ee1669e40 con 0x559ee170a980 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.205+0000 7f6a4ad9d640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f6a380a1520 0x7f6a380a39e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009fd0 tx=0x7f6a44013b00 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.205+0000 7f6a41d9b640 1 --2- 172.21.15.45:0/3232304503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee1713210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6a340a1f00 tx=0x7f6a340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf5de0025ac6d9b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.205+0000 7f6a40d99640 1 -- 172.21.15.45:0/3232304503 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3409e1f0 con 0x559ee170a980 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.205+0000 7f6a40d99640 1 -- 172.21.15.45:0/3232304503 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a340cc030 con 0x559ee170a980 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.205+0000 7f6a40d99640 1 -- 172.21.15.45:0/3232304503 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34005240 con 0x559ee170a980 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.206+0000 7f6a53bb1740 1 -- 172.21.15.45:0/3232304503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 msgr2=0x559ee1713210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.206+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/3232304503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee1713210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6a340a1f00 tx=0x7f6a340046c0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.206+0000 7f6a53bb1740 1 -- 172.21.15.45:0/3232304503 shutdown_connections 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.206+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/3232304503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee1713750 0x559ee1715b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.206+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/3232304503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee1713210 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.206+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/3232304503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee170a060 0x559ee170a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.206+0000 7f6a53bb1740 1 -- 172.21.15.45:0/3232304503 >> 172.21.15.45:0/3232304503 conn(0x559ee1704980 msgr2=0x559ee1704d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.206+0000 7f6a53bb1740 1 -- 172.21.15.45:0/3232304503 shutdown_connections 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.206+0000 7f6a53bb1740 1 -- 172.21.15.45:0/3232304503 wait complete. 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.206+0000 7f6a53bb1740 1 -- start start 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.206+0000 7f6a53bb1740 1 Processor -- start 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a53bb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee170a060 0x559ee1710800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a53bb1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee1710d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a53bb1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 0x559ee1711280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a53bb1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee162b1b0 con 0x559ee1713750 2024-01-31T05:34:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a53bb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee162ae50 con 0x559ee170a060 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a53bb1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee15f0090 con 0x559ee170a980 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a53bb1740 10 client.0 _ll_register_callbacks cb 0x559ee1615a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a4a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee170a060 0x559ee1710800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a4ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 0x559ee1711280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee1710d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a4a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee170a060 0x559ee1710800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:55914/0 (socket says 172.21.15.45:55914) 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a4a59c640 1 -- 172.21.15.45:0/1907265728 learned_addr learned my addr 172.21.15.45:0/1907265728 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a4ad9d640 1 -- 172.21.15.45:0/1907265728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 msgr2=0x559ee1710d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a4ad9d640 1 --2- 172.21.15.45:0/1907265728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee1710d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a4ad9d640 1 -- 172.21.15.45:0/1907265728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee170a060 msgr2=0x559ee1710800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a4ad9d640 1 --2- 172.21.15.45:0/1907265728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee170a060 0x559ee1710800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a4ad9d640 1 -- 172.21.15.45:0/1907265728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4c1a6490 con 0x559ee1713750 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a41d9b640 1 --2- 172.21.15.45:0/1907265728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee1710d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a4ad9d640 1 --2- 172.21.15.45:0/1907265728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 0x559ee1711280 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x559ee1705b10 tx=0x7f6a44004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdcd0fc96c8af0d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a31ffb640 1 -- 172.21.15.45:0/1907265728 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44010bd0 con 0x559ee1713750 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a31ffb640 1 -- 172.21.15.45:0/1907265728 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6a44010d30 con 0x559ee1713750 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.207+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1907265728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ee166b010 con 0x559ee1713750 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.208+0000 7f6a31ffb640 1 -- 172.21.15.45:0/1907265728 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a44020400 con 0x559ee1713750 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.208+0000 7f6a53bb1740 10 client.16317 fetch_fsmap learned FSMap version 29 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.208+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1907265728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559ee16706a0 con 0x559ee1713750 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.208+0000 7f6a31ffb640 1 -- 172.21.15.45:0/1907265728 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ee166b010 con 0x559ee1713750 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.208+0000 7f6a31ffb640 1 -- 172.21.15.45:0/1907265728 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6a4400e570 con 0x559ee1713750 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.208+0000 7f6a53bb1740 10 client.16317 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.208+0000 7f6a53bb1740 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.208+0000 7f6a53bb1740 20 client.16317 populate_metadata read hostname 'smithi045' 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.208+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/1907265728 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559ee17235e0 0x559ee1725a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.208+0000 7f6a53bb1740 4 client.16317 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.208+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1907265728 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ee166ad30 con 0x559ee17235e0 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.208+0000 7f6a4a59c640 1 --2- 172.21.15.45:0/1907265728 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559ee17235e0 0x559ee1725a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.209+0000 7f6a4a59c640 1 --2- 172.21.15.45:0/1907265728 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559ee17235e0 0x559ee1725a00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c494faf10fd468 server_cookie=8b77d41ea2345ac in_seq=0 out_seq=0 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.209+0000 7f6a31ffb640 10 client.16317 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.209+0000 7f6a31ffb640 1 -- 172.21.15.45:0/1907265728 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559ee166ad30 con 0x559ee17235e0 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.209+0000 7f6a31ffb640 10 client.16317 handle_command_reply: tid=0 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.209+0000 7f6a53bb1740 1 client.16317 shutdown 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.209+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1907265728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 msgr2=0x559ee1711280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.209+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/1907265728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 0x559ee1711280 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x559ee1705b10 tx=0x7f6a44004ab0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.210+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1907265728 shutdown_connections 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.210+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/1907265728 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559ee17235e0 0x559ee1725a00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.210+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/1907265728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 0x559ee1711280 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.210+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/1907265728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee1710d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.210+0000 7f6a31ffb640 0 client.16317 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.210+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/1907265728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee170a060 0x559ee1710800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.210+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1907265728 >> 172.21.15.45:0/1907265728 conn(0x559ee1704980 msgr2=0x559ee17066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.210+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1907265728 shutdown_connections 2024-01-31T05:34:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.210+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1907265728 wait complete. 2024-01-31T05:34:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.210+0000 7f6a53bb1740 20 client.16317 trim_cache size 0 max 16384 2024-01-31T05:34:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a53bb1740 1 Processor -- start 2024-01-31T05:34:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a53bb1740 1 -- start start 2024-01-31T05:34:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a53bb1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee170a060 0x559ee1712e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a53bb1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee17110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a53bb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee1713750 0x559ee170f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a53bb1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee170b6b0 con 0x559ee170a060 2024-01-31T05:34:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a53bb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee17200b0 con 0x559ee1713750 2024-01-31T05:34:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a53bb1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee1705980 con 0x559ee170a980 2024-01-31T05:34:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a4a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee170a060 0x559ee1712e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a4a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee170a060 0x559ee1712e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54718/0 (socket says 172.21.15.45:54718) 2024-01-31T05:34:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a4a59c640 1 -- 172.21.15.45:0/1348077177 learned_addr learned my addr 172.21.15.45:0/1348077177 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a41d9b640 1 --2- 172.21.15.45:0/1348077177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee17110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a4ad9d640 1 --2- 172.21.15.45:0/1348077177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee1713750 0x559ee170f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a4a59c640 1 -- 172.21.15.45:0/1348077177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 msgr2=0x559ee17110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a4a59c640 1 --2- 172.21.15.45:0/1348077177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee17110a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a4a59c640 1 -- 172.21.15.45:0/1348077177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee1713750 msgr2=0x559ee170f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a4a59c640 1 --2- 172.21.15.45:0/1348077177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee1713750 0x559ee170f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a4a59c640 1 -- 172.21.15.45:0/1348077177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ee160d030 con 0x559ee170a060 2024-01-31T05:34:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.224+0000 7f6a4a59c640 1 --2- 172.21.15.45:0/1348077177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee170a060 0x559ee1712e90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009930 tx=0x7f6a3c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840375eeee5b7565 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.225+0000 7f6a33fff640 1 -- 172.21.15.45:0/1348077177 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c045040 con 0x559ee170a060 2024-01-31T05:34:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.225+0000 7f6a33fff640 1 -- 172.21.15.45:0/1348077177 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a3c047040 con 0x559ee170a060 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.225+0000 7f6a33fff640 1 -- 172.21.15.45:0/1348077177 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c04c430 con 0x559ee170a060 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.225+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1348077177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee170a060 msgr2=0x559ee1712e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.225+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/1348077177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee170a060 0x559ee1712e90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009930 tx=0x7f6a3c005b50 comp rx=0 tx=0).stop 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.225+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1348077177 shutdown_connections 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.225+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/1348077177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee1713750 0x559ee170f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.225+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/1348077177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a980 0x559ee17110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.225+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/1348077177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee170a060 0x559ee1712e90 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.225+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1348077177 >> 172.21.15.45:0/1348077177 conn(0x559ee1704980 msgr2=0x559ee170bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.225+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1348077177 shutdown_connections 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.225+0000 7f6a53bb1740 1 -- 172.21.15.45:0/1348077177 wait complete. 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a53bb1740 1 -- start start 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a53bb1740 1 Processor -- start 2024-01-31T05:34:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a53bb1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a060 0x559ee171a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a53bb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee170a980 0x559ee171a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a53bb1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 0x559ee1718640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a53bb1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee1718d10 con 0x559ee1713750 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a53bb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee1718e80 con 0x559ee170a980 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a53bb1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee1719040 con 0x559ee170a060 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a53bb1740 10 client.0 _ll_register_callbacks cb 0x7f6a340963e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a4a59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a060 0x559ee171a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a4ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 0x559ee1718640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a4a59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a060 0x559ee171a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:38804/0 (socket says 172.21.15.45:38804) 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a4ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 0x559ee1718640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54728/0 (socket says 172.21.15.45:54728) 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a4a59c640 1 -- 172.21.15.45:0/819519676 learned_addr learned my addr 172.21.15.45:0/819519676 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a41d9b640 1 --2- 172.21.15.45:0/819519676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee170a980 0x559ee171a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a4ad9d640 1 -- 172.21.15.45:0/819519676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a060 msgr2=0x559ee171a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a4ad9d640 1 --2- 172.21.15.45:0/819519676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a060 0x559ee171a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a4ad9d640 1 -- 172.21.15.45:0/819519676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee170a980 msgr2=0x559ee171a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a4ad9d640 1 --2- 172.21.15.45:0/819519676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee170a980 0x559ee171a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.226+0000 7f6a4ad9d640 1 -- 172.21.15.45:0/819519676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4c10e3c0 con 0x559ee1713750 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a4ad9d640 1 --2- 172.21.15.45:0/819519676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 0x559ee1718640 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009ec0 tx=0x7f6a44010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2680f99d2141b4d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a31ffb640 1 -- 172.21.15.45:0/819519676 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4400f760 con 0x559ee1713750 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a53bb1740 1 -- 172.21.15.45:0/819519676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a340cc030 con 0x559ee1713750 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a31ffb640 1 -- 172.21.15.45:0/819519676 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6a44013460 con 0x559ee1713750 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a31ffb640 1 -- 172.21.15.45:0/819519676 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a4400ec40 con 0x559ee1713750 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a53bb1740 10 client.16323 fetch_fsmap learned FSMap version 29 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a53bb1740 1 -- 172.21.15.45:0/819519676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559ee1632a50 con 0x559ee1713750 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a31ffb640 1 -- 172.21.15.45:0/819519676 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a44041020 con 0x559ee1713750 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a31ffb640 1 -- 172.21.15.45:0/819519676 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6a44004040 con 0x559ee1713750 2024-01-31T05:34:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a53bb1740 10 client.16323 fetch_fsmap finished waiting for FSMap version 29 2024-01-31T05:34:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a53bb1740 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:34:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a53bb1740 20 client.16323 populate_metadata read hostname 'smithi045' 2024-01-31T05:34:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/819519676 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559ee177ac20 0x559ee177d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a53bb1740 4 client.16323 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:34:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a53bb1740 1 -- 172.21.15.45:0/819519676 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559ee1371600 con 0x559ee177ac20 2024-01-31T05:34:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.227+0000 7f6a4a59c640 1 --2- 172.21.15.45:0/819519676 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559ee177ac20 0x559ee177d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.228+0000 7f6a31ffb640 10 client.16323 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:03.228+0000 7f6a4a59c640 1 --2- 172.21.15.45:0/819519676 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559ee177ac20 0x559ee177d040 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb80ca565af17b9 server_cookie=acd17b65a6b55a4c in_seq=0 out_seq=0 2024-01-31T05:34:03.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1206499 10.55 MB/sec execute 483 sec latency 20.617 ms 2024-01-31T05:34:04.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1208030 10.55 MB/sec execute 484 sec latency 21.829 ms 2024-01-31T05:34:04.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:04 smithi045 ceph-mon[31585]: pgmap v359: 97 pgs: 97 active+clean; 567 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-01-31T05:34:04.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:04 smithi156 ceph-mon[35720]: pgmap v359: 97 pgs: 97 active+clean; 567 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-01-31T05:34:05.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a31ffb640 1 -- 172.21.15.45:0/819519676 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559ee1371600 con 0x559ee177ac20 2024-01-31T05:34:05.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a31ffb640 10 client.16323 handle_command_reply: tid=0 2024-01-31T05:34:05.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a53bb1740 1 client.16323 shutdown 2024-01-31T05:34:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a53bb1740 1 -- 172.21.15.45:0/819519676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 msgr2=0x559ee1718640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/819519676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 0x559ee1718640 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009ec0 tx=0x7f6a44010800 comp rx=0 tx=0).stop 2024-01-31T05:34:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a53bb1740 1 -- 172.21.15.45:0/819519676 shutdown_connections 2024-01-31T05:34:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/819519676 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559ee177ac20 0x559ee177d040 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/819519676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee1713750 0x559ee1718640 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/819519676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ee170a980 0x559ee171a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a31ffb640 0 client.16323 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a53bb1740 1 --2- 172.21.15.45:0/819519676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559ee170a060 0x559ee171a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a53bb1740 1 -- 172.21.15.45:0/819519676 >> 172.21.15.45:0/819519676 conn(0x559ee1704980 msgr2=0x559ee170f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a53bb1740 1 -- 172.21.15.45:0/819519676 shutdown_connections 2024-01-31T05:34:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a53bb1740 1 -- 172.21.15.45:0/819519676 wait complete. 2024-01-31T05:34:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.020+0000 7f6a53bb1740 20 client.16323 trim_cache size 0 max 16384 2024-01-31T05:34:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:34:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:34:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:34:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:34:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.021+0000 7f6a50d16640 1 -- 172.21.15.45:0/1460669512 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f6a380a1520 msgr2=0x7f6a380a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.021+0000 7f6a50d16640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f6a380a1520 0x7f6a380a39e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009fd0 tx=0x7f6a44013b00 comp rx=0 tx=0).stop 2024-01-31T05:34:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.021+0000 7f6a50d16640 1 -- 172.21.15.45:0/1460669512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a4c105fe0 msgr2=0x7f6a4c073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.021+0000 7f6a50d16640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a4c105fe0 0x7f6a4c073530 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400e9c0 tx=0x7f6a3400ee80 comp rx=0 tx=0).stop 2024-01-31T05:34:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.021+0000 7f6a50d16640 1 -- 172.21.15.45:0/1460669512 shutdown_connections 2024-01-31T05:34:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.021+0000 7f6a50d16640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f6a380a1520 0x7f6a380a39e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.021+0000 7f6a50d16640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a4c105fe0 0x7f6a4c073530 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.021+0000 7f6a50d16640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a4c073a70 0x7f6a4c1a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.021+0000 7f6a50d16640 1 --2- 172.21.15.45:0/1460669512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4c100f10 0x7f6a4c072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.021+0000 7f6a50d16640 1 -- 172.21.15.45:0/1460669512 >> 172.21.15.45:0/1460669512 conn(0x7f6a4c0fc7b0 msgr2=0x7f6a4c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.021+0000 7f6a50d16640 1 -- 172.21.15.45:0/1460669512 shutdown_connections 2024-01-31T05:34:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:05.021+0000 7f6a50d16640 1 -- 172.21.15.45:0/1460669512 wait complete. 2024-01-31T05:34:05.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:04 smithi186 ceph-mon[35659]: pgmap v359: 97 pgs: 97 active+clean; 567 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-01-31T05:34:05.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1209922 10.55 MB/sec execute 485 sec latency 55.782 ms 2024-01-31T05:34:06.147 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: *** Caught signal (Segmentation fault) ** 2024-01-31T05:34:06.148 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: in thread 7f0099195700 thread_name:md_log_replay 2024-01-31T05:34:06.148 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:34:06.148 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f00a9d55d20] 2024-01-31T05:34:06.148 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 2: /usr/bin/ceph-mds(+0x5040a2) [0x562ca3a100a2] 2024-01-31T05:34:06.148 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x562ca3a122f3] 2024-01-31T05:34:06.148 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 4: (MDLog::_replay_thread()+0x647) [0x562ca3a6e697] 2024-01-31T05:34:06.148 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 5: (MDLog::ReplayThread::entry()+0x11) [0x562ca3a77b31] 2024-01-31T05:34:06.148 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f00a9d4b1ca] 2024-01-31T05:34:06.149 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 7: clone() 2024-01-31T05:34:06.149 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 2024-01-31T05:34:05.762+0000 7f0099195700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:34:06.149 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: in thread 7f0099195700 thread_name:md_log_replay 2024-01-31T05:34:06.149 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 2024-01-31T05:34:06.149 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:34:06.149 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f00a9d55d20] 2024-01-31T05:34:06.149 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 2: /usr/bin/ceph-mds(+0x5040a2) [0x562ca3a100a2] 2024-01-31T05:34:06.149 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x562ca3a122f3] 2024-01-31T05:34:06.150 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 4: (MDLog::_replay_thread()+0x647) [0x562ca3a6e697] 2024-01-31T05:34:06.150 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 5: (MDLog::ReplayThread::entry()+0x11) [0x562ca3a77b31] 2024-01-31T05:34:06.150 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f00a9d4b1ca] 2024-01-31T05:34:06.150 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 7: clone() 2024-01-31T05:34:06.150 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:34:06.150 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 2024-01-31T05:34:06.150 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 0> 2024-01-31T05:34:05.762+0000 7f0099195700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:34:06.150 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: in thread 7f0099195700 thread_name:md_log_replay 2024-01-31T05:34:06.150 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 2024-01-31T05:34:06.150 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:34:06.151 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f00a9d55d20] 2024-01-31T05:34:06.151 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 2: /usr/bin/ceph-mds(+0x5040a2) [0x562ca3a100a2] 2024-01-31T05:34:06.151 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x562ca3a122f3] 2024-01-31T05:34:06.151 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 4: (MDLog::_replay_thread()+0x647) [0x562ca3a6e697] 2024-01-31T05:34:06.151 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 5: (MDLog::ReplayThread::entry()+0x11) [0x562ca3a77b31] 2024-01-31T05:34:06.151 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f00a9d4b1ca] 2024-01-31T05:34:06.151 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 7: clone() 2024-01-31T05:34:06.151 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:34:06.151 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 2024-01-31T05:34:06.152 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 0> 2024-01-31T05:34:05.762+0000 7f0099195700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:34:06.152 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: in thread 7f0099195700 thread_name:md_log_replay 2024-01-31T05:34:06.152 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 2024-01-31T05:34:06.152 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:34:06.152 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f00a9d55d20] 2024-01-31T05:34:06.152 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 2: /usr/bin/ceph-mds(+0x5040a2) [0x562ca3a100a2] 2024-01-31T05:34:06.152 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x562ca3a122f3] 2024-01-31T05:34:06.152 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 4: (MDLog::_replay_thread()+0x647) [0x562ca3a6e697] 2024-01-31T05:34:06.153 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 5: (MDLog::ReplayThread::entry()+0x11) [0x562ca3a77b31] 2024-01-31T05:34:06.153 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f00a9d4b1ca] 2024-01-31T05:34:06.153 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 7: clone() 2024-01-31T05:34:06.153 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:34:06.153 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[71874]: 2024-01-31T05:34:06.153 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:05 smithi156 podman[72657]: 2024-01-31 05:34:05.816195605 +0000 UTC m=+0.020663068 container died fa207d71408a5ac1576c418179239a826ffc4c7b547db49447453d8472200699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-01-31T05:34:06.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1211470 10.55 MB/sec execute 486 sec latency 15.888 ms 2024-01-31T05:34:06.826 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:06 smithi156 ceph-mon[35720]: pgmap v360: 97 pgs: 97 active+clean; 567 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 196 op/s 2024-01-31T05:34:06.826 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:06 smithi156 ceph-mon[35720]: osdmap e105: 12 total, 12 up, 12 in 2024-01-31T05:34:06.827 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:06 smithi156 podman[72657]: 2024-01-31 05:34:06.574070482 +0000 UTC m=+0.778537949 container remove fa207d71408a5ac1576c418179239a826ffc4c7b547db49447453d8472200699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-01-31T05:34:06.827 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:06 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-01-31T05:34:06.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:06 smithi045 ceph-mon[31585]: pgmap v360: 97 pgs: 97 active+clean; 567 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 196 op/s 2024-01-31T05:34:06.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:06 smithi045 ceph-mon[31585]: osdmap e105: 12 total, 12 up, 12 in 2024-01-31T05:34:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:06 smithi186 ceph-mon[35659]: pgmap v360: 97 pgs: 97 active+clean; 567 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 196 op/s 2024-01-31T05:34:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:06 smithi186 ceph-mon[35659]: osdmap e105: 12 total, 12 up, 12 in 2024-01-31T05:34:07.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:06 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-01-31T05:34:07.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1212722 10.54 MB/sec execute 487 sec latency 227.772 ms 2024-01-31T05:34:08.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1214751 10.54 MB/sec execute 488 sec latency 78.069 ms 2024-01-31T05:34:08.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:08 smithi045 ceph-mon[31585]: pgmap v362: 97 pgs: 97 active+clean; 577 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-01-31T05:34:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:08 smithi156 ceph-mon[35720]: pgmap v362: 97 pgs: 97 active+clean; 577 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-01-31T05:34:09.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:08 smithi186 ceph-mon[35659]: pgmap v362: 97 pgs: 97 active+clean; 577 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-01-31T05:34:09.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1216086 10.53 MB/sec execute 489 sec latency 17.517 ms 2024-01-31T05:34:10.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1218220 10.54 MB/sec execute 490 sec latency 48.470 ms 2024-01-31T05:34:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:10 smithi156 ceph-mon[35720]: pgmap v363: 97 pgs: 97 active+clean; 577 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-01-31T05:34:11.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:10 smithi186 ceph-mon[35659]: pgmap v363: 97 pgs: 97 active+clean; 577 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-01-31T05:34:11.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:10 smithi045 ceph-mon[31585]: pgmap v363: 97 pgs: 97 active+clean; 577 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-01-31T05:34:11.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1219491 10.52 MB/sec execute 491 sec latency 17.772 ms 2024-01-31T05:34:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:11 smithi156 ceph-mon[35720]: pgmap v364: 97 pgs: 97 active+clean; 577 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-01-31T05:34:12.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:11 smithi186 ceph-mon[35659]: pgmap v364: 97 pgs: 97 active+clean; 577 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-01-31T05:34:12.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:11 smithi045 ceph-mon[31585]: pgmap v364: 97 pgs: 97 active+clean; 577 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-01-31T05:34:12.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1221763 10.53 MB/sec execute 492 sec latency 74.897 ms 2024-01-31T05:34:13.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1222941 10.52 MB/sec execute 493 sec latency 19.883 ms 2024-01-31T05:34:13.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:13 smithi045 ceph-mon[31585]: osdmap e106: 12 total, 12 up, 12 in 2024-01-31T05:34:13.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:13 smithi156 ceph-mon[35720]: osdmap e106: 12 total, 12 up, 12 in 2024-01-31T05:34:14.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:13 smithi186 ceph-mon[35659]: osdmap e106: 12 total, 12 up, 12 in 2024-01-31T05:34:14.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1225369 10.53 MB/sec execute 494 sec latency 13.738 ms 2024-01-31T05:34:14.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:14 smithi045 ceph-mon[31585]: pgmap v366: 97 pgs: 97 active+clean; 589 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 249 op/s 2024-01-31T05:34:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:14 smithi156 ceph-mon[35720]: pgmap v366: 97 pgs: 97 active+clean; 589 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 249 op/s 2024-01-31T05:34:15.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:14 smithi186 ceph-mon[35659]: pgmap v366: 97 pgs: 97 active+clean; 589 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 249 op/s 2024-01-31T05:34:15.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1226214 10.51 MB/sec execute 495 sec latency 126.571 ms 2024-01-31T05:34:16.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1228789 10.53 MB/sec execute 496 sec latency 15.615 ms 2024-01-31T05:34:16.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:16 smithi045 ceph-mon[31585]: pgmap v367: 97 pgs: 97 active+clean; 589 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.3 MiB/s wr, 200 op/s 2024-01-31T05:34:16.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:16 smithi156 ceph-mon[35720]: pgmap v367: 97 pgs: 97 active+clean; 589 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.3 MiB/s wr, 200 op/s 2024-01-31T05:34:17.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:16 smithi186 ceph-mon[35659]: pgmap v367: 97 pgs: 97 active+clean; 589 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.3 MiB/s wr, 200 op/s 2024-01-31T05:34:17.359 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:16 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 5. 2024-01-31T05:34:17.359 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:16 smithi156 systemd[1]: Stopped Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:34:17.359 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:16 smithi156 systemd[1]: Starting Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:34:17.359 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:17 smithi156 podman[72764]: 2024-01-31 05:34:17.109309496 +0000 UTC m=+0.018408368 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:34:17.359 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:17 smithi156 podman[72764]: 2024-01-31 05:34:17.224579169 +0000 UTC m=+0.133678019 container create e5de3b395006d3a1707ed5c1f90e5629996b111d9c7f624d7c8e9ca59c80b004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-01-31T05:34:17.645 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:17 smithi156 podman[72764]: 2024-01-31 05:34:17.469408424 +0000 UTC m=+0.378507293 container init e5de3b395006d3a1707ed5c1f90e5629996b111d9c7f624d7c8e9ca59c80b004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS) 2024-01-31T05:34:17.645 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:17 smithi156 podman[72764]: 2024-01-31 05:34:17.472246251 +0000 UTC m=+0.381345118 container start e5de3b395006d3a1707ed5c1f90e5629996b111d9c7f624d7c8e9ca59c80b004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, ceph=True, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2) 2024-01-31T05:34:17.645 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:17 smithi156 ceph-mds[72783]: -- 172.21.15.156:0/2492423621 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ddfd6b7520 con 0x55ddfd68cc00 2024-01-31T05:34:17.645 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:17 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: starting mds.h at 2024-01-31T05:34:17.645 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:17 smithi156 bash[72764]: e5de3b395006d3a1707ed5c1f90e5629996b111d9c7f624d7c8e9ca59c80b004 2024-01-31T05:34:17.645 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:34:17 smithi156 systemd[1]: Started Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:34:17.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1229666 10.51 MB/sec execute 497 sec latency 19.178 ms 2024-01-31T05:34:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:17 smithi156 ceph-mon[35720]: daemon mds.h restarted 2024-01-31T05:34:17.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:17 smithi156 ceph-mon[35720]: osdmap e107: 12 total, 12 up, 12 in 2024-01-31T05:34:17.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:17 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.156:6836/3609932340,v1:172.21.15.156:6837/3609932340] up:boot 2024-01-31T05:34:17.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:17 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:34:17.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:17 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:34:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:17 smithi186 ceph-mon[35659]: daemon mds.h restarted 2024-01-31T05:34:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:17 smithi186 ceph-mon[35659]: osdmap e107: 12 total, 12 up, 12 in 2024-01-31T05:34:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:17 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.156:6836/3609932340,v1:172.21.15.156:6837/3609932340] up:boot 2024-01-31T05:34:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:17 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:34:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:17 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:34:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.111+0000 7f0d2903e640 1 -- 172.21.15.45:0/993453551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d24068370 msgr2=0x7f0d24068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.111+0000 7f0d2903e640 1 --2- 172.21.15.45:0/993453551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d24068370 0x7f0d24068750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009900 tx=0x7f0d1402bd10 comp rx=0 tx=0).stop 2024-01-31T05:34:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.111+0000 7f0d2903e640 1 -- 172.21.15.45:0/993453551 shutdown_connections 2024-01-31T05:34:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.111+0000 7f0d2903e640 1 --2- 172.21.15.45:0/993453551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d2410d9f0 0x7f0d2410fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.111+0000 7f0d2903e640 1 --2- 172.21.15.45:0/993453551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d24068c90 0x7f0d2410d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.111+0000 7f0d2903e640 1 --2- 172.21.15.45:0/993453551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d24068370 0x7f0d24068750 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.111+0000 7f0d2903e640 1 -- 172.21.15.45:0/993453551 >> 172.21.15.45:0/993453551 conn(0x7f0d24076a40 msgr2=0x7f0d24076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.111+0000 7f0d2903e640 1 -- 172.21.15.45:0/993453551 shutdown_connections 2024-01-31T05:34:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.111+0000 7f0d2903e640 1 -- 172.21.15.45:0/993453551 wait complete. 2024-01-31T05:34:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d2903e640 1 Processor -- start 2024-01-31T05:34:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d2903e640 1 -- start start 2024-01-31T05:34:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d2903e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d24068370 0x7f0d24199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d2903e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d24068c90 0x7f0d24199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d2903e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d2410d9f0 0x7f0d2419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d2903e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2419e300 con 0x7f0d2410d9f0 2024-01-31T05:34:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d2903e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2419e470 con 0x7f0d24068c90 2024-01-31T05:34:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d2903e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2419e630 con 0x7f0d24068370 2024-01-31T05:34:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d2359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d2410d9f0 0x7f0d2419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d22d9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d24068370 0x7f0d24199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d2359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d2410d9f0 0x7f0d2419dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38278/0 (socket says 172.21.15.45:38278) 2024-01-31T05:34:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d2359e640 1 -- 172.21.15.45:0/1955671191 learned_addr learned my addr 172.21.15.45:0/1955671191 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.112+0000 7f0d1a59c640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d24068c90 0x7f0d24199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.113+0000 7f0d2359e640 1 -- 172.21.15.45:0/1955671191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d24068370 msgr2=0x7f0d24199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.113+0000 7f0d2359e640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d24068370 0x7f0d24199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.113+0000 7f0d2359e640 1 -- 172.21.15.45:0/1955671191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d24068c90 msgr2=0x7f0d24199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.113+0000 7f0d2359e640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d24068c90 0x7f0d24199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.113+0000 7f0d2359e640 1 -- 172.21.15.45:0/1955671191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d14009560 con 0x7f0d2410d9f0 2024-01-31T05:34:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.113+0000 7f0d1a59c640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d24068c90 0x7f0d24199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:34:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.113+0000 7f0d2359e640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d2410d9f0 0x7f0d2419dd00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00eab0 tx=0x7f0d1c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.113+0000 7f0d20d99640 1 -- 172.21.15.45:0/1955671191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c00ce40 con 0x7f0d2410d9f0 2024-01-31T05:34:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.113+0000 7f0d2903e640 1 -- 172.21.15.45:0/1955671191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2419e910 con 0x7f0d2410d9f0 2024-01-31T05:34:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.113+0000 7f0d20d99640 1 -- 172.21.15.45:0/1955671191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d1c005020 con 0x7f0d2410d9f0 2024-01-31T05:34:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.113+0000 7f0d20d99640 1 -- 172.21.15.45:0/1955671191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c010690 con 0x7f0d2410d9f0 2024-01-31T05:34:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.113+0000 7f0d2903e640 1 -- 172.21.15.45:0/1955671191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d241a6190 con 0x7f0d2410d9f0 2024-01-31T05:34:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.115+0000 7f0d20d99640 1 -- 172.21.15.45:0/1955671191 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f0d1c0053e0 con 0x7f0d2410d9f0 2024-01-31T05:34:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.116+0000 7f0d2be03740 1 Processor -- start 2024-01-31T05:34:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.116+0000 7f0d2be03740 1 -- start start 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.116+0000 7f0d20d99640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0d100a14f0 0x7f0d100a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.116+0000 7f0d1a59c640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0d100a14f0 0x7f0d100a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.116+0000 7f0d2be03740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9060 0x555acfeb9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.116+0000 7f0d2be03740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9980 0x555acfec2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.116+0000 7f0d2be03740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfec2750 0x555acfec4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.116+0000 7f0d2be03740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acfe28040 con 0x555acfeb9060 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.116+0000 7f0d2be03740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acfe25730 con 0x555acfec2750 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.116+0000 7f0d2be03740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acfe21020 con 0x555acfeb9980 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.116+0000 7f0d20d99640 1 -- 172.21.15.45:0/1955671191 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0d1c014070 con 0x7f0d2410d9f0 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.116+0000 7f0d22d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9060 0x555acfeb9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d22d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9060 0x555acfeb9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38282/0 (socket says 172.21.15.45:38282) 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d22d9d640 1 -- 172.21.15.45:0/1779049448 learned_addr learned my addr 172.21.15.45:0/1779049448 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d1a59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9980 0x555acfec2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d2359e640 1 --2- 172.21.15.45:0/1779049448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfec2750 0x555acfec4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d22d9d640 1 -- 172.21.15.45:0/1779049448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9980 msgr2=0x555acfec2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d22d9d640 1 --2- 172.21.15.45:0/1779049448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9980 0x555acfec2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d22d9d640 1 -- 172.21.15.45:0/1779049448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfec2750 msgr2=0x555acfec4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d22d9d640 1 --2- 172.21.15.45:0/1779049448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfec2750 0x555acfec4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d1a59c640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0d100a14f0 0x7f0d100a39b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009fd0 tx=0x7f0d0c011c90 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d22d9d640 1 -- 172.21.15.45:0/1779049448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555acfe18e40 con 0x555acfeb9060 2024-01-31T05:34:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d22d9d640 1 --2- 172.21.15.45:0/1779049448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9060 0x555acfeb9440 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009900 tx=0x7f0d14005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c735800ef6020e2a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d1959a640 1 -- 172.21.15.45:0/1779049448 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d140046c0 con 0x555acfeb9060 2024-01-31T05:34:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d1959a640 1 -- 172.21.15.45:0/1779049448 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d14004500 con 0x555acfeb9060 2024-01-31T05:34:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.117+0000 7f0d1959a640 1 -- 172.21.15.45:0/1779049448 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1402c800 con 0x555acfeb9060 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.118+0000 7f0d2be03740 1 -- 172.21.15.45:0/1779049448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9060 msgr2=0x555acfeb9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.118+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1779049448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9060 0x555acfeb9440 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009900 tx=0x7f0d14005e40 comp rx=0 tx=0).stop 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.118+0000 7f0d2be03740 1 -- 172.21.15.45:0/1779049448 shutdown_connections 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.118+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1779049448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfec2750 0x555acfec4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.118+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1779049448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9980 0x555acfec2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.118+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1779049448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9060 0x555acfeb9440 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.118+0000 7f0d2be03740 1 -- 172.21.15.45:0/1779049448 >> 172.21.15.45:0/1779049448 conn(0x555acfeb3980 msgr2=0x555acfeb3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.118+0000 7f0d2be03740 1 -- 172.21.15.45:0/1779049448 shutdown_connections 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.118+0000 7f0d2be03740 1 -- 172.21.15.45:0/1779049448 wait complete. 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.118+0000 7f0d2be03740 1 -- start start 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.118+0000 7f0d2be03740 1 Processor -- start 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d2be03740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9060 0x555acfec21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d2be03740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9980 0x555acfebf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d2be03740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfec2750 0x555acfebfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d2be03740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acfdda1b0 con 0x555acfeb9980 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d2be03740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acfdd9e50 con 0x555acfeb9060 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d2be03740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acfd9f090 con 0x555acfec2750 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d2be03740 10 client.0 _ll_register_callbacks cb 0x555acfdc4a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d1a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9980 0x555acfebf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d1a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9980 0x555acfebf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38294/0 (socket says 172.21.15.45:38294) 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d1a59c640 1 -- 172.21.15.45:0/1671733352 learned_addr learned my addr 172.21.15.45:0/1671733352 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d2359e640 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfec2750 0x555acfebfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d22d9d640 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9060 0x555acfec21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d1a59c640 1 -- 172.21.15.45:0/1671733352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfec2750 msgr2=0x555acfebfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d1a59c640 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfec2750 0x555acfebfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d1a59c640 1 -- 172.21.15.45:0/1671733352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9060 msgr2=0x555acfec21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d1a59c640 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9060 0x555acfec21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d1a59c640 1 -- 172.21.15.45:0/1671733352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d241105d0 con 0x555acfeb9980 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d1a59c640 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9980 0x555acfebf570 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00fef0 tx=0x7f0d0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78960aeba31f6fb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.119+0000 7f0d0a7fc640 1 -- 172.21.15.45:0/1671733352 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c01f7c0 con 0x555acfeb9980 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d0a7fc640 1 -- 172.21.15.45:0/1671733352 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0d0c020e20 con 0x555acfeb9980 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d2be03740 1 -- 172.21.15.45:0/1671733352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555acfe19d30 con 0x555acfeb9980 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d0a7fc640 1 -- 172.21.15.45:0/1671733352 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d0c01f920 con 0x555acfeb9980 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d2be03740 10 client.16338 fetch_fsmap learned FSMap version 30 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d2be03740 1 -- 172.21.15.45:0/1671733352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555acfe1f6a0 con 0x555acfeb9980 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d0a7fc640 1 -- 172.21.15.45:0/1671733352 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555acfe19d30 con 0x555acfeb9980 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d0a7fc640 1 -- 172.21.15.45:0/1671733352 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0d0c026070 con 0x555acfeb9980 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d2be03740 10 client.16338 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d2be03740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d2be03740 20 client.16338 populate_metadata read hostname 'smithi045' 2024-01-31T05:34:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555acfed2490 0x555acfed48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d2be03740 4 client.16338 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d2be03740 1 -- 172.21.15.45:0/1671733352 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555acfb20600 con 0x555acfed2490 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.120+0000 7f0d22d9d640 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555acfed2490 0x555acfed48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d22d9d640 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555acfed2490 0x555acfed48b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf376a05416c805 server_cookie=77c17da2a13bc58 in_seq=0 out_seq=0 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d0a7fc640 10 client.16338 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d0a7fc640 1 -- 172.21.15.45:0/1671733352 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555acfb20600 con 0x555acfed2490 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d0a7fc640 10 client.16338 handle_command_reply: tid=0 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d2be03740 1 client.16338 shutdown 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d2be03740 1 -- 172.21.15.45:0/1671733352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9980 msgr2=0x555acfebf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9980 0x555acfebf570 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00fef0 tx=0x7f0d0c004730 comp rx=0 tx=0).stop 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d2be03740 1 -- 172.21.15.45:0/1671733352 shutdown_connections 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555acfed2490 0x555acfed48b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfeb9980 0x555acfebf570 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d0a7fc640 0 client.16338 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfec2750 0x555acfebfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1671733352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9060 0x555acfec21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.121+0000 7f0d2be03740 1 -- 172.21.15.45:0/1671733352 >> 172.21.15.45:0/1671733352 conn(0x555acfeb3980 msgr2=0x555acfeb56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.122+0000 7f0d2be03740 1 -- 172.21.15.45:0/1671733352 shutdown_connections 2024-01-31T05:34:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.122+0000 7f0d2be03740 1 -- 172.21.15.45:0/1671733352 wait complete. 2024-01-31T05:34:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.122+0000 7f0d2be03740 20 client.16338 trim_cache size 0 max 16384 2024-01-31T05:34:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.135+0000 7f0d2be03740 1 Processor -- start 2024-01-31T05:34:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2be03740 1 -- start start 2024-01-31T05:34:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2be03740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9060 0x555acfec1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2be03740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9980 0x555acfec21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2be03740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 0x555acfec93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2be03740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acfb20600 con 0x555acfec2750 2024-01-31T05:34:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2be03740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acfe19d30 con 0x555acfeb9980 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2be03740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acfec0530 con 0x555acfeb9060 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 0x555acfec93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 0x555acfec93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38298/0 (socket says 172.21.15.45:38298) 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2359e640 1 -- 172.21.15.45:0/1086819298 learned_addr learned my addr 172.21.15.45:0/1086819298 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d22d9d640 1 --2- 172.21.15.45:0/1086819298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9060 0x555acfec1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d1a59c640 1 --2- 172.21.15.45:0/1086819298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9980 0x555acfec21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2359e640 1 -- 172.21.15.45:0/1086819298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9060 msgr2=0x555acfec1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2359e640 1 --2- 172.21.15.45:0/1086819298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9060 0x555acfec1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2359e640 1 -- 172.21.15.45:0/1086819298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9980 msgr2=0x555acfec21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2359e640 1 --2- 172.21.15.45:0/1086819298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9980 0x555acfec21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2359e640 1 -- 172.21.15.45:0/1086819298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555acfdbc030 con 0x555acfec2750 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.136+0000 7f0d2359e640 1 --2- 172.21.15.45:0/1086819298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 0x555acfec93e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0cad50 tx=0x7f0d1c099dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95ace1f0bbc3d47 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.137+0000 7f0d18d99640 1 -- 172.21.15.45:0/1086819298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c09e4f0 con 0x555acfec2750 2024-01-31T05:34:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.137+0000 7f0d18d99640 1 -- 172.21.15.45:0/1086819298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d1c0a2730 con 0x555acfec2750 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.137+0000 7f0d18d99640 1 -- 172.21.15.45:0/1086819298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c09e150 con 0x555acfec2750 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.137+0000 7f0d2be03740 1 -- 172.21.15.45:0/1086819298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 msgr2=0x555acfec93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.137+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1086819298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 0x555acfec93e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0cad50 tx=0x7f0d1c099dc0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.137+0000 7f0d2be03740 1 -- 172.21.15.45:0/1086819298 shutdown_connections 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.137+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1086819298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 0x555acfec93e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.137+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1086819298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9980 0x555acfec21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.137+0000 7f0d2be03740 1 --2- 172.21.15.45:0/1086819298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9060 0x555acfec1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.137+0000 7f0d2be03740 1 -- 172.21.15.45:0/1086819298 >> 172.21.15.45:0/1086819298 conn(0x555acfeb3980 msgr2=0x555acfecde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.137+0000 7f0d2be03740 1 -- 172.21.15.45:0/1086819298 shutdown_connections 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.137+0000 7f0d2be03740 1 -- 172.21.15.45:0/1086819298 wait complete. 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2be03740 1 -- start start 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2be03740 1 Processor -- start 2024-01-31T05:34:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2be03740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9060 0x555acfec9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2be03740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9980 0x555acfec96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2be03740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 0x555acfec7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2be03740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acfec7f00 con 0x555acfec2750 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2be03740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acfec8070 con 0x555acfeb9060 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2be03740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acfec8230 con 0x555acfeb9980 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2be03740 10 client.0 _ll_register_callbacks cb 0x7f0d0c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 0x555acfec7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d22d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9060 0x555acfec9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 0x555acfec7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38314/0 (socket says 172.21.15.45:38314) 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2359e640 1 -- 172.21.15.45:0/4071969632 learned_addr learned my addr 172.21.15.45:0/4071969632 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d1a59c640 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9980 0x555acfec96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2359e640 1 -- 172.21.15.45:0/4071969632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9980 msgr2=0x555acfec96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2359e640 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9980 0x555acfec96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2359e640 1 -- 172.21.15.45:0/4071969632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9060 msgr2=0x555acfec9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2359e640 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9060 0x555acfec9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d2359e640 1 -- 172.21.15.45:0/4071969632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d241a6190 con 0x555acfec2750 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.138+0000 7f0d22d9d640 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9060 0x555acfec9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d2359e640 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 0x555acfec7800 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00ea80 tx=0x7f0d1c09a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e2a37a792ebd36 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d0a7fc640 1 -- 172.21.15.45:0/4071969632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c098800 con 0x555acfec2750 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d0a7fc640 1 -- 172.21.15.45:0/4071969632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0d1c0b82c0 con 0x555acfec2750 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d2be03740 1 -- 172.21.15.45:0/4071969632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555acfe1a010 con 0x555acfec2750 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d0a7fc640 1 -- 172.21.15.45:0/4071969632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d1c0b8480 con 0x555acfec2750 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d2be03740 10 client.16344 fetch_fsmap learned FSMap version 30 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d2be03740 1 -- 172.21.15.45:0/4071969632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555acfde1a50 con 0x555acfec2750 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d0a7fc640 1 -- 172.21.15.45:0/4071969632 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555acfe1a010 con 0x555acfec2750 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d0a7fc640 1 -- 172.21.15.45:0/4071969632 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0d1c07b060 con 0x555acfec2750 2024-01-31T05:34:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d2be03740 10 client.16344 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:34:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d2be03740 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:34:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d2be03740 20 client.16344 populate_metadata read hostname 'smithi045' 2024-01-31T05:34:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d2be03740 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555acff298d0 0x555acff2bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d2be03740 4 client.16344 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:34:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d2be03740 1 -- 172.21.15.45:0/4071969632 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555acfb73c50 con 0x555acff298d0 2024-01-31T05:34:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.139+0000 7f0d22d9d640 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555acff298d0 0x555acff2bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.140+0000 7f0d22d9d640 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555acff298d0 0x555acff2bcf0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2beb86e0381a4d19 server_cookie=3b10559e6e899b33 in_seq=0 out_seq=0 2024-01-31T05:34:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.140+0000 7f0d0a7fc640 10 client.16344 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:18.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:17 smithi045 ceph-mon[31585]: daemon mds.h restarted 2024-01-31T05:34:18.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:17 smithi045 ceph-mon[31585]: osdmap e107: 12 total, 12 up, 12 in 2024-01-31T05:34:18.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:17 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.156:6836/3609932340,v1:172.21.15.156:6837/3609932340] up:boot 2024-01-31T05:34:18.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:17 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:34:18.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:17 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.152+0000 7f0d0a7fc640 1 -- 172.21.15.45:0/4071969632 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555acfb73c50 con 0x555acff298d0 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.152+0000 7f0d0a7fc640 10 client.16344 handle_command_reply: tid=0 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.152+0000 7f0d2be03740 1 client.16344 shutdown 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.152+0000 7f0d2be03740 1 -- 172.21.15.45:0/4071969632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 msgr2=0x555acfec7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.152+0000 7f0d2be03740 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 0x555acfec7800 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00ea80 tx=0x7f0d1c09a520 comp rx=0 tx=0).stop 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.152+0000 7f0d2be03740 1 -- 172.21.15.45:0/4071969632 shutdown_connections 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.152+0000 7f0d2be03740 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x555acff298d0 0x555acff2bcf0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.152+0000 7f0d2be03740 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555acfec2750 0x555acfec7800 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.152+0000 7f0d2be03740 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555acfeb9980 0x555acfec96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.152+0000 7f0d2be03740 1 --2- 172.21.15.45:0/4071969632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555acfeb9060 0x555acfec9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.152+0000 7f0d0a7fc640 0 client.16344 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.152+0000 7f0d2be03740 1 -- 172.21.15.45:0/4071969632 >> 172.21.15.45:0/4071969632 conn(0x555acfeb3980 msgr2=0x555acfebee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2be03740 1 -- 172.21.15.45:0/4071969632 shutdown_connections 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2be03740 1 -- 172.21.15.45:0/4071969632 wait complete. 2024-01-31T05:34:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2be03740 20 client.16344 trim_cache size 0 max 16384 2024-01-31T05:34:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:34:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:34:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:34:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:34:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2903e640 1 -- 172.21.15.45:0/1955671191 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0d100a14f0 msgr2=0x7f0d100a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2903e640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0d100a14f0 0x7f0d100a39b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009fd0 tx=0x7f0d0c011c90 comp rx=0 tx=0).stop 2024-01-31T05:34:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2903e640 1 -- 172.21.15.45:0/1955671191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d2410d9f0 msgr2=0x7f0d2419dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2903e640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d2410d9f0 0x7f0d2419dd00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00eab0 tx=0x7f0d1c009f80 comp rx=0 tx=0).stop 2024-01-31T05:34:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2903e640 1 -- 172.21.15.45:0/1955671191 shutdown_connections 2024-01-31T05:34:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2903e640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0d100a14f0 0x7f0d100a39b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2903e640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d2410d9f0 0x7f0d2419dd00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2903e640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d24068c90 0x7f0d24199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2903e640 1 --2- 172.21.15.45:0/1955671191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d24068370 0x7f0d24199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2903e640 1 -- 172.21.15.45:0/1955671191 >> 172.21.15.45:0/1955671191 conn(0x7f0d24076a40 msgr2=0x7f0d2410eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2903e640 1 -- 172.21.15.45:0/1955671191 shutdown_connections 2024-01-31T05:34:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:18.153+0000 7f0d2903e640 1 -- 172.21.15.45:0/1955671191 wait complete. 2024-01-31T05:34:18.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1231816 10.51 MB/sec execute 498 sec latency 60.365 ms 2024-01-31T05:34:18.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:18 smithi156 ceph-mon[35720]: pgmap v368: 97 pgs: 97 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.2 MiB/s wr, 193 op/s 2024-01-31T05:34:19.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:18 smithi186 ceph-mon[35659]: pgmap v368: 97 pgs: 97 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.2 MiB/s wr, 193 op/s 2024-01-31T05:34:19.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:18 smithi045 ceph-mon[31585]: pgmap v368: 97 pgs: 97 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.2 MiB/s wr, 193 op/s 2024-01-31T05:34:19.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1232899 10.50 MB/sec execute 499 sec latency 94.810 ms 2024-01-31T05:34:19.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:19 smithi156 ceph-mon[35720]: pgmap v370: 97 pgs: 97 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 241 op/s 2024-01-31T05:34:20.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:19 smithi186 ceph-mon[35659]: pgmap v370: 97 pgs: 97 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 241 op/s 2024-01-31T05:34:20.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:19 smithi045 ceph-mon[31585]: pgmap v370: 97 pgs: 97 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 241 op/s 2024-01-31T05:34:20.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1234828 10.50 MB/sec execute 500 sec latency 105.818 ms 2024-01-31T05:34:21.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1236317 10.50 MB/sec execute 501 sec latency 29.380 ms 2024-01-31T05:34:21.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:21 smithi156 ceph-mon[35720]: pgmap v371: 97 pgs: 97 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 112 op/s 2024-01-31T05:34:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:21 smithi186 ceph-mon[35659]: pgmap v371: 97 pgs: 97 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 112 op/s 2024-01-31T05:34:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:21 smithi045 ceph-mon[31585]: pgmap v371: 97 pgs: 97 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 112 op/s 2024-01-31T05:34:22.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1238270 10.49 MB/sec execute 502 sec latency 23.421 ms 2024-01-31T05:34:23.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1239843 10.50 MB/sec execute 503 sec latency 23.765 ms 2024-01-31T05:34:24.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1241375 10.49 MB/sec execute 504 sec latency 49.881 ms 2024-01-31T05:34:24.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:24 smithi045 ceph-mon[31585]: pgmap v372: 97 pgs: 97 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 200 op/s 2024-01-31T05:34:24.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:24 smithi156 ceph-mon[35720]: pgmap v372: 97 pgs: 97 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 200 op/s 2024-01-31T05:34:25.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:24 smithi186 ceph-mon[35659]: pgmap v372: 97 pgs: 97 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 200 op/s 2024-01-31T05:34:25.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1243184 10.49 MB/sec execute 505 sec latency 106.385 ms 2024-01-31T05:34:26.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1244717 10.48 MB/sec execute 506 sec latency 22.818 ms 2024-01-31T05:34:26.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:26 smithi045 ceph-mon[31585]: pgmap v373: 97 pgs: 97 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 200 op/s 2024-01-31T05:34:26.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:26 smithi156 ceph-mon[35720]: pgmap v373: 97 pgs: 97 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 200 op/s 2024-01-31T05:34:27.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:26 smithi186 ceph-mon[35659]: pgmap v373: 97 pgs: 97 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 200 op/s 2024-01-31T05:34:27.800 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1246806 10.49 MB/sec execute 507 sec latency 19.537 ms 2024-01-31T05:34:28.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1248277 10.48 MB/sec execute 508 sec latency 21.114 ms 2024-01-31T05:34:28.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:28 smithi045 ceph-mon[31585]: pgmap v374: 97 pgs: 97 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 204 op/s 2024-01-31T05:34:28.893 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:28 smithi186 ceph-mon[35659]: pgmap v374: 97 pgs: 97 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 204 op/s 2024-01-31T05:34:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:28 smithi156 ceph-mon[35720]: pgmap v374: 97 pgs: 97 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 204 op/s 2024-01-31T05:34:29.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1250239 10.48 MB/sec execute 509 sec latency 42.666 ms 2024-01-31T05:34:30.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1251609 10.47 MB/sec execute 510 sec latency 46.603 ms 2024-01-31T05:34:30.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:30 smithi156 ceph-mon[35720]: pgmap v375: 97 pgs: 97 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 170 op/s 2024-01-31T05:34:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:30 smithi186 ceph-mon[35659]: pgmap v375: 97 pgs: 97 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 170 op/s 2024-01-31T05:34:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:30 smithi045 ceph-mon[31585]: pgmap v375: 97 pgs: 97 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 170 op/s 2024-01-31T05:34:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.241+0000 7fdae5259640 1 -- 172.21.15.45:0/1054899307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae0104c00 msgr2=0x7fdae0104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.241+0000 7fdae5259640 1 --2- 172.21.15.45:0/1054899307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae0104c00 0x7fdae0104fe0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009900 tx=0x7fdad002bcd0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.241+0000 7fdae5259640 1 -- 172.21.15.45:0/1054899307 shutdown_connections 2024-01-31T05:34:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.241+0000 7fdae5259640 1 --2- 172.21.15.45:0/1054899307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0100800 0x7fdae010ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.241+0000 7fdae5259640 1 --2- 172.21.15.45:0/1054899307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdae00ffe40 0x7fdae01002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.241+0000 7fdae5259640 1 --2- 172.21.15.45:0/1054899307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae0104c00 0x7fdae0104fe0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.241+0000 7fdae5259640 1 -- 172.21.15.45:0/1054899307 >> 172.21.15.45:0/1054899307 conn(0x7fdae0076a50 msgr2=0x7fdae0076e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.241+0000 7fdae5259640 1 -- 172.21.15.45:0/1054899307 shutdown_connections 2024-01-31T05:34:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.241+0000 7fdae5259640 1 -- 172.21.15.45:0/1054899307 wait complete. 2024-01-31T05:34:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdae5259640 1 Processor -- start 2024-01-31T05:34:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdae5259640 1 -- start start 2024-01-31T05:34:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdae5259640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdae00ffe40 0x7fdae0101910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdae5259640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae0100800 0x7fdae0101e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdae5259640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0104c00 0x7fdae0102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdae5259640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae01a55a0 con 0x7fdae0100800 2024-01-31T05:34:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdae5259640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae01a5710 con 0x7fdae0104c00 2024-01-31T05:34:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdae5259640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae01a58d0 con 0x7fdae00ffe40 2024-01-31T05:34:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae0100800 0x7fdae0101e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdaded9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdae00ffe40 0x7fdae0101910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdaded9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdae00ffe40 0x7fdae0101910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:40108/0 (socket says 172.21.15.45:40108) 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae0100800 0x7fdae0101e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39098/0 (socket says 172.21.15.45:39098) 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdaded9d640 1 -- 172.21.15.45:0/2133411078 learned_addr learned my addr 172.21.15.45:0/2133411078 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdadf59e640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0104c00 0x7fdae0102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdad7fff640 1 -- 172.21.15.45:0/2133411078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdae00ffe40 msgr2=0x7fdae0101910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdad7fff640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdae00ffe40 0x7fdae0101910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdad7fff640 1 -- 172.21.15.45:0/2133411078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0104c00 msgr2=0x7fdae0102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.242+0000 7fdad7fff640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0104c00 0x7fdae0102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.243+0000 7fdad7fff640 1 -- 172.21.15.45:0/2133411078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad8009840 con 0x7fdae0100800 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.243+0000 7fdad7fff640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae0100800 0x7fdae0101e50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdac800e9f0 tx=0x7fdac800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.243+0000 7fdadcd99640 1 -- 172.21.15.45:0/2133411078 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac800ccc0 con 0x7fdae0100800 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.243+0000 7fdadcd99640 1 -- 172.21.15.45:0/2133411078 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdac800ce20 con 0x7fdae0100800 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.243+0000 7fdae5259640 1 -- 172.21.15.45:0/2133411078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad0009560 con 0x7fdae0100800 2024-01-31T05:34:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.243+0000 7fdadcd99640 1 -- 172.21.15.45:0/2133411078 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8004a00 con 0x7fdae0100800 2024-01-31T05:34:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.243+0000 7fdae5259640 1 -- 172.21.15.45:0/2133411078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae01a5ee0 con 0x7fdae0100800 2024-01-31T05:34:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.245+0000 7fdadcd99640 1 -- 172.21.15.45:0/2133411078 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fdac8005020 con 0x7fdae0100800 2024-01-31T05:34:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.245+0000 7fdae8015740 1 Processor -- start 2024-01-31T05:34:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.245+0000 7fdae8015740 1 -- start start 2024-01-31T05:34:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdae8015740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab08440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdae8015740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab11210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdae8015740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab11750 0x55aeaab13b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdae8015740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeaaa77040 con 0x55aeaab08980 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdae8015740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeaaa74730 con 0x55aeaab11750 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdae8015740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeaaa70020 con 0x55aeaab08060 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdaded9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab11210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdadf59e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab08440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdadf59e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab08440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:40110/0 (socket says 172.21.15.45:40110) 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdadf59e640 1 -- 172.21.15.45:0/4269519735 learned_addr learned my addr 172.21.15.45:0/4269519735 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdaded9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab11210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39114/0 (socket says 172.21.15.45:39114) 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdad7fff640 1 --2- 172.21.15.45:0/4269519735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab11750 0x55aeaab13b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdadcd99640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fdacc0a14f0 0x7fdacc0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdadf59e640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fdacc0a14f0 0x7fdacc0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdaded9d640 1 -- 172.21.15.45:0/4269519735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 msgr2=0x55aeaab08440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdaded9d640 1 --2- 172.21.15.45:0/4269519735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab08440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdaded9d640 1 -- 172.21.15.45:0/4269519735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab11750 msgr2=0x55aeaab13b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdaded9d640 1 --2- 172.21.15.45:0/4269519735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab11750 0x55aeaab13b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdaded9d640 1 -- 172.21.15.45:0/4269519735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeaaa67e40 con 0x55aeaab08980 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdadf59e640 1 --2- 172.21.15.45:0/4269519735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab08440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdadcd99640 1 -- 172.21.15.45:0/2133411078 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdac8014070 con 0x7fdae0100800 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.246+0000 7fdaded9d640 1 --2- 172.21.15.45:0/4269519735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab11210 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009230 tx=0x7fdad0005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260527ed20ec7bce server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdad57fa640 1 -- 172.21.15.45:0/4269519735 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0004780 con 0x55aeaab08980 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdadf59e640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fdacc0a14f0 0x7fdacc0a39b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009fd0 tx=0x7fdad8004ab0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdad57fa640 1 -- 172.21.15.45:0/4269519735 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdad0004500 con 0x55aeaab08980 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdad57fa640 1 -- 172.21.15.45:0/4269519735 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad002cea0 con 0x55aeaab08980 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdae8015740 1 -- 172.21.15.45:0/4269519735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 msgr2=0x55aeaab11210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdae8015740 1 --2- 172.21.15.45:0/4269519735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab11210 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009230 tx=0x7fdad0005e10 comp rx=0 tx=0).stop 2024-01-31T05:34:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdae8015740 1 -- 172.21.15.45:0/4269519735 shutdown_connections 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdae8015740 1 --2- 172.21.15.45:0/4269519735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab11210 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdae8015740 1 --2- 172.21.15.45:0/4269519735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab11750 0x55aeaab13b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdae8015740 1 --2- 172.21.15.45:0/4269519735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab08440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdae8015740 1 -- 172.21.15.45:0/4269519735 >> 172.21.15.45:0/4269519735 conn(0x55aeaab02980 msgr2=0x55aeaab02d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdae8015740 1 -- 172.21.15.45:0/4269519735 shutdown_connections 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.247+0000 7fdae8015740 1 -- 172.21.15.45:0/4269519735 wait complete. 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdae8015740 1 -- start start 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdae8015740 1 Processor -- start 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdae8015740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab0e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdae8015740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab0ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdae8015740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab11750 0x55aeaab0f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdae8015740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeaaa291b0 con 0x55aeaab08980 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdae8015740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeaaa28e50 con 0x55aeaab11750 2024-01-31T05:34:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdae8015740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeaa9ee090 con 0x55aeaab08060 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdae8015740 10 client.0 _ll_register_callbacks cb 0x55aeaaa13a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab0ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab0ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39122/0 (socket says 172.21.15.45:39122) 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdad7fff640 1 -- 172.21.15.45:0/1965944007 learned_addr learned my addr 172.21.15.45:0/1965944007 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdaded9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab0e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdadf59e640 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab11750 0x55aeaab0f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdaded9d640 1 -- 172.21.15.45:0/1965944007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab11750 msgr2=0x55aeaab0f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdaded9d640 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab11750 0x55aeaab0f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdaded9d640 1 -- 172.21.15.45:0/1965944007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 msgr2=0x55aeaab0ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdaded9d640 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab0ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdaded9d640 1 -- 172.21.15.45:0/1965944007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae010d360 con 0x55aeaab08060 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdad7fff640 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab0ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:34:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.248+0000 7fdadf59e640 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab11750 0x55aeaab0f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.249+0000 7fdaded9d640 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab0e680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009230 tx=0x7fdad0004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2e3d92e2e2734a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.249+0000 7fdac67fc640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.249+0000 7fdac67fc640 1 -- 172.21.15.45:0/1965944007 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0004450 con 0x55aeaab08060 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.249+0000 7fdae8015740 1 -- 172.21.15.45:0/1965944007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aeaaa68d30 con 0x55aeaab08060 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.249+0000 7fdac67fc640 1 -- 172.21.15.45:0/1965944007 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdad0005000 con 0x55aeaab08060 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.249+0000 7fdac67fc640 1 -- 172.21.15.45:0/1965944007 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdad0005200 con 0x55aeaab08060 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.249+0000 7fdae8015740 10 client.25918 fetch_fsmap learned FSMap version 30 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.249+0000 7fdae8015740 1 -- 172.21.15.45:0/1965944007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55aeaaa6e6a0 con 0x55aeaab08060 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.249+0000 7fdac67fc640 1 -- 172.21.15.45:0/1965944007 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aeaaa68d30 con 0x55aeaab08060 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.249+0000 7fdac67fc640 1 -- 172.21.15.45:0/1965944007 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdad003c050 con 0x55aeaab08060 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.250+0000 7fdae8015740 10 client.25918 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.250+0000 7fdae8015740 10 client.25918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.250+0000 7fdae8015740 20 client.25918 populate_metadata read hostname 'smithi045' 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.250+0000 7fdae8015740 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55aeaab21300 0x55aeaab236f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.250+0000 7fdae8015740 4 client.25918 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.250+0000 7fdae8015740 1 -- 172.21.15.45:0/1965944007 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aeaa76f600 con 0x55aeaab21300 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.250+0000 7fdad7fff640 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55aeaab21300 0x55aeaab236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.250+0000 7fdad7fff640 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55aeaab21300 0x55aeaab236f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c73fc0933812d17 server_cookie=bc053f828906f146 in_seq=0 out_seq=0 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.250+0000 7fdac67fc640 10 client.25918 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdac67fc640 1 -- 172.21.15.45:0/1965944007 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aeaa76f600 con 0x55aeaab21300 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdac67fc640 10 client.25918 handle_command_reply: tid=0 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdae8015740 1 client.25918 shutdown 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdae8015740 1 -- 172.21.15.45:0/1965944007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 msgr2=0x55aeaab0e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdae8015740 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab0e680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009230 tx=0x7fdad0004a30 comp rx=0 tx=0).stop 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdae8015740 1 -- 172.21.15.45:0/1965944007 shutdown_connections 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdae8015740 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55aeaab21300 0x55aeaab236f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdae8015740 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab0ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdae8015740 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab11750 0x55aeaab0f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdac67fc640 0 client.25918 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdae8015740 1 --2- 172.21.15.45:0/1965944007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab0e680 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdae8015740 1 -- 172.21.15.45:0/1965944007 >> 172.21.15.45:0/1965944007 conn(0x55aeaab02980 msgr2=0x55aeaab046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdae8015740 1 -- 172.21.15.45:0/1965944007 shutdown_connections 2024-01-31T05:34:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdae8015740 1 -- 172.21.15.45:0/1965944007 wait complete. 2024-01-31T05:34:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.251+0000 7fdae8015740 20 client.25918 trim_cache size 0 max 16384 2024-01-31T05:34:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.264+0000 7fdae8015740 1 Processor -- start 2024-01-31T05:34:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdae8015740 1 -- start start 2024-01-31T05:34:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdae8015740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab08060 0x55aeaab10db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdae8015740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab0ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdae8015740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab11750 0x55aeaab28050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdae8015740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeaaafe270 con 0x55aeaab08980 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdae8015740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeaab0cbd0 con 0x55aeaab08060 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdae8015740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeaab23a20 con 0x55aeaab11750 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab0ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab0ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39128/0 (socket says 172.21.15.45:39128) 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdad7fff640 1 -- 172.21.15.45:0/3349674410 learned_addr learned my addr 172.21.15.45:0/3349674410 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdaded9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab08060 0x55aeaab10db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdadf59e640 1 --2- 172.21.15.45:0/3349674410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab11750 0x55aeaab28050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdad7fff640 1 -- 172.21.15.45:0/3349674410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab11750 msgr2=0x55aeaab28050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdad7fff640 1 --2- 172.21.15.45:0/3349674410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab11750 0x55aeaab28050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdad7fff640 1 -- 172.21.15.45:0/3349674410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab08060 msgr2=0x55aeaab10db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdad7fff640 1 --2- 172.21.15.45:0/3349674410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab08060 0x55aeaab10db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdad7fff640 1 -- 172.21.15.45:0/3349674410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeaaa0b030 con 0x55aeaab08980 2024-01-31T05:34:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdaded9d640 1 --2- 172.21.15.45:0/3349674410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab08060 0x55aeaab10db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:34:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.265+0000 7fdad7fff640 1 --2- 172.21.15.45:0/3349674410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab0ef20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdac8004770 tx=0x7fdac80cf270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142e1ed16f1d874 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.266+0000 7fdad4ff9640 1 -- 172.21.15.45:0/3349674410 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac809ded0 con 0x55aeaab08980 2024-01-31T05:34:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.266+0000 7fdad4ff9640 1 -- 172.21.15.45:0/3349674410 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdac80b1070 con 0x55aeaab08980 2024-01-31T05:34:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.266+0000 7fdad4ff9640 1 -- 172.21.15.45:0/3349674410 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8099fe0 con 0x55aeaab08980 2024-01-31T05:34:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.266+0000 7fdae8015740 1 -- 172.21.15.45:0/3349674410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 msgr2=0x55aeaab0ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.266+0000 7fdae8015740 1 --2- 172.21.15.45:0/3349674410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab0ef20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdac8004770 tx=0x7fdac80cf270 comp rx=0 tx=0).stop 2024-01-31T05:34:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.266+0000 7fdae8015740 1 -- 172.21.15.45:0/3349674410 shutdown_connections 2024-01-31T05:34:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.266+0000 7fdae8015740 1 --2- 172.21.15.45:0/3349674410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab08980 0x55aeaab0ef20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.266+0000 7fdae8015740 1 --2- 172.21.15.45:0/3349674410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab11750 0x55aeaab28050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.266+0000 7fdae8015740 1 --2- 172.21.15.45:0/3349674410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab08060 0x55aeaab10db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.266+0000 7fdae8015740 1 -- 172.21.15.45:0/3349674410 >> 172.21.15.45:0/3349674410 conn(0x55aeaab02980 msgr2=0x55aeaab08ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.266+0000 7fdae8015740 1 -- 172.21.15.45:0/3349674410 shutdown_connections 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.266+0000 7fdae8015740 1 -- 172.21.15.45:0/3349674410 wait complete. 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdae8015740 1 -- start start 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdae8015740 1 Processor -- start 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdae8015740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab180b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdae8015740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab08980 0x55aeaab185f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdae8015740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab11750 0x55aeaab165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdae8015740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeaab16c00 con 0x55aeaab11750 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdae8015740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeaab16d70 con 0x55aeaab08980 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdae8015740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeaab16f30 con 0x55aeaab08060 2024-01-31T05:34:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdae8015740 10 client.0 _ll_register_callbacks cb 0x7fdac8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdadf59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab11750 0x55aeaab165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdaded9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab180b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab08980 0x55aeaab185f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdadf59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab11750 0x55aeaab165a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39130/0 (socket says 172.21.15.45:39130) 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdadf59e640 1 -- 172.21.15.45:0/1036006241 learned_addr learned my addr 172.21.15.45:0/1036006241 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdadf59e640 1 -- 172.21.15.45:0/1036006241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 msgr2=0x55aeaab180b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdadf59e640 1 --2- 172.21.15.45:0/1036006241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab180b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdadf59e640 1 -- 172.21.15.45:0/1036006241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab08980 msgr2=0x55aeaab185f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdadf59e640 1 --2- 172.21.15.45:0/1036006241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab08980 0x55aeaab185f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdadf59e640 1 -- 172.21.15.45:0/1036006241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aeaaa6e6a0 con 0x55aeaab11750 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.267+0000 7fdad7fff640 1 --2- 172.21.15.45:0/1036006241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab08980 0x55aeaab185f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdadf59e640 1 --2- 172.21.15.45:0/1036006241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab11750 0x55aeaab165a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdad8022670 tx=0x7fdad8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de76b2928bb44c2d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdac67fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:34:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdac67fc640 1 -- 172.21.15.45:0/1036006241 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad8027070 con 0x55aeaab11750 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdac67fc640 1 -- 172.21.15.45:0/1036006241 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdad8004b20 con 0x55aeaab11750 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdae8015740 1 -- 172.21.15.45:0/1036006241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aeaaa69010 con 0x55aeaab11750 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdac67fc640 1 -- 172.21.15.45:0/1036006241 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdad8004ce0 con 0x55aeaab11750 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdae8015740 10 client.16362 fetch_fsmap learned FSMap version 30 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdae8015740 1 -- 172.21.15.45:0/1036006241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fdae01a5ee0 con 0x55aeaab11750 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdac67fc640 1 -- 172.21.15.45:0/1036006241 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aeaaa69010 con 0x55aeaab11750 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdac67fc640 1 -- 172.21.15.45:0/1036006241 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdad80051e0 con 0x55aeaab11750 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdae8015740 10 client.16362 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdae8015740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdae8015740 20 client.16362 populate_metadata read hostname 'smithi045' 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdae8015740 1 --2- 172.21.15.45:0/1036006241 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55aeaab78980 0x55aeaab7ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdae8015740 4 client.16362 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdae8015740 1 -- 172.21.15.45:0/1036006241 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aeaa7c2c50 con 0x55aeaab78980 2024-01-31T05:34:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.268+0000 7fdaded9d640 1 --2- 172.21.15.45:0/1036006241 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55aeaab78980 0x55aeaab7ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.269+0000 7fdaded9d640 1 --2- 172.21.15.45:0/1036006241 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55aeaab78980 0x55aeaab7ada0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361e8d73c737fa58 server_cookie=e58740e68f5480f in_seq=0 out_seq=0 2024-01-31T05:34:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.269+0000 7fdac67fc640 10 client.16362 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.281+0000 7fdac67fc640 1 -- 172.21.15.45:0/1036006241 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aeaa7c2c50 con 0x55aeaab78980 2024-01-31T05:34:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.281+0000 7fdac67fc640 10 client.16362 handle_command_reply: tid=0 2024-01-31T05:34:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.281+0000 7fdae8015740 1 client.16362 shutdown 2024-01-31T05:34:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.281+0000 7fdae8015740 1 -- 172.21.15.45:0/1036006241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab11750 msgr2=0x55aeaab165a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.281+0000 7fdae8015740 1 --2- 172.21.15.45:0/1036006241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab11750 0x55aeaab165a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdad8022670 tx=0x7fdad8004a40 comp rx=0 tx=0).stop 2024-01-31T05:34:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.281+0000 7fdae8015740 1 -- 172.21.15.45:0/1036006241 shutdown_connections 2024-01-31T05:34:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.281+0000 7fdae8015740 1 --2- 172.21.15.45:0/1036006241 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55aeaab78980 0x55aeaab7ada0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.281+0000 7fdae8015740 1 --2- 172.21.15.45:0/1036006241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aeaab11750 0x55aeaab165a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.281+0000 7fdae8015740 1 --2- 172.21.15.45:0/1036006241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aeaab08980 0x55aeaab185f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.281+0000 7fdae8015740 1 --2- 172.21.15.45:0/1036006241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aeaab08060 0x55aeaab180b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.281+0000 7fdae8015740 1 -- 172.21.15.45:0/1036006241 >> 172.21.15.45:0/1036006241 conn(0x55aeaab02980 msgr2=0x55aeaab28050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.281+0000 7fdac67fc640 0 client.16362 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.282+0000 7fdae8015740 1 -- 172.21.15.45:0/1036006241 shutdown_connections 2024-01-31T05:34:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.282+0000 7fdae8015740 1 -- 172.21.15.45:0/1036006241 wait complete. 2024-01-31T05:34:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.282+0000 7fdae8015740 20 client.16362 trim_cache size 0 max 16384 2024-01-31T05:34:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:34:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:34:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:34:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:34:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.283+0000 7fdae5259640 1 -- 172.21.15.45:0/2133411078 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fdacc0a14f0 msgr2=0x7fdacc0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.283+0000 7fdae5259640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fdacc0a14f0 0x7fdacc0a39b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009fd0 tx=0x7fdad8004ab0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.283+0000 7fdae5259640 1 -- 172.21.15.45:0/2133411078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae0100800 msgr2=0x7fdae0101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.283+0000 7fdae5259640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae0100800 0x7fdae0101e50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdac800e9f0 tx=0x7fdac800eeb0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.283+0000 7fdae5259640 1 -- 172.21.15.45:0/2133411078 shutdown_connections 2024-01-31T05:34:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.283+0000 7fdae5259640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fdacc0a14f0 0x7fdacc0a39b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.283+0000 7fdae5259640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae0100800 0x7fdae0101e50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.283+0000 7fdae5259640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae0104c00 0x7fdae0102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.283+0000 7fdae5259640 1 --2- 172.21.15.45:0/2133411078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdae00ffe40 0x7fdae0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.283+0000 7fdae5259640 1 -- 172.21.15.45:0/2133411078 >> 172.21.15.45:0/2133411078 conn(0x7fdae0076a50 msgr2=0x7fdae0069850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.283+0000 7fdae5259640 1 -- 172.21.15.45:0/2133411078 shutdown_connections 2024-01-31T05:34:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:31.283+0000 7fdae5259640 1 -- 172.21.15.45:0/2133411078 wait complete. 2024-01-31T05:34:31.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1253739 10.48 MB/sec execute 511 sec latency 18.320 ms 2024-01-31T05:34:32.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1255171 10.47 MB/sec execute 512 sec latency 16.308 ms 2024-01-31T05:34:32.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:32 smithi156 ceph-mon[35720]: pgmap v376: 97 pgs: 97 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 170 op/s 2024-01-31T05:34:33.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:32 smithi186 ceph-mon[35659]: pgmap v376: 97 pgs: 97 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 170 op/s 2024-01-31T05:34:33.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:32 smithi045 ceph-mon[31585]: pgmap v376: 97 pgs: 97 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 170 op/s 2024-01-31T05:34:33.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1257315 10.48 MB/sec execute 513 sec latency 15.898 ms 2024-01-31T05:34:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:33 smithi156 ceph-mon[35720]: pgmap v377: 97 pgs: 97 active+clean; 568 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-01-31T05:34:34.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:33 smithi186 ceph-mon[35659]: pgmap v377: 97 pgs: 97 active+clean; 568 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-01-31T05:34:34.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:33 smithi045 ceph-mon[31585]: pgmap v377: 97 pgs: 97 active+clean; 568 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-01-31T05:34:34.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1258370 10.46 MB/sec execute 514 sec latency 111.477 ms 2024-01-31T05:34:35.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1260717 10.47 MB/sec execute 515 sec latency 68.667 ms 2024-01-31T05:34:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:35 smithi156 ceph-mon[35720]: pgmap v378: 97 pgs: 97 active+clean; 568 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 174 op/s 2024-01-31T05:34:36.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:35 smithi186 ceph-mon[35659]: pgmap v378: 97 pgs: 97 active+clean; 568 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 174 op/s 2024-01-31T05:34:36.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:35 smithi045 ceph-mon[31585]: pgmap v378: 97 pgs: 97 active+clean; 568 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 174 op/s 2024-01-31T05:34:36.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1261565 10.46 MB/sec execute 516 sec latency 74.683 ms 2024-01-31T05:34:37.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1264103 10.46 MB/sec execute 517 sec latency 15.754 ms 2024-01-31T05:34:38.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1265106 10.45 MB/sec execute 518 sec latency 15.996 ms 2024-01-31T05:34:38.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:38 smithi045 ceph-mon[31585]: pgmap v379: 97 pgs: 97 active+clean; 579 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.2 MiB/s wr, 254 op/s 2024-01-31T05:34:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:38 smithi156 ceph-mon[35720]: pgmap v379: 97 pgs: 97 active+clean; 579 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.2 MiB/s wr, 254 op/s 2024-01-31T05:34:39.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:38 smithi186 ceph-mon[35659]: pgmap v379: 97 pgs: 97 active+clean; 579 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.2 MiB/s wr, 254 op/s 2024-01-31T05:34:39.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1267504 10.46 MB/sec execute 519 sec latency 17.386 ms 2024-01-31T05:34:40.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1268376 10.45 MB/sec execute 520 sec latency 67.003 ms 2024-01-31T05:34:40.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:40 smithi045 ceph-mon[31585]: pgmap v380: 97 pgs: 97 active+clean; 579 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 166 op/s 2024-01-31T05:34:40.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:40 smithi156 ceph-mon[35720]: pgmap v380: 97 pgs: 97 active+clean; 579 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 166 op/s 2024-01-31T05:34:41.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:40 smithi186 ceph-mon[35659]: pgmap v380: 97 pgs: 97 active+clean; 579 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 166 op/s 2024-01-31T05:34:41.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1270193 10.44 MB/sec execute 521 sec latency 100.372 ms 2024-01-31T05:34:42.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1271771 10.44 MB/sec execute 522 sec latency 22.076 ms 2024-01-31T05:34:42.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:42 smithi156 ceph-mon[35720]: pgmap v381: 97 pgs: 97 active+clean; 579 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 166 op/s 2024-01-31T05:34:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:42 smithi186 ceph-mon[35659]: pgmap v381: 97 pgs: 97 active+clean; 579 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 166 op/s 2024-01-31T05:34:43.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:42 smithi045 ceph-mon[31585]: pgmap v381: 97 pgs: 97 active+clean; 579 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 166 op/s 2024-01-31T05:34:43.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1273784 10.44 MB/sec execute 523 sec latency 19.310 ms 2024-01-31T05:34:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.374+0000 7f40c7e67640 1 -- 172.21.15.45:0/932862809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40c0100f30 msgr2=0x7f40c0101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.374+0000 7f40c7e67640 1 --2- 172.21.15.45:0/932862809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40c0100f30 0x7f40c0101310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009890 tx=0x7f40b802bb70 comp rx=0 tx=0).stop 2024-01-31T05:34:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.374+0000 7f40c7e67640 1 -- 172.21.15.45:0/932862809 shutdown_connections 2024-01-31T05:34:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.374+0000 7f40c7e67640 1 --2- 172.21.15.45:0/932862809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40c0106000 0x7f40c010dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.374+0000 7f40c7e67640 1 --2- 172.21.15.45:0/932862809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40c01018e0 0x7f40c0105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.374+0000 7f40c7e67640 1 --2- 172.21.15.45:0/932862809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40c0100f30 0x7f40c0101310 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.374+0000 7f40c7e67640 1 -- 172.21.15.45:0/932862809 >> 172.21.15.45:0/932862809 conn(0x7f40c00fc7d0 msgr2=0x7f40c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.374+0000 7f40c7e67640 1 -- 172.21.15.45:0/932862809 shutdown_connections 2024-01-31T05:34:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.374+0000 7f40c7e67640 1 -- 172.21.15.45:0/932862809 wait complete. 2024-01-31T05:34:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c7e67640 1 Processor -- start 2024-01-31T05:34:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c7e67640 1 -- start start 2024-01-31T05:34:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c7e67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40c0100f30 0x7f40c0072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c7e67640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40c0106000 0x7f40c0073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c7e67640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40c0073a70 0x7f40c01a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c7e67640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c01a5a40 con 0x7f40c0100f30 2024-01-31T05:34:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c7e67640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c01a5bb0 con 0x7f40c0073a70 2024-01-31T05:34:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c7e67640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c01a5d70 con 0x7f40c0106000 2024-01-31T05:34:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c5c03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40c0100f30 0x7f40c0072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c5c03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40c0100f30 0x7f40c0072ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49610/0 (socket says 172.21.15.45:49610) 2024-01-31T05:34:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c5c03640 1 -- 172.21.15.45:0/2343748153 learned_addr learned my addr 172.21.15.45:0/2343748153 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c5402640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40c0106000 0x7f40c0073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c6404640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40c0073a70 0x7f40c01a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c5c03640 1 -- 172.21.15.45:0/2343748153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40c0106000 msgr2=0x7f40c0073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c5c03640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40c0106000 0x7f40c0073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c5c03640 1 -- 172.21.15.45:0/2343748153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40c0073a70 msgr2=0x7f40c01a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c5c03640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40c0073a70 0x7f40c01a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.375+0000 7f40c5c03640 1 -- 172.21.15.45:0/2343748153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b0009900 con 0x7f40c0100f30 2024-01-31T05:34:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.376+0000 7f40c5c03640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40c0100f30 0x7f40c0072ff0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f40b8035fd0 tx=0x7f40b8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.376+0000 7f40beffd640 1 -- 172.21.15.45:0/2343748153 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8034850 con 0x7f40c0100f30 2024-01-31T05:34:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.376+0000 7f40c7e67640 1 -- 172.21.15.45:0/2343748153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b8009490 con 0x7f40c0100f30 2024-01-31T05:34:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.376+0000 7f40beffd640 1 -- 172.21.15.45:0/2343748153 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b802c9d0 con 0x7f40c0100f30 2024-01-31T05:34:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.376+0000 7f40beffd640 1 -- 172.21.15.45:0/2343748153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8032e30 con 0x7f40c0100f30 2024-01-31T05:34:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.376+0000 7f40c7e67640 1 -- 172.21.15.45:0/2343748153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c01a6290 con 0x7f40c0100f30 2024-01-31T05:34:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.378+0000 7f40beffd640 1 -- 172.21.15.45:0/2343748153 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f40b803b040 con 0x7f40c0100f30 2024-01-31T05:34:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40beffd640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f40a80a1520 0x7f40a80a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40cac2d740 1 Processor -- start 2024-01-31T05:34:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40beffd640 1 -- 172.21.15.45:0/2343748153 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f40b80ee740 con 0x7f40c0100f30 2024-01-31T05:34:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40c5402640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f40a80a1520 0x7f40a80a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40cac2d740 1 -- start start 2024-01-31T05:34:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40cac2d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf0870 0x55f086bf0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40cac2d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf1190 0x55f086bf9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40cac2d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 0x55f086bfc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40cac2d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f086b57040 con 0x55f086bf9f60 2024-01-31T05:34:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40cac2d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f086b54730 con 0x55f086bf1190 2024-01-31T05:34:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40cac2d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f086b50020 con 0x55f086bf0870 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40c5402640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f40a80a1520 0x7f40a80a39e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f40c0074a80 tx=0x7f40b4005f90 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40c6404640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 0x55f086bfc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40c6404640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 0x55f086bfc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49626/0 (socket says 172.21.15.45:49626) 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40c6404640 1 -- 172.21.15.45:0/1887157914 learned_addr learned my addr 172.21.15.45:0/1887157914 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40c5c03640 1 --2- 172.21.15.45:0/1887157914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf0870 0x55f086bf0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.379+0000 7f40c5402640 1 --2- 172.21.15.45:0/1887157914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf1190 0x55f086bf9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.380+0000 7f40c6404640 1 -- 172.21.15.45:0/1887157914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf0870 msgr2=0x55f086bf0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.380+0000 7f40c6404640 1 --2- 172.21.15.45:0/1887157914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf0870 0x55f086bf0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.380+0000 7f40c6404640 1 -- 172.21.15.45:0/1887157914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf1190 msgr2=0x55f086bf9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.380+0000 7f40c6404640 1 --2- 172.21.15.45:0/1887157914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf1190 0x55f086bf9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.380+0000 7f40c6404640 1 -- 172.21.15.45:0/1887157914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f086b47e40 con 0x55f086bf9f60 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.380+0000 7f40c5c03640 1 --2- 172.21.15.45:0/1887157914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf0870 0x55f086bf0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:34:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.380+0000 7f40c5402640 1 --2- 172.21.15.45:0/1887157914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf1190 0x55f086bf9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:34:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.380+0000 7f40c6404640 1 --2- 172.21.15.45:0/1887157914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 0x55f086bfc350 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009fd0 tx=0x7f40b00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5329f61483b2d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.380+0000 7f40a7fff640 1 -- 172.21.15.45:0/1887157914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0013040 con 0x55f086bf9f60 2024-01-31T05:34:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.380+0000 7f40a7fff640 1 -- 172.21.15.45:0/1887157914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b0003e40 con 0x55f086bf9f60 2024-01-31T05:34:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.380+0000 7f40a7fff640 1 -- 172.21.15.45:0/1887157914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b000db40 con 0x55f086bf9f60 2024-01-31T05:34:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.381+0000 7f40cac2d740 1 -- 172.21.15.45:0/1887157914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 msgr2=0x55f086bfc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.381+0000 7f40cac2d740 1 --2- 172.21.15.45:0/1887157914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 0x55f086bfc350 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009fd0 tx=0x7f40b00095c0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.381+0000 7f40cac2d740 1 -- 172.21.15.45:0/1887157914 shutdown_connections 2024-01-31T05:34:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.381+0000 7f40cac2d740 1 --2- 172.21.15.45:0/1887157914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 0x55f086bfc350 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.381+0000 7f40cac2d740 1 --2- 172.21.15.45:0/1887157914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf1190 0x55f086bf9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.381+0000 7f40cac2d740 1 --2- 172.21.15.45:0/1887157914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf0870 0x55f086bf0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.381+0000 7f40cac2d740 1 -- 172.21.15.45:0/1887157914 >> 172.21.15.45:0/1887157914 conn(0x55f086beb190 msgr2=0x55f086beb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.381+0000 7f40cac2d740 1 -- 172.21.15.45:0/1887157914 shutdown_connections 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.381+0000 7f40cac2d740 1 -- 172.21.15.45:0/1887157914 wait complete. 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.381+0000 7f40cac2d740 1 -- start start 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.381+0000 7f40cac2d740 1 Processor -- start 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40cac2d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf0870 0x55f086bf6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40cac2d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf1190 0x55f086bf7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40cac2d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf9f60 0x55f086bf7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40cac2d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f086b091b0 con 0x55f086bf1190 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40cac2d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f086b08e50 con 0x55f086bf0870 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40cac2d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f086ace090 con 0x55f086bf9f60 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40cac2d740 10 client.0 _ll_register_callbacks cb 0x55f086af3a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40c5402640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf1190 0x55f086bf7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40c5402640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf1190 0x55f086bf7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49640/0 (socket says 172.21.15.45:49640) 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40c5402640 1 -- 172.21.15.45:0/3311102846 learned_addr learned my addr 172.21.15.45:0/3311102846 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40c5c03640 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf0870 0x55f086bf6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40c6404640 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf9f60 0x55f086bf7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40c5402640 1 -- 172.21.15.45:0/3311102846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf9f60 msgr2=0x55f086bf7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40c5402640 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf9f60 0x55f086bf7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40c5402640 1 -- 172.21.15.45:0/3311102846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf0870 msgr2=0x55f086bf6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40c5402640 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf0870 0x55f086bf6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40c5402640 1 -- 172.21.15.45:0/3311102846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b0009db0 con 0x55f086bf1190 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40c5402640 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf1190 0x55f086bf7440 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f40b40039c0 tx=0x7f40b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a276c10dbb206a3d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40a57fa640 1 -- 172.21.15.45:0/3311102846 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b401eb40 con 0x55f086bf1190 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40a57fa640 1 -- 172.21.15.45:0/3311102846 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f40b401eca0 con 0x55f086bf1190 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.382+0000 7f40cac2d740 1 -- 172.21.15.45:0/3311102846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f086b49010 con 0x55f086bf1190 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40a57fa640 1 -- 172.21.15.45:0/3311102846 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b401eea0 con 0x55f086bf1190 2024-01-31T05:34:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40cac2d740 10 client.16374 fetch_fsmap learned FSMap version 30 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40cac2d740 1 -- 172.21.15.45:0/3311102846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f086b4e6a0 con 0x55f086bf1190 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40a57fa640 1 -- 172.21.15.45:0/3311102846 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f086b49010 con 0x55f086bf1190 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40a57fa640 1 -- 172.21.15.45:0/3311102846 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f40b401f4c0 con 0x55f086bf1190 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40cac2d740 10 client.16374 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40cac2d740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40cac2d740 20 client.16374 populate_metadata read hostname 'smithi045' 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40cac2d740 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f086c09e80 0x55f086c0c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40cac2d740 4 client.16374 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40cac2d740 1 -- 172.21.15.45:0/3311102846 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f086b48d30 con 0x55f086c09e80 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40c5c03640 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f086c09e80 0x55f086c0c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40a57fa640 10 client.16374 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.383+0000 7f40c5c03640 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f086c09e80 0x55f086c0c2a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37186fcaee45ed7e server_cookie=388461961434b4a1 in_seq=0 out_seq=0 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40a57fa640 1 -- 172.21.15.45:0/3311102846 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f086b48d30 con 0x55f086c09e80 2024-01-31T05:34:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40a57fa640 10 client.16374 handle_command_reply: tid=0 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40cac2d740 1 client.16374 shutdown 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40cac2d740 1 -- 172.21.15.45:0/3311102846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf1190 msgr2=0x55f086bf7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40cac2d740 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf1190 0x55f086bf7440 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f40b40039c0 tx=0x7f40b4004730 comp rx=0 tx=0).stop 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40cac2d740 1 -- 172.21.15.45:0/3311102846 shutdown_connections 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40cac2d740 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f086c09e80 0x55f086c0c2a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40cac2d740 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf1190 0x55f086bf7440 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40cac2d740 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf9f60 0x55f086bf7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40cac2d740 1 --2- 172.21.15.45:0/3311102846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf0870 0x55f086bf6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40a57fa640 0 client.16374 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40cac2d740 1 -- 172.21.15.45:0/3311102846 >> 172.21.15.45:0/3311102846 conn(0x55f086beb190 msgr2=0x55f086becee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40cac2d740 1 -- 172.21.15.45:0/3311102846 shutdown_connections 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40cac2d740 1 -- 172.21.15.45:0/3311102846 wait complete. 2024-01-31T05:34:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.384+0000 7f40cac2d740 20 client.16374 trim_cache size 0 max 16384 2024-01-31T05:34:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.397+0000 7f40cac2d740 1 Processor -- start 2024-01-31T05:34:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40cac2d740 1 -- start start 2024-01-31T05:34:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40cac2d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf0870 0x55f086bf77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40cac2d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf1190 0x55f086beccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40cac2d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 0x55f086c00c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40cac2d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f086bec1c0 con 0x55f086bf9f60 2024-01-31T05:34:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40cac2d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f086bebfc0 con 0x55f086bf0870 2024-01-31T05:34:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40cac2d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f086c11530 con 0x55f086bf1190 2024-01-31T05:34:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40c6404640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 0x55f086c00c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40c6404640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 0x55f086c00c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49656/0 (socket says 172.21.15.45:49656) 2024-01-31T05:34:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40c6404640 1 -- 172.21.15.45:0/3233907479 learned_addr learned my addr 172.21.15.45:0/3233907479 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40c5402640 1 --2- 172.21.15.45:0/3233907479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf1190 0x55f086beccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40c5c03640 1 --2- 172.21.15.45:0/3233907479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf0870 0x55f086bf77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40c6404640 1 -- 172.21.15.45:0/3233907479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf1190 msgr2=0x55f086beccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40c6404640 1 --2- 172.21.15.45:0/3233907479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf1190 0x55f086beccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40c6404640 1 -- 172.21.15.45:0/3233907479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf0870 msgr2=0x55f086bf77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40c6404640 1 --2- 172.21.15.45:0/3233907479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf0870 0x55f086bf77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40c6404640 1 -- 172.21.15.45:0/3233907479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f086aeb030 con 0x55f086bf9f60 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40c5c03640 1 --2- 172.21.15.45:0/3233907479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf0870 0x55f086bf77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40c6404640 1 --2- 172.21.15.45:0/3233907479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 0x55f086c00c50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f40b0006010 tx=0x7f40b00097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fba211caafdb9b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40a77fe640 1 -- 172.21.15.45:0/3233907479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b000d780 con 0x55f086bf9f60 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40a77fe640 1 -- 172.21.15.45:0/3233907479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b000dda0 con 0x55f086bf9f60 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.398+0000 7f40a77fe640 1 -- 172.21.15.45:0/3233907479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0005920 con 0x55f086bf9f60 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.399+0000 7f40cac2d740 1 -- 172.21.15.45:0/3233907479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 msgr2=0x55f086c00c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.399+0000 7f40cac2d740 1 --2- 172.21.15.45:0/3233907479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 0x55f086c00c50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f40b0006010 tx=0x7f40b00097e0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.399+0000 7f40cac2d740 1 -- 172.21.15.45:0/3233907479 shutdown_connections 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.399+0000 7f40cac2d740 1 --2- 172.21.15.45:0/3233907479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf9f60 0x55f086c00c50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.399+0000 7f40cac2d740 1 --2- 172.21.15.45:0/3233907479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf1190 0x55f086beccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.399+0000 7f40cac2d740 1 --2- 172.21.15.45:0/3233907479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf0870 0x55f086bf77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.399+0000 7f40cac2d740 1 -- 172.21.15.45:0/3233907479 >> 172.21.15.45:0/3233907479 conn(0x55f086beb190 msgr2=0x55f086bec5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.399+0000 7f40cac2d740 1 -- 172.21.15.45:0/3233907479 shutdown_connections 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.399+0000 7f40cac2d740 1 -- 172.21.15.45:0/3233907479 wait complete. 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.399+0000 7f40cac2d740 1 -- start start 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.399+0000 7f40cac2d740 1 Processor -- start 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40cac2d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf0870 0x55f086c5aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40cac2d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf1190 0x55f086c5af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40cac2d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf9f60 0x55f086c001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40cac2d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f086c007b0 con 0x55f086bf0870 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40cac2d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f086c00920 con 0x55f086bf9f60 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40cac2d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f086c00ae0 con 0x55f086bf1190 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40cac2d740 10 client.0 _ll_register_callbacks cb 0x7f40b0009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40c5c03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf0870 0x55f086c5aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40c5c03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf0870 0x55f086c5aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49658/0 (socket says 172.21.15.45:49658) 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40c5402640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf1190 0x55f086c5af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40c5c03640 1 -- 172.21.15.45:0/2222819398 learned_addr learned my addr 172.21.15.45:0/2222819398 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40c6404640 1 --2- 172.21.15.45:0/2222819398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf9f60 0x55f086c001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40c5c03640 1 -- 172.21.15.45:0/2222819398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf1190 msgr2=0x55f086c5af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40c5c03640 1 --2- 172.21.15.45:0/2222819398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf1190 0x55f086c5af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40c5c03640 1 -- 172.21.15.45:0/2222819398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf9f60 msgr2=0x55f086c001f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40c5c03640 1 --2- 172.21.15.45:0/2222819398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf9f60 0x55f086c001f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40c5c03640 1 -- 172.21.15.45:0/2222819398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f086b47e40 con 0x55f086bf0870 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40c5c03640 1 --2- 172.21.15.45:0/2222819398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf0870 0x55f086c5aa10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f40b80ec020 tx=0x7f40b80a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95235b20b3dc9f10 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.400+0000 7f40a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40a57fa640 1 -- 172.21.15.45:0/2222819398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b80bf7c0 con 0x55f086bf0870 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40a57fa640 1 -- 172.21.15.45:0/2222819398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f40b80c3a00 con 0x55f086bf0870 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40cac2d740 1 -- 172.21.15.45:0/2222819398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40b0003e40 con 0x55f086bf0870 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40a57fa640 1 -- 172.21.15.45:0/2222819398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b80a60b0 con 0x55f086bf0870 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40cac2d740 10 client.16380 fetch_fsmap learned FSMap version 30 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40cac2d740 1 -- 172.21.15.45:0/2222819398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f086b10a50 con 0x55f086bf0870 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40a57fa640 1 -- 172.21.15.45:0/2222819398 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b0003e40 con 0x55f086bf0870 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40a57fa640 1 -- 172.21.15.45:0/2222819398 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f40b80d0070 con 0x55f086bf0870 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40cac2d740 10 client.16380 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40cac2d740 10 client.16380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:34:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40cac2d740 20 client.16380 populate_metadata read hostname 'smithi045' 2024-01-31T05:34:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40cac2d740 1 --2- 172.21.15.45:0/2222819398 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f086c62440 0x55f086c64830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40cac2d740 4 client.16380 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:34:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40cac2d740 1 -- 172.21.15.45:0/2222819398 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f08684f600 con 0x55f086c62440 2024-01-31T05:34:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.401+0000 7f40c5402640 1 --2- 172.21.15.45:0/2222819398 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f086c62440 0x55f086c64830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.402+0000 7f40c5402640 1 --2- 172.21.15.45:0/2222819398 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f086c62440 0x55f086c64830 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a57fb1372e9b15d9 server_cookie=3cffc73904e28358 in_seq=0 out_seq=0 2024-01-31T05:34:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.402+0000 7f40a57fa640 10 client.16380 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.414+0000 7f40a57fa640 1 -- 172.21.15.45:0/2222819398 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f08684f600 con 0x55f086c62440 2024-01-31T05:34:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.414+0000 7f40a57fa640 10 client.16380 handle_command_reply: tid=0 2024-01-31T05:34:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.414+0000 7f40cac2d740 1 client.16380 shutdown 2024-01-31T05:34:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.414+0000 7f40cac2d740 1 -- 172.21.15.45:0/2222819398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf0870 msgr2=0x55f086c5aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.414+0000 7f40cac2d740 1 --2- 172.21.15.45:0/2222819398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf0870 0x55f086c5aa10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f40b80ec020 tx=0x7f40b80a6040 comp rx=0 tx=0).stop 2024-01-31T05:34:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40cac2d740 1 -- 172.21.15.45:0/2222819398 shutdown_connections 2024-01-31T05:34:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40cac2d740 1 --2- 172.21.15.45:0/2222819398 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55f086c62440 0x55f086c64830 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40cac2d740 1 --2- 172.21.15.45:0/2222819398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f086bf9f60 0x55f086c001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40a57fa640 0 client.16380 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40cac2d740 1 --2- 172.21.15.45:0/2222819398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f086bf1190 0x55f086c5af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40cac2d740 1 --2- 172.21.15.45:0/2222819398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f086bf0870 0x55f086c5aa10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40cac2d740 1 -- 172.21.15.45:0/2222819398 >> 172.21.15.45:0/2222819398 conn(0x55f086beb190 msgr2=0x55f086be1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40cac2d740 1 -- 172.21.15.45:0/2222819398 shutdown_connections 2024-01-31T05:34:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40cac2d740 1 -- 172.21.15.45:0/2222819398 wait complete. 2024-01-31T05:34:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40cac2d740 20 client.16380 trim_cache size 0 max 16384 2024-01-31T05:34:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:34:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:34:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:34:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:34:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40c7e67640 1 -- 172.21.15.45:0/2343748153 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f40a80a1520 msgr2=0x7f40a80a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40c7e67640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f40a80a1520 0x7f40a80a39e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f40c0074a80 tx=0x7f40b4005f90 comp rx=0 tx=0).stop 2024-01-31T05:34:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40c7e67640 1 -- 172.21.15.45:0/2343748153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40c0100f30 msgr2=0x7f40c0072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40c7e67640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40c0100f30 0x7f40c0072ff0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f40b8035fd0 tx=0x7f40b8004360 comp rx=0 tx=0).stop 2024-01-31T05:34:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40c7e67640 1 -- 172.21.15.45:0/2343748153 shutdown_connections 2024-01-31T05:34:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40c7e67640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f40a80a1520 0x7f40a80a39e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40c7e67640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40c0073a70 0x7f40c01a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40c7e67640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40c0106000 0x7f40c0073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40c7e67640 1 --2- 172.21.15.45:0/2343748153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40c0100f30 0x7f40c0072ff0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40c7e67640 1 -- 172.21.15.45:0/2343748153 >> 172.21.15.45:0/2343748153 conn(0x7f40c00fc7d0 msgr2=0x7f40c00feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40c7e67640 1 -- 172.21.15.45:0/2343748153 shutdown_connections 2024-01-31T05:34:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:44.415+0000 7f40c7e67640 1 -- 172.21.15.45:0/2343748153 wait complete. 2024-01-31T05:34:44.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1275261 10.44 MB/sec execute 524 sec latency 52.296 ms 2024-01-31T05:34:44.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:44 smithi156 ceph-mon[35720]: pgmap v382: 97 pgs: 97 active+clean; 589 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 246 op/s 2024-01-31T05:34:45.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:44 smithi186 ceph-mon[35659]: pgmap v382: 97 pgs: 97 active+clean; 589 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 246 op/s 2024-01-31T05:34:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:44 smithi045 ceph-mon[31585]: pgmap v382: 97 pgs: 97 active+clean; 589 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 246 op/s 2024-01-31T05:34:45.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1277144 10.43 MB/sec execute 525 sec latency 19.033 ms 2024-01-31T05:34:46.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1278720 10.44 MB/sec execute 526 sec latency 38.366 ms 2024-01-31T05:34:46.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:46 smithi156 ceph-mon[35720]: pgmap v383: 97 pgs: 97 active+clean; 589 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 160 op/s 2024-01-31T05:34:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:46 smithi186 ceph-mon[35659]: pgmap v383: 97 pgs: 97 active+clean; 589 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 160 op/s 2024-01-31T05:34:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:46 smithi045 ceph-mon[31585]: pgmap v383: 97 pgs: 97 active+clean; 589 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 160 op/s 2024-01-31T05:34:47.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1280570 10.43 MB/sec execute 527 sec latency 19.005 ms 2024-01-31T05:34:48.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1282328 10.43 MB/sec execute 528 sec latency 17.526 ms 2024-01-31T05:34:48.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:48 smithi045 ceph-mon[31585]: pgmap v384: 97 pgs: 97 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 250 op/s 2024-01-31T05:34:48.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:48 smithi156 ceph-mon[35720]: pgmap v384: 97 pgs: 97 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 250 op/s 2024-01-31T05:34:49.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:48 smithi186 ceph-mon[35659]: pgmap v384: 97 pgs: 97 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 250 op/s 2024-01-31T05:34:49.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1284169 10.43 MB/sec execute 529 sec latency 28.676 ms 2024-01-31T05:34:50.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1285941 10.43 MB/sec execute 530 sec latency 19.131 ms 2024-01-31T05:34:50.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:50 smithi156 ceph-mon[35720]: pgmap v385: 97 pgs: 97 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 170 op/s 2024-01-31T05:34:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:50 smithi186 ceph-mon[35659]: pgmap v385: 97 pgs: 97 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 170 op/s 2024-01-31T05:34:51.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:50 smithi045 ceph-mon[31585]: pgmap v385: 97 pgs: 97 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 170 op/s 2024-01-31T05:34:51.801 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1287275 10.42 MB/sec execute 531 sec latency 101.882 ms 2024-01-31T05:34:52.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1289267 10.43 MB/sec execute 532 sec latency 15.452 ms 2024-01-31T05:34:52.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:52 smithi156 ceph-mon[35720]: pgmap v386: 97 pgs: 97 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 170 op/s 2024-01-31T05:34:53.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:52 smithi186 ceph-mon[35659]: pgmap v386: 97 pgs: 97 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 170 op/s 2024-01-31T05:34:53.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:52 smithi045 ceph-mon[31585]: pgmap v386: 97 pgs: 97 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 170 op/s 2024-01-31T05:34:53.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1290787 10.42 MB/sec execute 533 sec latency 21.054 ms 2024-01-31T05:34:53.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:53 smithi156 ceph-mon[35720]: pgmap v387: 97 pgs: 97 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 251 op/s 2024-01-31T05:34:54.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:53 smithi186 ceph-mon[35659]: pgmap v387: 97 pgs: 97 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 251 op/s 2024-01-31T05:34:54.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:53 smithi045 ceph-mon[31585]: pgmap v387: 97 pgs: 97 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 251 op/s 2024-01-31T05:34:54.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1292803 10.42 MB/sec execute 534 sec latency 19.832 ms 2024-01-31T05:34:55.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1294348 10.41 MB/sec execute 535 sec latency 16.949 ms 2024-01-31T05:34:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:55 smithi156 ceph-mon[35720]: pgmap v388: 97 pgs: 97 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 171 op/s 2024-01-31T05:34:56.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:55 smithi186 ceph-mon[35659]: pgmap v388: 97 pgs: 97 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 171 op/s 2024-01-31T05:34:56.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:55 smithi045 ceph-mon[31585]: pgmap v388: 97 pgs: 97 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 171 op/s 2024-01-31T05:34:56.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1296364 10.42 MB/sec execute 536 sec latency 52.809 ms 2024-01-31T05:34:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.508+0000 7ff1a73ee640 1 -- 172.21.15.45:0/55882898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a00ffe10 msgr2=0x7ff1a0105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.508+0000 7ff1a73ee640 1 --2- 172.21.15.45:0/55882898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a00ffe10 0x7ff1a0105940 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff190009900 tx=0x7ff19002bc30 comp rx=0 tx=0).stop 2024-01-31T05:34:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.508+0000 7ff1a73ee640 1 -- 172.21.15.45:0/55882898 shutdown_connections 2024-01-31T05:34:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.508+0000 7ff1a73ee640 1 --2- 172.21.15.45:0/55882898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a00ffe10 0x7ff1a0105940 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.508+0000 7ff1a73ee640 1 --2- 172.21.15.45:0/55882898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1a0106070 0x7ff1a010dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.508+0000 7ff1a73ee640 1 --2- 172.21.15.45:0/55882898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a00ff460 0x7ff1a00ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.508+0000 7ff1a73ee640 1 -- 172.21.15.45:0/55882898 >> 172.21.15.45:0/55882898 conn(0x7ff1a0076180 msgr2=0x7ff1a00fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.508+0000 7ff1a73ee640 1 -- 172.21.15.45:0/55882898 shutdown_connections 2024-01-31T05:34:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.508+0000 7ff1a73ee640 1 -- 172.21.15.45:0/55882898 wait complete. 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.508+0000 7ff1a73ee640 1 Processor -- start 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.508+0000 7ff1a73ee640 1 -- start start 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a73ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a00ff460 0x7ff1a01971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a73ee640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1a0106070 0x7ff1a01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a73ee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a019ba60 0x7ff1a019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a73ee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a019c450 con 0x7ff1a00ff460 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a73ee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a019c5c0 con 0x7ff1a019ba60 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a73ee640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a019c730 con 0x7ff1a0106070 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a518a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a00ff460 0x7ff1a01971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a518a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a00ff460 0x7ff1a01971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49582/0 (socket says 172.21.15.45:49582) 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a518a640 1 -- 172.21.15.45:0/3467864508 learned_addr learned my addr 172.21.15.45:0/3467864508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a4989640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1a0106070 0x7ff1a01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a598b640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a019ba60 0x7ff1a019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a518a640 1 -- 172.21.15.45:0/3467864508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1a0106070 msgr2=0x7ff1a01976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a518a640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1a0106070 0x7ff1a01976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a518a640 1 -- 172.21.15.45:0/3467864508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a019ba60 msgr2=0x7ff1a019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a518a640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a019ba60 0x7ff1a019bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a518a640 1 -- 172.21.15.45:0/3467864508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c009490 con 0x7ff1a00ff460 2024-01-31T05:34:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a518a640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a00ff460 0x7ff1a01971a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00ea60 tx=0x7ff19c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1967fc640 1 -- 172.21.15.45:0/3467864508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c00cde0 con 0x7ff1a00ff460 2024-01-31T05:34:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.509+0000 7ff1a73ee640 1 -- 172.21.15.45:0/3467864508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff190009560 con 0x7ff1a00ff460 2024-01-31T05:34:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.510+0000 7ff1967fc640 1 -- 172.21.15.45:0/3467864508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c0035f0 con 0x7ff1a00ff460 2024-01-31T05:34:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.510+0000 7ff1967fc640 1 -- 172.21.15.45:0/3467864508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0044e0 con 0x7ff1a00ff460 2024-01-31T05:34:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.510+0000 7ff1a73ee640 1 -- 172.21.15.45:0/3467864508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a01a4050 con 0x7ff1a00ff460 2024-01-31T05:34:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.512+0000 7ff1967fc640 1 -- 172.21.15.45:0/3467864508 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7ff19c003a60 con 0x7ff1a00ff460 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.512+0000 7ff1aa1b1740 1 Processor -- start 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.512+0000 7ff1aa1b1740 1 -- start start 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1aa1b1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5b98440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1aa1b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5ba1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1aa1b1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 0x55fde5ba3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1967fc640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff18c0a14f0 0x7ff18c0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1aa1b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde5b07040 con 0x55fde5b98980 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1aa1b1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde5b04730 con 0x55fde5b98060 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1aa1b1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde5b00020 con 0x55fde5ba1750 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1a598b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5b98440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1a598b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5b98440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:37326/0 (socket says 172.21.15.45:37326) 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1a598b640 1 -- 172.21.15.45:0/1288442508 learned_addr learned my addr 172.21.15.45:0/1288442508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1967fc640 1 -- 172.21.15.45:0/3467864508 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff19c014070 con 0x7ff1a00ff460 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1a518a640 1 --2- 172.21.15.45:0/1288442508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5ba1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1a598b640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff18c0a14f0 0x7ff18c0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1a4989640 1 --2- 172.21.15.45:0/1288442508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 0x55fde5ba3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1a598b640 1 -- 172.21.15.45:0/1288442508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 msgr2=0x55fde5ba3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1a598b640 1 --2- 172.21.15.45:0/1288442508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 0x55fde5ba3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1a598b640 1 -- 172.21.15.45:0/1288442508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 msgr2=0x55fde5ba1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1a598b640 1 --2- 172.21.15.45:0/1288442508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5ba1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.513+0000 7ff1a598b640 1 -- 172.21.15.45:0/1288442508 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fde5af7e40 con 0x55fde5b98060 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.514+0000 7ff1a598b640 1 --2- 172.21.15.45:0/1288442508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5b98440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff198011e00 tx=0x7ff19800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc8e9bed3b070de1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.514+0000 7ff1a598b640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff18c0a14f0 0x7ff18c0a39b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff19800fc90 tx=0x7ff198018000 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.514+0000 7ff18b7fe640 1 -- 172.21.15.45:0/1288442508 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198009ea0 con 0x55fde5b98060 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.514+0000 7ff18b7fe640 1 -- 172.21.15.45:0/1288442508 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19800ed10 con 0x55fde5b98060 2024-01-31T05:34:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.514+0000 7ff18b7fe640 1 -- 172.21.15.45:0/1288442508 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1980058c0 con 0x55fde5b98060 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.514+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/1288442508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 msgr2=0x55fde5b98440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.514+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/1288442508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5b98440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff198011e00 tx=0x7ff19800f790 comp rx=0 tx=0).stop 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/1288442508 shutdown_connections 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/1288442508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5ba1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/1288442508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 0x55fde5ba3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/1288442508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5b98440 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/1288442508 >> 172.21.15.45:0/1288442508 conn(0x55fde5b92980 msgr2=0x55fde5b92d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/1288442508 shutdown_connections 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/1288442508 wait complete. 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 -- start start 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 Processor -- start 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5b9e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5b9ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 0x55fde5b9f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde5ab91b0 con 0x55fde5b98980 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde5ab8e50 con 0x55fde5b98060 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde5a7e090 con 0x55fde5ba1750 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1aa1b1740 10 client.0 _ll_register_callbacks cb 0x55fde5aa3a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.515+0000 7ff1a4989640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5b9ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1a518a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5b9e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1a4989640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5b9ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49590/0 (socket says 172.21.15.45:49590) 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1a4989640 1 -- 172.21.15.45:0/4028477543 learned_addr learned my addr 172.21.15.45:0/4028477543 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1a598b640 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 0x55fde5b9f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1a4989640 1 -- 172.21.15.45:0/4028477543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 msgr2=0x55fde5b9f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1a4989640 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 0x55fde5b9f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1a4989640 1 -- 172.21.15.45:0/4028477543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 msgr2=0x55fde5b9e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1a4989640 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5b9e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1a4989640 1 -- 172.21.15.45:0/4028477543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff198019040 con 0x55fde5b98980 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1a518a640 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5b9e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1a4989640 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5b9ecf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff190006010 tx=0x7ff19002c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1e9ea6c20a1137 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff188ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff188ff9640 1 -- 172.21.15.45:0/4028477543 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190004dd0 con 0x55fde5b98980 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff188ff9640 1 -- 172.21.15.45:0/4028477543 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff190045070 con 0x55fde5b98980 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/4028477543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fde5af9010 con 0x55fde5b98980 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff188ff9640 1 -- 172.21.15.45:0/4028477543 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff190004040 con 0x55fde5b98980 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1aa1b1740 10 client.16392 fetch_fsmap learned FSMap version 30 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.516+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/4028477543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fde5afe6a0 con 0x55fde5b98980 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.517+0000 7ff188ff9640 1 -- 172.21.15.45:0/4028477543 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fde5af9010 con 0x55fde5b98980 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.517+0000 7ff188ff9640 1 -- 172.21.15.45:0/4028477543 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff19003c040 con 0x55fde5b98980 2024-01-31T05:34:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.517+0000 7ff1aa1b1740 10 client.16392 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.517+0000 7ff1aa1b1740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.517+0000 7ff1aa1b1740 20 client.16392 populate_metadata read hostname 'smithi045' 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.517+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fde5bb5970 0x55fde5bb7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.517+0000 7ff1aa1b1740 4 client.16392 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.517+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/4028477543 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fde5af8d30 con 0x55fde5bb5970 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.517+0000 7ff1a518a640 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fde5bb5970 0x55fde5bb7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.517+0000 7ff188ff9640 10 client.16392 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.517+0000 7ff1a518a640 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fde5bb5970 0x55fde5bb7d90 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=648b29afc6908a9b server_cookie=673835a099b25404 in_seq=0 out_seq=0 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff188ff9640 1 -- 172.21.15.45:0/4028477543 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fde5af8d30 con 0x55fde5bb5970 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff188ff9640 10 client.16392 handle_command_reply: tid=0 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff1aa1b1740 1 client.16392 shutdown 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/4028477543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 msgr2=0x55fde5b9ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5b9ecf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff190006010 tx=0x7ff19002c9b0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/4028477543 shutdown_connections 2024-01-31T05:34:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fde5bb5970 0x55fde5bb7d90 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5b9ecf0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 0x55fde5b9f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff188ff9640 0 client.16392 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/4028477543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5b9e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/4028477543 >> 172.21.15.45:0/4028477543 conn(0x55fde5b92980 msgr2=0x55fde5b946d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/4028477543 shutdown_connections 2024-01-31T05:34:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/4028477543 wait complete. 2024-01-31T05:34:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.518+0000 7ff1aa1b1740 20 client.16392 trim_cache size 0 max 16384 2024-01-31T05:34:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.537+0000 7ff1aa1b1740 1 Processor -- start 2024-01-31T05:34:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.537+0000 7ff1aa1b1740 1 -- start start 2024-01-31T05:34:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.537+0000 7ff1aa1b1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5ba0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.537+0000 7ff1aa1b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5b9f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.537+0000 7ff1aa1b1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 0x55fde5b9ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.537+0000 7ff1aa1b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde5af9010 con 0x55fde5b98980 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.537+0000 7ff1aa1b1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde5bb24b0 con 0x55fde5b98060 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.537+0000 7ff1aa1b1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde5b94c90 con 0x55fde5ba1750 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a4989640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5b9f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a518a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5ba0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a4989640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5b9f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49602/0 (socket says 172.21.15.45:49602) 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a4989640 1 -- 172.21.15.45:0/1598341313 learned_addr learned my addr 172.21.15.45:0/1598341313 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a518a640 1 --2- 172.21.15.45:0/1598341313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5ba0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:37344/0 (socket says 172.21.15.45:37344) 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a598b640 1 --2- 172.21.15.45:0/1598341313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 0x55fde5b9ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a4989640 1 -- 172.21.15.45:0/1598341313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 msgr2=0x55fde5b9ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a4989640 1 --2- 172.21.15.45:0/1598341313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 0x55fde5b9ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a4989640 1 -- 172.21.15.45:0/1598341313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 msgr2=0x55fde5ba0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a4989640 1 --2- 172.21.15.45:0/1598341313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5ba0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a4989640 1 -- 172.21.15.45:0/1598341313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fde5a9b030 con 0x55fde5b98980 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a518a640 1 --2- 172.21.15.45:0/1598341313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5ba0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff1a4989640 1 --2- 172.21.15.45:0/1598341313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5b9f050 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff19002c170 tx=0x7ff19002cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3592913efcb1f376 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff18affd640 1 -- 172.21.15.45:0/1598341313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190057070 con 0x55fde5b98980 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff18affd640 1 -- 172.21.15.45:0/1598341313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff190043070 con 0x55fde5b98980 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.538+0000 7ff18affd640 1 -- 172.21.15.45:0/1598341313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190003b70 con 0x55fde5b98980 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.539+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/1598341313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 msgr2=0x55fde5b9f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.539+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/1598341313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5b9f050 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff19002c170 tx=0x7ff19002cd30 comp rx=0 tx=0).stop 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.539+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/1598341313 shutdown_connections 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.539+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/1598341313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5b98980 0x55fde5b9f050 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.539+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/1598341313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5ba1750 0x55fde5b9ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.539+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/1598341313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98060 0x55fde5ba0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.539+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/1598341313 >> 172.21.15.45:0/1598341313 conn(0x55fde5b92980 msgr2=0x55fde5b9d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.539+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/1598341313 shutdown_connections 2024-01-31T05:34:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.539+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/1598341313 wait complete. 2024-01-31T05:34:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1aa1b1740 1 -- start start 2024-01-31T05:34:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1aa1b1740 1 Processor -- start 2024-01-31T05:34:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1aa1b1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5b98060 0x55fde5bab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1aa1b1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98980 0x55fde5bab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1aa1b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5ba1750 0x55fde5babcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1aa1b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde5c0bec0 con 0x55fde5ba1750 2024-01-31T05:34:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1aa1b1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde5c0c000 con 0x55fde5b98980 2024-01-31T05:34:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1aa1b1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde5c0c140 con 0x55fde5b98060 2024-01-31T05:34:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1aa1b1740 10 client.0 _ll_register_callbacks cb 0x7ff198011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:34:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1a598b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5ba1750 0x55fde5babcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1a518a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5b98060 0x55fde5bab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1a4989640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98980 0x55fde5bab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1a518a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5b98060 0x55fde5bab270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:58820/0 (socket says 172.21.15.45:58820) 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1a4989640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98980 0x55fde5bab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:37348/0 (socket says 172.21.15.45:37348) 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.540+0000 7ff1a518a640 1 -- 172.21.15.45:0/972633191 learned_addr learned my addr 172.21.15.45:0/972633191 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff1a518a640 1 -- 172.21.15.45:0/972633191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98980 msgr2=0x55fde5bab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff1a518a640 1 --2- 172.21.15.45:0/972633191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98980 0x55fde5bab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff1a518a640 1 -- 172.21.15.45:0/972633191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5ba1750 msgr2=0x55fde5babcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff1a518a640 1 --2- 172.21.15.45:0/972633191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5ba1750 0x55fde5babcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff1a518a640 1 -- 172.21.15.45:0/972633191 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a01a4050 con 0x55fde5b98060 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff1a598b640 1 --2- 172.21.15.45:0/972633191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5ba1750 0x55fde5babcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff1a518a640 1 --2- 172.21.15.45:0/972633191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5b98060 0x55fde5bab270 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0cab90 tx=0x7ff19c09a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=323fab8c9b3132f4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff188ff9640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff188ff9640 1 -- 172.21.15.45:0/972633191 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0a2520 con 0x55fde5b98060 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/972633191 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff19800ed10 con 0x55fde5b98060 2024-01-31T05:34:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff188ff9640 1 -- 172.21.15.45:0/972633191 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff19c063da0 con 0x55fde5b98060 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff188ff9640 1 -- 172.21.15.45:0/972633191 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff19c003760 con 0x55fde5b98060 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff1aa1b1740 10 client.25960 fetch_fsmap learned FSMap version 30 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.541+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/972633191 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fde5afe6a0 con 0x55fde5b98060 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.542+0000 7ff188ff9640 1 -- 172.21.15.45:0/972633191 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19800ed10 con 0x55fde5b98060 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.542+0000 7ff188ff9640 1 -- 172.21.15.45:0/972633191 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff19c0a2090 con 0x55fde5b98060 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.542+0000 7ff1aa1b1740 10 client.25960 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.542+0000 7ff1aa1b1740 10 client.25960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.542+0000 7ff1aa1b1740 20 client.25960 populate_metadata read hostname 'smithi045' 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.542+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/972633191 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fde5c0d820 0x55fde5c0fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.542+0000 7ff1aa1b1740 4 client.25960 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.542+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/972633191 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fde57ff600 con 0x55fde5c0d820 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.542+0000 7ff1a4989640 1 --2- 172.21.15.45:0/972633191 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fde5c0d820 0x55fde5c0fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.542+0000 7ff188ff9640 10 client.25960 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.542+0000 7ff1a4989640 1 --2- 172.21.15.45:0/972633191 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fde5c0d820 0x55fde5c0fc40 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d6ce8313c4edfd5 server_cookie=a013b5bfdbb6393e in_seq=0 out_seq=0 2024-01-31T05:34:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff188ff9640 1 -- 172.21.15.45:0/972633191 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fde57ff600 con 0x55fde5c0d820 2024-01-31T05:34:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff188ff9640 10 client.25960 handle_command_reply: tid=0 2024-01-31T05:34:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff1aa1b1740 1 client.25960 shutdown 2024-01-31T05:34:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/972633191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5b98060 msgr2=0x55fde5bab270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/972633191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5b98060 0x55fde5bab270 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0cab90 tx=0x7ff19c09a080 comp rx=0 tx=0).stop 2024-01-31T05:34:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/972633191 shutdown_connections 2024-01-31T05:34:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/972633191 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55fde5c0d820 0x55fde5c0fc40 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/972633191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fde5ba1750 0x55fde5babcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/972633191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fde5b98980 0x55fde5bab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff1aa1b1740 1 --2- 172.21.15.45:0/972633191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fde5b98060 0x55fde5bab270 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff188ff9640 0 client.25960 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:34:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/972633191 >> 172.21.15.45:0/972633191 conn(0x55fde5b92980 msgr2=0x55fde5b9d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/972633191 shutdown_connections 2024-01-31T05:34:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.555+0000 7ff1aa1b1740 1 -- 172.21.15.45:0/972633191 wait complete. 2024-01-31T05:34:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1aa1b1740 20 client.25960 trim_cache size 0 max 16384 2024-01-31T05:34:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:34:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:34:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:34:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:34:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1a73ee640 1 -- 172.21.15.45:0/3467864508 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff18c0a14f0 msgr2=0x7ff18c0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1a73ee640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff18c0a14f0 0x7ff18c0a39b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff19800fc90 tx=0x7ff198018000 comp rx=0 tx=0).stop 2024-01-31T05:34:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1a73ee640 1 -- 172.21.15.45:0/3467864508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a00ff460 msgr2=0x7ff1a01971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:34:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1a73ee640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a00ff460 0x7ff1a01971a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00ea60 tx=0x7ff19c00ef20 comp rx=0 tx=0).stop 2024-01-31T05:34:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1a73ee640 1 -- 172.21.15.45:0/3467864508 shutdown_connections 2024-01-31T05:34:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1a73ee640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff18c0a14f0 0x7ff18c0a39b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1a73ee640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a019ba60 0x7ff1a019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1a73ee640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1a0106070 0x7ff1a01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1a73ee640 1 --2- 172.21.15.45:0/3467864508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a00ff460 0x7ff1a01971a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:34:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1a73ee640 1 -- 172.21.15.45:0/3467864508 >> 172.21.15.45:0/3467864508 conn(0x7ff1a0076180 msgr2=0x7ff1a00fb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:34:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1a73ee640 1 -- 172.21.15.45:0/3467864508 shutdown_connections 2024-01-31T05:34:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:34:57.556+0000 7ff1a73ee640 1 -- 172.21.15.45:0/3467864508 wait complete. 2024-01-31T05:34:57.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1297704 10.41 MB/sec execute 537 sec latency 24.221 ms 2024-01-31T05:34:58.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1299912 10.42 MB/sec execute 538 sec latency 20.423 ms 2024-01-31T05:34:58.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:58 smithi156 ceph-mon[35720]: pgmap v389: 97 pgs: 97 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 252 op/s 2024-01-31T05:34:59.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:58 smithi186 ceph-mon[35659]: pgmap v389: 97 pgs: 97 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 252 op/s 2024-01-31T05:34:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:58 smithi045 ceph-mon[31585]: pgmap v389: 97 pgs: 97 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 252 op/s 2024-01-31T05:34:59.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1301217 10.41 MB/sec execute 539 sec latency 19.745 ms 2024-01-31T05:34:59.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:34:59 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:35:00.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:34:59 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:35:00.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:34:59 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:35:00.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1303360 10.41 MB/sec execute 540 sec latency 28.728 ms 2024-01-31T05:35:00.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:00 smithi156 ceph-mon[35720]: pgmap v390: 97 pgs: 97 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 162 op/s 2024-01-31T05:35:00.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:00 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:00.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:00 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:00.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:00 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:00.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:00 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:01.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:00 smithi186 ceph-mon[35659]: pgmap v390: 97 pgs: 97 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 162 op/s 2024-01-31T05:35:01.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:00 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:01.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:00 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:01.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:00 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:01.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:00 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:01.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:00 smithi045 ceph-mon[31585]: pgmap v390: 97 pgs: 97 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 162 op/s 2024-01-31T05:35:01.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:00 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:01.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:00 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:01.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:00 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:01.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:00 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:01.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1303744 10.40 MB/sec execute 541 sec latency 76.853 ms 2024-01-31T05:35:01.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:01 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:35:01.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:01 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:35:01.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:01 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:01.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:01 smithi156 ceph-mon[35720]: pgmap v391: 97 pgs: 97 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 162 op/s 2024-01-31T05:35:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:01 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:35:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:01 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:35:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:01 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:01 smithi186 ceph-mon[35659]: pgmap v391: 97 pgs: 97 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 162 op/s 2024-01-31T05:35:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:01 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:35:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:01 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:35:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:01 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:35:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:01 smithi045 ceph-mon[31585]: pgmap v391: 97 pgs: 97 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 162 op/s 2024-01-31T05:35:02.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1304817 10.39 MB/sec execute 542 sec latency 281.402 ms 2024-01-31T05:35:03.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1306986 10.39 MB/sec execute 543 sec latency 14.434 ms 2024-01-31T05:35:04.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1308317 10.38 MB/sec execute 544 sec latency 20.783 ms 2024-01-31T05:35:04.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:04 smithi156 ceph-mon[35720]: pgmap v392: 97 pgs: 97 active+clean; 575 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 264 op/s 2024-01-31T05:35:05.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:04 smithi186 ceph-mon[35659]: pgmap v392: 97 pgs: 97 active+clean; 575 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 264 op/s 2024-01-31T05:35:05.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:04 smithi045 ceph-mon[31585]: pgmap v392: 97 pgs: 97 active+clean; 575 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 264 op/s 2024-01-31T05:35:05.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1310530 10.39 MB/sec execute 545 sec latency 14.837 ms 2024-01-31T05:35:06.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1311610 10.37 MB/sec execute 546 sec latency 64.722 ms 2024-01-31T05:35:06.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:06 smithi156 ceph-mon[35720]: pgmap v393: 97 pgs: 97 active+clean; 575 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 182 op/s 2024-01-31T05:35:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:06 smithi186 ceph-mon[35659]: pgmap v393: 97 pgs: 97 active+clean; 575 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 182 op/s 2024-01-31T05:35:07.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:06 smithi045 ceph-mon[31585]: pgmap v393: 97 pgs: 97 active+clean; 575 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 182 op/s 2024-01-31T05:35:07.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1313988 10.38 MB/sec execute 547 sec latency 15.584 ms 2024-01-31T05:35:08.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1314665 10.37 MB/sec execute 548 sec latency 119.097 ms 2024-01-31T05:35:08.945 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:08 smithi045 ceph-mon[31585]: pgmap v394: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.3 MiB/s wr, 280 op/s 2024-01-31T05:35:08.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:08 smithi156 ceph-mon[35720]: pgmap v394: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.3 MiB/s wr, 280 op/s 2024-01-31T05:35:09.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:08 smithi186 ceph-mon[35659]: pgmap v394: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.3 MiB/s wr, 280 op/s 2024-01-31T05:35:09.802 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1317007 10.37 MB/sec execute 549 sec latency 15.251 ms 2024-01-31T05:35:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.658+0000 7ffb25142640 1 -- 172.21.15.45:0/3523505722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb2010d720 msgr2=0x7ffb2010db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.658+0000 7ffb25142640 1 --2- 172.21.15.45:0/3523505722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb2010d720 0x7ffb2010db00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009930 tx=0x7ffb1802be50 comp rx=0 tx=0).stop 2024-01-31T05:35:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.658+0000 7ffb25142640 1 -- 172.21.15.45:0/3523505722 shutdown_connections 2024-01-31T05:35:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.658+0000 7ffb25142640 1 --2- 172.21.15.45:0/3523505722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb201086b0 0x7ffb20114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.658+0000 7ffb25142640 1 --2- 172.21.15.45:0/3523505722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb20107cf0 0x7ffb20108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.658+0000 7ffb25142640 1 --2- 172.21.15.45:0/3523505722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb2010d720 0x7ffb2010db00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.658+0000 7ffb25142640 1 -- 172.21.15.45:0/3523505722 >> 172.21.15.45:0/3523505722 conn(0x7ffb2006c220 msgr2=0x7ffb2006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.659+0000 7ffb25142640 1 -- 172.21.15.45:0/3523505722 shutdown_connections 2024-01-31T05:35:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.659+0000 7ffb25142640 1 -- 172.21.15.45:0/3523505722 wait complete. 2024-01-31T05:35:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.660+0000 7ffb25142640 1 Processor -- start 2024-01-31T05:35:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.661+0000 7ffb25142640 1 -- start start 2024-01-31T05:35:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.661+0000 7ffb25142640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb20107cf0 0x7ffb201a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.661+0000 7ffb25142640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb201086b0 0x7ffb201a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.661+0000 7ffb25142640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2010d720 0x7ffb201aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.661+0000 7ffb25142640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb201aaee0 con 0x7ffb2010d720 2024-01-31T05:35:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.661+0000 7ffb25142640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb201ab050 con 0x7ffb20107cf0 2024-01-31T05:35:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.661+0000 7ffb25142640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb201ab210 con 0x7ffb201086b0 2024-01-31T05:35:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.662+0000 7ffb1f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2010d720 0x7ffb201aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.662+0000 7ffb1f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2010d720 0x7ffb201aa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32782/0 (socket says 172.21.15.45:32782) 2024-01-31T05:35:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.662+0000 7ffb1f59e640 1 -- 172.21.15.45:0/559611284 learned_addr learned my addr 172.21.15.45:0/559611284 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.662+0000 7ffb1f59e640 1 -- 172.21.15.45:0/559611284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb201086b0 msgr2=0x7ffb201a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.662+0000 7ffb1f59e640 1 --2- 172.21.15.45:0/559611284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb201086b0 0x7ffb201a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.662+0000 7ffb1f59e640 1 -- 172.21.15.45:0/559611284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb20107cf0 msgr2=0x7ffb201a6020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:35:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.662+0000 7ffb1f59e640 1 --2- 172.21.15.45:0/559611284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb20107cf0 0x7ffb201a6020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.662+0000 7ffb1f59e640 1 -- 172.21.15.45:0/559611284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb18009560 con 0x7ffb2010d720 2024-01-31T05:35:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.662+0000 7ffb1f59e640 1 --2- 172.21.15.45:0/559611284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2010d720 0x7ffb201aa8e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000d750 tx=0x7ffb1000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.663+0000 7ffb07fff640 1 -- 172.21.15.45:0/559611284 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10009bd0 con 0x7ffb2010d720 2024-01-31T05:35:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.664+0000 7ffb07fff640 1 -- 172.21.15.45:0/559611284 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb1000de60 con 0x7ffb2010d720 2024-01-31T05:35:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.664+0000 7ffb07fff640 1 -- 172.21.15.45:0/559611284 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10004e00 con 0x7ffb2010d720 2024-01-31T05:35:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.664+0000 7ffb25142640 1 -- 172.21.15.45:0/559611284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb201ab4f0 con 0x7ffb2010d720 2024-01-31T05:35:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.664+0000 7ffb25142640 1 -- 172.21.15.45:0/559611284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb201b2d70 con 0x7ffb2010d720 2024-01-31T05:35:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.669+0000 7ffb07fff640 1 -- 172.21.15.45:0/559611284 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7ffb10004f60 con 0x7ffb2010d720 2024-01-31T05:35:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.671+0000 7ffb07fff640 1 --2- 172.21.15.45:0/559611284 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ffb080a1720 0x7ffb080a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.671+0000 7ffb27fe8740 1 Processor -- start 2024-01-31T05:35:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.671+0000 7ffb07fff640 1 -- 172.21.15.45:0/559611284 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8571+0+0 (secure 0 0 0) 0x7ffb100cc8f0 con 0x7ffb2010d720 2024-01-31T05:35:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.672+0000 7ffb27fe8740 1 -- start start 2024-01-31T05:35:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.672+0000 7ffb27fe8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 0x5581bf574440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.672+0000 7ffb27fe8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf57d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.672+0000 7ffb27fe8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf57fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.673+0000 7ffb27fe8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bf4e3040 con 0x5581bf57d750 2024-01-31T05:35:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.673+0000 7ffb27fe8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bf4e0730 con 0x5581bf574060 2024-01-31T05:35:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.673+0000 7ffb27fe8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bf4dc020 con 0x5581bf574980 2024-01-31T05:35:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.673+0000 7ffb1e59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf57d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.673+0000 7ffb1e59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf57d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:50856/0 (socket says 172.21.15.45:50856) 2024-01-31T05:35:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.673+0000 7ffb1e59c640 1 -- 172.21.15.45:0/3677549628 learned_addr learned my addr 172.21.15.45:0/3677549628 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.674+0000 7ffb1e59c640 1 -- 172.21.15.45:0/3677549628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 msgr2=0x5581bf574440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-31T05:35:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.674+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/3677549628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 0x5581bf574440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.674+0000 7ffb1e59c640 1 -- 172.21.15.45:0/3677549628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 msgr2=0x5581bf57fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.674+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/3677549628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf57fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.674+0000 7ffb1e59c640 1 -- 172.21.15.45:0/3677549628 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581bf4d3e40 con 0x5581bf574980 2024-01-31T05:35:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.675+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/3677549628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf57d210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400bac0 tx=0x7ffb1400bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f0bd2ca664aec4f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.675+0000 7ffb1ed9d640 1 --2- 172.21.15.45:0/559611284 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ffb080a1720 0x7ffb080a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.676+0000 7ffb1ed9d640 1 --2- 172.21.15.45:0/559611284 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ffb080a1720 0x7ffb080a3be0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffb18037040 tx=0x7ffb18000f40 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.677+0000 7ffb057fa640 1 -- 172.21.15.45:0/3677549628 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14010070 con 0x5581bf574980 2024-01-31T05:35:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.677+0000 7ffb057fa640 1 -- 172.21.15.45:0/3677549628 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb140054e0 con 0x5581bf574980 2024-01-31T05:35:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.677+0000 7ffb057fa640 1 -- 172.21.15.45:0/3677549628 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14004e80 con 0x5581bf574980 2024-01-31T05:35:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.678+0000 7ffb27fe8740 1 -- 172.21.15.45:0/3677549628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 msgr2=0x5581bf57d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.678+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/3677549628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf57d210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400bac0 tx=0x7ffb1400bf80 comp rx=0 tx=0).stop 2024-01-31T05:35:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.678+0000 7ffb27fe8740 1 -- 172.21.15.45:0/3677549628 shutdown_connections 2024-01-31T05:35:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.678+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/3677549628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf57fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.678+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/3677549628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf57d210 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.678+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/3677549628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 0x5581bf574440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.678+0000 7ffb27fe8740 1 -- 172.21.15.45:0/3677549628 >> 172.21.15.45:0/3677549628 conn(0x5581bf56e980 msgr2=0x5581bf56ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.678+0000 7ffb27fe8740 1 -- 172.21.15.45:0/3677549628 shutdown_connections 2024-01-31T05:35:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.678+0000 7ffb27fe8740 1 -- 172.21.15.45:0/3677549628 wait complete. 2024-01-31T05:35:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.679+0000 7ffb27fe8740 1 -- start start 2024-01-31T05:35:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.679+0000 7ffb27fe8740 1 Processor -- start 2024-01-31T05:35:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.679+0000 7ffb27fe8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf574060 0x5581bf57a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.680+0000 7ffb27fe8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574980 0x5581bf57ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.680+0000 7ffb27fe8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf57d750 0x5581bf57b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.680+0000 7ffb27fe8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bf4951b0 con 0x5581bf574060 2024-01-31T05:35:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.680+0000 7ffb1ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf574060 0x5581bf57a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.680+0000 7ffb1ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf574060 0x5581bf57a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32806/0 (socket says 172.21.15.45:32806) 2024-01-31T05:35:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.680+0000 7ffb1ed9d640 1 -- 172.21.15.45:0/1341407744 learned_addr learned my addr 172.21.15.45:0/1341407744 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.680+0000 7ffb1f59e640 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf57d750 0x5581bf57b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.680+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574980 0x5581bf57ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.680+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1341407744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bf494e50 con 0x5581bf574980 2024-01-31T05:35:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.680+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1341407744 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bf45a090 con 0x5581bf57d750 2024-01-31T05:35:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.680+0000 7ffb27fe8740 10 client.0 _ll_register_callbacks cb 0x5581bf47fa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:35:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.681+0000 7ffb1ed9d640 1 -- 172.21.15.45:0/1341407744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf57d750 msgr2=0x5581bf57b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.681+0000 7ffb1ed9d640 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf57d750 0x5581bf57b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.681+0000 7ffb1ed9d640 1 -- 172.21.15.45:0/1341407744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574980 msgr2=0x5581bf57ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.681+0000 7ffb1ed9d640 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574980 0x5581bf57ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.681+0000 7ffb1ed9d640 1 -- 172.21.15.45:0/1341407744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1400b7a0 con 0x5581bf574060 2024-01-31T05:35:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.681+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574980 0x5581bf57ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:35:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.681+0000 7ffb1ed9d640 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf574060 0x5581bf57a700 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ffb18004320 tx=0x7ffb1802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01a58bd630a67b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.681+0000 7ffb02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:35:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.682+0000 7ffb02ff5640 1 -- 172.21.15.45:0/1341407744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180047d0 con 0x5581bf574060 2024-01-31T05:35:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.682+0000 7ffb02ff5640 1 -- 172.21.15.45:0/1341407744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8571+0+0 (secure 0 0 0) 0x7ffb1803f440 con 0x5581bf574060 2024-01-31T05:35:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.682+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1341407744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581bf4d5010 con 0x5581bf574060 2024-01-31T05:35:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.682+0000 7ffb02ff5640 1 -- 172.21.15.45:0/1341407744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb1803f600 con 0x5581bf574060 2024-01-31T05:35:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.683+0000 7ffb27fe8740 10 client.16407 fetch_fsmap learned FSMap version 30 2024-01-31T05:35:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.683+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1341407744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5581bf4da6a0 con 0x5581bf574060 2024-01-31T05:35:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.683+0000 7ffb02ff5640 1 -- 172.21.15.45:0/1341407744 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb18043070 con 0x5581bf574060 2024-01-31T05:35:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.683+0000 7ffb02ff5640 1 -- 172.21.15.45:0/1341407744 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffb18034730 con 0x5581bf574060 2024-01-31T05:35:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.683+0000 7ffb27fe8740 10 client.16407 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:35:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.683+0000 7ffb27fe8740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:35:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.683+0000 7ffb27fe8740 20 client.16407 populate_metadata read hostname 'smithi045' 2024-01-31T05:35:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.683+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5581bf58d540 0x5581bf58f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.684+0000 7ffb27fe8740 4 client.16407 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:35:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.684+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1341407744 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581bf4d5010 con 0x5581bf58d540 2024-01-31T05:35:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.684+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5581bf58d540 0x5581bf58f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.684+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5581bf58d540 0x5581bf58f9e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10003acabe26f019 server_cookie=e689695b74a6058f in_seq=0 out_seq=0 2024-01-31T05:35:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.684+0000 7ffb02ff5640 10 client.16407 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.685+0000 7ffb02ff5640 1 -- 172.21.15.45:0/1341407744 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5581bf4d5010 con 0x5581bf58d540 2024-01-31T05:35:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.685+0000 7ffb02ff5640 10 client.16407 handle_command_reply: tid=0 2024-01-31T05:35:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.685+0000 7ffb27fe8740 1 client.16407 shutdown 2024-01-31T05:35:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.685+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1341407744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf574060 msgr2=0x5581bf57a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.685+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf574060 0x5581bf57a700 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ffb18004320 tx=0x7ffb1802be50 comp rx=0 tx=0).stop 2024-01-31T05:35:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.686+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1341407744 shutdown_connections 2024-01-31T05:35:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.686+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5581bf58d540 0x5581bf58f9e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.686+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf57d750 0x5581bf57b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.686+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574980 0x5581bf57ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.686+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/1341407744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf574060 0x5581bf57a700 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.686+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1341407744 >> 172.21.15.45:0/1341407744 conn(0x5581bf56e980 msgr2=0x5581bf5706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.686+0000 7ffb02ff5640 0 client.16407 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.686+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1341407744 shutdown_connections 2024-01-31T05:35:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.686+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1341407744 wait complete. 2024-01-31T05:35:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.686+0000 7ffb27fe8740 20 client.16407 trim_cache size 0 max 16384 2024-01-31T05:35:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.702+0000 7ffb27fe8740 1 Processor -- start 2024-01-31T05:35:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.702+0000 7ffb27fe8740 1 -- start start 2024-01-31T05:35:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.702+0000 7ffb27fe8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 0x5581bf57afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.702+0000 7ffb27fe8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf570470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.702+0000 7ffb27fe8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf594090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.702+0000 7ffb27fe8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bf589a00 con 0x5581bf57d750 2024-01-31T05:35:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.702+0000 7ffb27fe8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bf57b870 con 0x5581bf574060 2024-01-31T05:35:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.702+0000 7ffb27fe8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bf589350 con 0x5581bf574980 2024-01-31T05:35:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb1f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf594090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb1e59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf570470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb1f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf594090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32812/0 (socket says 172.21.15.45:32812) 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb1ed9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 0x5581bf57afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb1f59e640 1 -- 172.21.15.45:0/2718612978 learned_addr learned my addr 172.21.15.45:0/2718612978 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb1f59e640 1 -- 172.21.15.45:0/2718612978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 msgr2=0x5581bf570470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb1f59e640 1 --2- 172.21.15.45:0/2718612978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf570470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb1f59e640 1 -- 172.21.15.45:0/2718612978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 msgr2=0x5581bf57afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb1f59e640 1 --2- 172.21.15.45:0/2718612978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 0x5581bf57afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb1f59e640 1 -- 172.21.15.45:0/2718612978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581bf477030 con 0x5581bf57d750 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb1f59e640 1 --2- 172.21.15.45:0/2718612978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf594090 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009610 tx=0x7ffb100991a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a60341dba7c8282 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb04ff9640 1 -- 172.21.15.45:0/2718612978 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10099660 con 0x5581bf57d750 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb04ff9640 1 -- 172.21.15.45:0/2718612978 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb1009d7b0 con 0x5581bf57d750 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.703+0000 7ffb04ff9640 1 -- 172.21.15.45:0/2718612978 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100a1560 con 0x5581bf57d750 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.704+0000 7ffb27fe8740 1 -- 172.21.15.45:0/2718612978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 msgr2=0x5581bf594090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.704+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/2718612978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf594090 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009610 tx=0x7ffb100991a0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.704+0000 7ffb27fe8740 1 -- 172.21.15.45:0/2718612978 shutdown_connections 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.704+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/2718612978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf594090 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.704+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/2718612978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf570470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.704+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/2718612978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 0x5581bf57afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.704+0000 7ffb27fe8740 1 -- 172.21.15.45:0/2718612978 >> 172.21.15.45:0/2718612978 conn(0x5581bf56e980 msgr2=0x5581bf56fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.704+0000 7ffb27fe8740 1 -- 172.21.15.45:0/2718612978 shutdown_connections 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.704+0000 7ffb27fe8740 1 -- 172.21.15.45:0/2718612978 wait complete. 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.704+0000 7ffb27fe8740 1 -- start start 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.704+0000 7ffb27fe8740 1 Processor -- start 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb27fe8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 0x5581bf593790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb27fe8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf593cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb27fe8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf5dc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb27fe8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bf5dceb0 con 0x5581bf57d750 2024-01-31T05:35:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb27fe8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bf5dd020 con 0x5581bf574060 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb27fe8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bf5dd1e0 con 0x5581bf574980 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb27fe8740 10 client.0 _ll_register_callbacks cb 0x7ffb1400b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf5dc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf5dc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32814/0 (socket says 172.21.15.45:32814) 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1e59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf593cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1f59e640 1 -- 172.21.15.45:0/1881306322 learned_addr learned my addr 172.21.15.45:0/1881306322 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1ed9d640 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 0x5581bf593790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1f59e640 1 -- 172.21.15.45:0/1881306322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 msgr2=0x5581bf593cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1f59e640 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf593cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1f59e640 1 -- 172.21.15.45:0/1881306322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 msgr2=0x5581bf593790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1f59e640 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 0x5581bf593790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1f59e640 1 -- 172.21.15.45:0/1881306322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb201b2d70 con 0x5581bf57d750 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1ed9d640 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 0x5581bf593790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:35:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf593cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb1f59e640 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf5dc970 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ffb100046e0 tx=0x7ffb100c9f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9f483273fe7c1f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb02ff5640 1 -- 172.21.15.45:0/1881306322 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10099660 con 0x5581bf57d750 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb02ff5640 1 -- 172.21.15.45:0/1881306322 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8571+0+0 (secure 0 0 0) 0x7ffb100a19f0 con 0x5581bf57d750 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.705+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1881306322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb140054e0 con 0x5581bf57d750 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.706+0000 7ffb02ff5640 1 -- 172.21.15.45:0/1881306322 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb100ca230 con 0x5581bf57d750 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.706+0000 7ffb27fe8740 10 client.16413 fetch_fsmap learned FSMap version 30 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.706+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1881306322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5581bf49ca50 con 0x5581bf57d750 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.706+0000 7ffb02ff5640 1 -- 172.21.15.45:0/1881306322 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb140054e0 con 0x5581bf57d750 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.706+0000 7ffb02ff5640 1 -- 172.21.15.45:0/1881306322 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffb100986c0 con 0x5581bf57d750 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.706+0000 7ffb27fe8740 10 client.16413 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.706+0000 7ffb27fe8740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:35:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.706+0000 7ffb27fe8740 20 client.16413 populate_metadata read hostname 'smithi045' 2024-01-31T05:35:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.706+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5581bf5e58f0 0x5581bf5e7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.706+0000 7ffb27fe8740 4 client.16413 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:35:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.706+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1881306322 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5581bf4d4d30 con 0x5581bf5e58f0 2024-01-31T05:35:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.706+0000 7ffb1ed9d640 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5581bf5e58f0 0x5581bf5e7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.707+0000 7ffb02ff5640 10 client.16413 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.707+0000 7ffb1ed9d640 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5581bf5e58f0 0x5581bf5e7ce0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2150056213e359cc server_cookie=dfd05e005258e3b6 in_seq=0 out_seq=0 2024-01-31T05:35:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.721+0000 7ffb02ff5640 1 -- 172.21.15.45:0/1881306322 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5581bf4d4d30 con 0x5581bf5e58f0 2024-01-31T05:35:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.721+0000 7ffb02ff5640 10 client.16413 handle_command_reply: tid=0 2024-01-31T05:35:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.721+0000 7ffb27fe8740 1 client.16413 shutdown 2024-01-31T05:35:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.722+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1881306322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 msgr2=0x5581bf5dc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.722+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf5dc970 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ffb100046e0 tx=0x7ffb100c9f50 comp rx=0 tx=0).stop 2024-01-31T05:35:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.722+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1881306322 shutdown_connections 2024-01-31T05:35:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.722+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5581bf5e58f0 0x5581bf5e7ce0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.722+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5581bf57d750 0x5581bf5dc970 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.722+0000 7ffb02ff5640 0 client.16413 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.722+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581bf574980 0x5581bf593cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.722+0000 7ffb27fe8740 1 --2- 172.21.15.45:0/1881306322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581bf574060 0x5581bf593790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.722+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1881306322 >> 172.21.15.45:0/1881306322 conn(0x5581bf56e980 msgr2=0x5581bf56da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.722+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1881306322 shutdown_connections 2024-01-31T05:35:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.722+0000 7ffb27fe8740 1 -- 172.21.15.45:0/1881306322 wait complete. 2024-01-31T05:35:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.722+0000 7ffb27fe8740 20 client.16413 trim_cache size 0 max 16384 2024-01-31T05:35:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:35:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:35:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:35:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:35:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.723+0000 7ffb25142640 1 -- 172.21.15.45:0/559611284 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ffb080a1720 msgr2=0x7ffb080a3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.723+0000 7ffb25142640 1 --2- 172.21.15.45:0/559611284 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ffb080a1720 0x7ffb080a3be0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffb18037040 tx=0x7ffb18000f40 comp rx=0 tx=0).stop 2024-01-31T05:35:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.723+0000 7ffb25142640 1 -- 172.21.15.45:0/559611284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2010d720 msgr2=0x7ffb201aa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.723+0000 7ffb25142640 1 --2- 172.21.15.45:0/559611284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2010d720 0x7ffb201aa8e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000d750 tx=0x7ffb1000dc10 comp rx=0 tx=0).stop 2024-01-31T05:35:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.723+0000 7ffb25142640 1 -- 172.21.15.45:0/559611284 shutdown_connections 2024-01-31T05:35:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.723+0000 7ffb25142640 1 --2- 172.21.15.45:0/559611284 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ffb080a1720 0x7ffb080a3be0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.723+0000 7ffb25142640 1 --2- 172.21.15.45:0/559611284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2010d720 0x7ffb201aa8e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.723+0000 7ffb25142640 1 --2- 172.21.15.45:0/559611284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb201086b0 0x7ffb201a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.723+0000 7ffb25142640 1 --2- 172.21.15.45:0/559611284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb20107cf0 0x7ffb201a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.723+0000 7ffb25142640 1 -- 172.21.15.45:0/559611284 >> 172.21.15.45:0/559611284 conn(0x7ffb2006c220 msgr2=0x7ffb20071610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.723+0000 7ffb25142640 1 -- 172.21.15.45:0/559611284 shutdown_connections 2024-01-31T05:35:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:10.723+0000 7ffb25142640 1 -- 172.21.15.45:0/559611284 wait complete. 2024-01-31T05:35:10.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1318140 10.36 MB/sec execute 550 sec latency 14.248 ms 2024-01-31T05:35:10.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:10 smithi156 ceph-mon[35720]: pgmap v395: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 199 op/s 2024-01-31T05:35:11.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:10 smithi186 ceph-mon[35659]: pgmap v395: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 199 op/s 2024-01-31T05:35:11.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:10 smithi045 ceph-mon[31585]: pgmap v395: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 199 op/s 2024-01-31T05:35:11.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1320014 10.36 MB/sec execute 551 sec latency 71.670 ms 2024-01-31T05:35:11.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:11 smithi156 ceph-mon[35720]: osdmap e108: 12 total, 12 up, 12 in 2024-01-31T05:35:11.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:11 smithi156 ceph-mon[35720]: pgmap v397: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 239 op/s 2024-01-31T05:35:12.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:11 smithi186 ceph-mon[35659]: osdmap e108: 12 total, 12 up, 12 in 2024-01-31T05:35:12.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:11 smithi186 ceph-mon[35659]: pgmap v397: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 239 op/s 2024-01-31T05:35:12.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:11 smithi045 ceph-mon[31585]: osdmap e108: 12 total, 12 up, 12 in 2024-01-31T05:35:12.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:11 smithi045 ceph-mon[31585]: pgmap v397: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 239 op/s 2024-01-31T05:35:12.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1321416 10.36 MB/sec execute 552 sec latency 16.472 ms 2024-01-31T05:35:13.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1322980 10.35 MB/sec execute 553 sec latency 88.677 ms 2024-01-31T05:35:14.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1324885 10.36 MB/sec execute 554 sec latency 20.678 ms 2024-01-31T05:35:14.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:14 smithi156 ceph-mon[35720]: pgmap v398: 97 pgs: 97 active+clean; 591 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 201 op/s 2024-01-31T05:35:15.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:14 smithi186 ceph-mon[35659]: pgmap v398: 97 pgs: 97 active+clean; 591 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 201 op/s 2024-01-31T05:35:15.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:14 smithi045 ceph-mon[31585]: pgmap v398: 97 pgs: 97 active+clean; 591 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 201 op/s 2024-01-31T05:35:15.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1326841 10.35 MB/sec execute 555 sec latency 20.596 ms 2024-01-31T05:35:16.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1328343 10.35 MB/sec execute 556 sec latency 101.820 ms 2024-01-31T05:35:16.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:16 smithi156 ceph-mon[35720]: pgmap v399: 97 pgs: 97 active+clean; 591 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 201 op/s 2024-01-31T05:35:17.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:16 smithi186 ceph-mon[35659]: pgmap v399: 97 pgs: 97 active+clean; 591 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 201 op/s 2024-01-31T05:35:17.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:16 smithi045 ceph-mon[31585]: pgmap v399: 97 pgs: 97 active+clean; 591 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 201 op/s 2024-01-31T05:35:17.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1329955 10.35 MB/sec execute 557 sec latency 25.264 ms 2024-01-31T05:35:18.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1331788 10.35 MB/sec execute 558 sec latency 223.931 ms 2024-01-31T05:35:18.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:18 smithi156 ceph-mon[35720]: pgmap v400: 97 pgs: 97 active+clean; 609 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.5 MiB/s wr, 192 op/s 2024-01-31T05:35:18.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:18 smithi156 ceph-mon[35720]: osdmap e109: 12 total, 12 up, 12 in 2024-01-31T05:35:19.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:18 smithi186 ceph-mon[35659]: pgmap v400: 97 pgs: 97 active+clean; 609 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.5 MiB/s wr, 192 op/s 2024-01-31T05:35:19.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:18 smithi186 ceph-mon[35659]: osdmap e109: 12 total, 12 up, 12 in 2024-01-31T05:35:19.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:18 smithi045 ceph-mon[31585]: pgmap v400: 97 pgs: 97 active+clean; 609 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.5 MiB/s wr, 192 op/s 2024-01-31T05:35:19.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:18 smithi045 ceph-mon[31585]: osdmap e109: 12 total, 12 up, 12 in 2024-01-31T05:35:19.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1333153 10.34 MB/sec execute 559 sec latency 53.099 ms 2024-01-31T05:35:20.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1335362 10.35 MB/sec execute 560 sec latency 21.731 ms 2024-01-31T05:35:20.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:20 smithi156 ceph-mon[35720]: pgmap v402: 97 pgs: 97 active+clean; 609 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 213 op/s 2024-01-31T05:35:21.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:20 smithi186 ceph-mon[35659]: pgmap v402: 97 pgs: 97 active+clean; 609 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 213 op/s 2024-01-31T05:35:21.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:20 smithi045 ceph-mon[31585]: pgmap v402: 97 pgs: 97 active+clean; 609 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 213 op/s 2024-01-31T05:35:21.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1336656 10.34 MB/sec execute 561 sec latency 24.364 ms 2024-01-31T05:35:21.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:21 smithi156 ceph-mon[35720]: pgmap v403: 97 pgs: 97 active+clean; 609 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.5 MiB/s wr, 192 op/s 2024-01-31T05:35:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:21 smithi186 ceph-mon[35659]: pgmap v403: 97 pgs: 97 active+clean; 609 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.5 MiB/s wr, 192 op/s 2024-01-31T05:35:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:21 smithi045 ceph-mon[31585]: pgmap v403: 97 pgs: 97 active+clean; 609 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.5 MiB/s wr, 192 op/s 2024-01-31T05:35:22.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1338620 10.34 MB/sec execute 562 sec latency 123.346 ms 2024-01-31T05:35:23.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1339610 10.33 MB/sec execute 563 sec latency 20.964 ms 2024-01-31T05:35:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.816+0000 7fd399ab8640 1 -- 172.21.15.45:0/1613871075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd394104b90 msgr2=0x7fd394104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.816+0000 7fd399ab8640 1 --2- 172.21.15.45:0/1613871075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd394104b90 0x7fd394104f70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd384009900 tx=0x7fd38402bc30 comp rx=0 tx=0).stop 2024-01-31T05:35:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.816+0000 7fd399ab8640 1 -- 172.21.15.45:0/1613871075 shutdown_connections 2024-01-31T05:35:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.816+0000 7fd399ab8640 1 --2- 172.21.15.45:0/1613871075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd394100800 0x7fd39410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.816+0000 7fd399ab8640 1 --2- 172.21.15.45:0/1613871075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3940ffe40 0x7fd3941002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.816+0000 7fd399ab8640 1 --2- 172.21.15.45:0/1613871075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd394104b90 0x7fd394104f70 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.816+0000 7fd399ab8640 1 -- 172.21.15.45:0/1613871075 >> 172.21.15.45:0/1613871075 conn(0x7fd394076a40 msgr2=0x7fd394076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.816+0000 7fd399ab8640 1 -- 172.21.15.45:0/1613871075 shutdown_connections 2024-01-31T05:35:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.816+0000 7fd399ab8640 1 -- 172.21.15.45:0/1613871075 wait complete. 2024-01-31T05:35:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.817+0000 7fd399ab8640 1 Processor -- start 2024-01-31T05:35:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.817+0000 7fd399ab8640 1 -- start start 2024-01-31T05:35:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.817+0000 7fd399ab8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3940ffe40 0x7fd3941972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.817+0000 7fd399ab8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd394100800 0x7fd394197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.817+0000 7fd399ab8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd394104b90 0x7fd39419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.817+0000 7fd399ab8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39419c140 con 0x7fd3940ffe40 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.817+0000 7fd399ab8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39419c2b0 con 0x7fd394100800 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.817+0000 7fd399ab8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39419c470 con 0x7fd394104b90 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.817+0000 7fd3937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3940ffe40 0x7fd3941972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd393fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd394104b90 0x7fd39419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd3937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3940ffe40 0x7fd3941972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53966/0 (socket says 172.21.15.45:53966) 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd392ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd394100800 0x7fd394197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd3937fe640 1 -- 172.21.15.45:0/2084249571 learned_addr learned my addr 172.21.15.45:0/2084249571 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd3937fe640 1 -- 172.21.15.45:0/2084249571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd394104b90 msgr2=0x7fd39419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd3937fe640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd394104b90 0x7fd39419bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd3937fe640 1 -- 172.21.15.45:0/2084249571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd394100800 msgr2=0x7fd394197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd3937fe640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd394100800 0x7fd394197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd3937fe640 1 -- 172.21.15.45:0/2084249571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd384009560 con 0x7fd3940ffe40 2024-01-31T05:35:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd392ffd640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd394100800 0x7fd394197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:35:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd3937fe640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3940ffe40 0x7fd3941972d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd384005940 tx=0x7fd38402c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd390ff9640 1 -- 172.21.15.45:0/2084249571 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384004250 con 0x7fd3940ffe40 2024-01-31T05:35:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd390ff9640 1 -- 172.21.15.45:0/2084249571 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3840043b0 con 0x7fd3940ffe40 2024-01-31T05:35:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd390ff9640 1 -- 172.21.15.45:0/2084249571 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384003b90 con 0x7fd3940ffe40 2024-01-31T05:35:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd399ab8640 1 -- 172.21.15.45:0/2084249571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39419c6f0 con 0x7fd3940ffe40 2024-01-31T05:35:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.818+0000 7fd399ab8640 1 -- 172.21.15.45:0/2084249571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3941a3fd0 con 0x7fd3940ffe40 2024-01-31T05:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.820+0000 7fd390ff9640 1 -- 172.21.15.45:0/2084249571 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fd384003cf0 con 0x7fd3940ffe40 2024-01-31T05:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd39c876740 1 Processor -- start 2024-01-31T05:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd39c876740 1 -- start start 2024-01-31T05:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd390ff9640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd37c0a14f0 0x7fd37c0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd39c876740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 0x55db81cc7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd39c876740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd393fff640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd37c0a14f0 0x7fd37c0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd390ff9640 1 -- 172.21.15.45:0/2084249571 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd38402de40 con 0x7fd3940ffe40 2024-01-31T05:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd39c876740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cd5b60 0x55db81cd7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd39c876740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db81c37040 con 0x55db81cc8420 2024-01-31T05:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd39c876740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db81c34730 con 0x55db81cc7b00 2024-01-31T05:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd39c876740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db81c30020 con 0x55db81cd5b60 2024-01-31T05:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd3937fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 0x55db81cc7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd392ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd3937fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 0x55db81cc7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:49726/0 (socket says 172.21.15.45:49726) 2024-01-31T05:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd393fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cd5b60 0x55db81cd7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.821+0000 7fd3937fe640 1 -- 172.21.15.45:0/3293338862 learned_addr learned my addr 172.21.15.45:0/3293338862 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.822+0000 7fd3937fe640 1 -- 172.21.15.45:0/3293338862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cd5b60 msgr2=0x55db81cd7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.822+0000 7fd3937fe640 1 --2- 172.21.15.45:0/3293338862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cd5b60 0x55db81cd7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.822+0000 7fd3937fe640 1 -- 172.21.15.45:0/3293338862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 msgr2=0x55db81cd5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.822+0000 7fd3937fe640 1 --2- 172.21.15.45:0/3293338862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.822+0000 7fd3937fe640 1 -- 172.21.15.45:0/3293338862 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db81c27e40 con 0x55db81cc7b00 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.822+0000 7fd393fff640 1 --2- 172.21.15.45:0/3293338862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cd5b60 0x55db81cd7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.822+0000 7fd393fff640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd37c0a14f0 0x7fd37c0a39b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd38c005fd0 tx=0x7fd38c013b60 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.822+0000 7fd3937fe640 1 --2- 172.21.15.45:0/3293338862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 0x55db81cc7ee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd3840bc2f0 tx=0x7fd3840bc1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468cadd7e1a0d5c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.822+0000 7fd379ffb640 1 -- 172.21.15.45:0/3293338862 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840c0740 con 0x55db81cc7b00 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.822+0000 7fd379ffb640 1 -- 172.21.15.45:0/3293338862 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3840c4980 con 0x55db81cc7b00 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.822+0000 7fd379ffb640 1 -- 172.21.15.45:0/3293338862 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840bbce0 con 0x55db81cc7b00 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.823+0000 7fd39c876740 1 -- 172.21.15.45:0/3293338862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 msgr2=0x55db81cc7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.823+0000 7fd39c876740 1 --2- 172.21.15.45:0/3293338862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 0x55db81cc7ee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd3840bc2f0 tx=0x7fd3840bc1f0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.823+0000 7fd39c876740 1 -- 172.21.15.45:0/3293338862 shutdown_connections 2024-01-31T05:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.823+0000 7fd39c876740 1 --2- 172.21.15.45:0/3293338862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.823+0000 7fd39c876740 1 --2- 172.21.15.45:0/3293338862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cd5b60 0x55db81cd7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.823+0000 7fd39c876740 1 --2- 172.21.15.45:0/3293338862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 0x55db81cc7ee0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.823+0000 7fd39c876740 1 -- 172.21.15.45:0/3293338862 >> 172.21.15.45:0/3293338862 conn(0x55db81cc2980 msgr2=0x55db81cc2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.823+0000 7fd39c876740 1 -- 172.21.15.45:0/3293338862 shutdown_connections 2024-01-31T05:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.823+0000 7fd39c876740 1 -- 172.21.15.45:0/3293338862 wait complete. 2024-01-31T05:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.823+0000 7fd39c876740 1 -- start start 2024-01-31T05:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd39c876740 1 Processor -- start 2024-01-31T05:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd39c876740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 0x55db81cd2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd39c876740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd39c876740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cd5b60 0x55db81cd3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd39c876740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db81be91b0 con 0x55db81cc8420 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd39c876740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db81be8e50 con 0x55db81cc7b00 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd39c876740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db81bae090 con 0x55db81cd5b60 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd39c876740 10 client.0 _ll_register_callbacks cb 0x55db81bd3a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd392ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd392ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53976/0 (socket says 172.21.15.45:53976) 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd3937fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 0x55db81cd2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd392ffd640 1 -- 172.21.15.45:0/3451597912 learned_addr learned my addr 172.21.15.45:0/3451597912 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd393fff640 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cd5b60 0x55db81cd3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd3937fe640 1 -- 172.21.15.45:0/3451597912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cd5b60 msgr2=0x55db81cd3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd3937fe640 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cd5b60 0x55db81cd3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd3937fe640 1 -- 172.21.15.45:0/3451597912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 msgr2=0x55db81cd2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd3937fe640 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd3937fe640 1 -- 172.21.15.45:0/3451597912 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3941a3fd0 con 0x55db81cc7b00 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.824+0000 7fd392ffd640 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd3937fe640 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 0x55db81cd2a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd3840b82e0 tx=0x7fd3840bc260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66cee0a694c3d554 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd372ffd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd372ffd640 1 -- 172.21.15.45:0/3451597912 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840b8ec0 con 0x55db81cc7b00 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd372ffd640 1 -- 172.21.15.45:0/3451597912 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd3840c0740 con 0x55db81cc7b00 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd39c876740 1 -- 172.21.15.45:0/3451597912 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db81c29010 con 0x55db81cc7b00 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd372ffd640 1 -- 172.21.15.45:0/3451597912 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3840bbd90 con 0x55db81cc7b00 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd39c876740 10 client.25922 fetch_fsmap learned FSMap version 30 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd39c876740 1 -- 172.21.15.45:0/3451597912 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55db81c2e6a0 con 0x55db81cc7b00 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd372ffd640 1 -- 172.21.15.45:0/3451597912 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db81c29010 con 0x55db81cc7b00 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd372ffd640 1 -- 172.21.15.45:0/3451597912 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd38403c070 con 0x55db81cc7b00 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd39c876740 10 client.25922 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd39c876740 10 client.25922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:35:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd39c876740 20 client.25922 populate_metadata read hostname 'smithi045' 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd39c876740 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55db81ce52a0 0x55db81ce76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd39c876740 4 client.25922 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.825+0000 7fd39c876740 1 -- 172.21.15.45:0/3451597912 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db81c28d30 con 0x55db81ce52a0 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.826+0000 7fd392ffd640 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55db81ce52a0 0x55db81ce76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.826+0000 7fd372ffd640 10 client.25922 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.826+0000 7fd392ffd640 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55db81ce52a0 0x55db81ce76c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=979882515cc7c71a server_cookie=8750cddd76b630df in_seq=0 out_seq=0 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd372ffd640 1 -- 172.21.15.45:0/3451597912 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db81c28d30 con 0x55db81ce52a0 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd372ffd640 10 client.25922 handle_command_reply: tid=0 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd39c876740 1 client.25922 shutdown 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd39c876740 1 -- 172.21.15.45:0/3451597912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 msgr2=0x55db81cd2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd39c876740 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 0x55db81cd2a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd3840b82e0 tx=0x7fd3840bc260 comp rx=0 tx=0).stop 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd39c876740 1 -- 172.21.15.45:0/3451597912 shutdown_connections 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd39c876740 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55db81ce52a0 0x55db81ce76c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd39c876740 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd39c876740 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cd5b60 0x55db81cd3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd372ffd640 0 client.25922 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd39c876740 1 --2- 172.21.15.45:0/3451597912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cc7b00 0x55db81cd2a00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd39c876740 1 -- 172.21.15.45:0/3451597912 >> 172.21.15.45:0/3451597912 conn(0x55db81cc2980 msgr2=0x55db81cc46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd39c876740 1 -- 172.21.15.45:0/3451597912 shutdown_connections 2024-01-31T05:35:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd39c876740 1 -- 172.21.15.45:0/3451597912 wait complete. 2024-01-31T05:35:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.827+0000 7fd39c876740 20 client.25922 trim_cache size 0 max 16384 2024-01-31T05:35:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.840+0000 7fd39c876740 1 Processor -- start 2024-01-31T05:35:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.840+0000 7fd39c876740 1 -- start start 2024-01-31T05:35:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.840+0000 7fd39c876740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cc7b00 0x55db81cd5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.840+0000 7fd39c876740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.840+0000 7fd39c876740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cd5b60 0x55db81ccfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.840+0000 7fd39c876740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db81cec8f0 con 0x55db81cc8420 2024-01-31T05:35:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.840+0000 7fd39c876740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db81cd82d0 con 0x55db81cd5b60 2024-01-31T05:35:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.840+0000 7fd39c876740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db81ce1a50 con 0x55db81cc7b00 2024-01-31T05:35:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.840+0000 7fd392ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.840+0000 7fd392ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53980/0 (socket says 172.21.15.45:53980) 2024-01-31T05:35:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.840+0000 7fd3937fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cc7b00 0x55db81cd5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.840+0000 7fd392ffd640 1 -- 172.21.15.45:0/2052195092 learned_addr learned my addr 172.21.15.45:0/2052195092 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.841+0000 7fd393fff640 1 --2- 172.21.15.45:0/2052195092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cd5b60 0x55db81ccfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.841+0000 7fd392ffd640 1 -- 172.21.15.45:0/2052195092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cc7b00 msgr2=0x55db81cd5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.841+0000 7fd392ffd640 1 --2- 172.21.15.45:0/2052195092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cc7b00 0x55db81cd5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.841+0000 7fd392ffd640 1 -- 172.21.15.45:0/2052195092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cd5b60 msgr2=0x55db81ccfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.841+0000 7fd392ffd640 1 --2- 172.21.15.45:0/2052195092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cd5b60 0x55db81ccfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.841+0000 7fd392ffd640 1 -- 172.21.15.45:0/2052195092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db81bcb030 con 0x55db81cc8420 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.841+0000 7fd393fff640 1 --2- 172.21.15.45:0/2052195092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cd5b60 0x55db81ccfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.841+0000 7fd3937fe640 1 --2- 172.21.15.45:0/2052195092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cc7b00 0x55db81cd5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.841+0000 7fd392ffd640 1 --2- 172.21.15.45:0/2052195092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd32a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x55db81ce7ff0 tx=0x7fd388005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4951af2afe40f53c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.841+0000 7fd3797fa640 1 -- 172.21.15.45:0/2052195092 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388017070 con 0x55db81cc8420 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.841+0000 7fd3797fa640 1 -- 172.21.15.45:0/2052195092 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd388018040 con 0x55db81cc8420 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.841+0000 7fd3797fa640 1 -- 172.21.15.45:0/2052195092 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388005670 con 0x55db81cc8420 2024-01-31T05:35:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.842+0000 7fd39c876740 1 -- 172.21.15.45:0/2052195092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 msgr2=0x55db81cd32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.842+0000 7fd39c876740 1 --2- 172.21.15.45:0/2052195092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd32a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x55db81ce7ff0 tx=0x7fd388005f90 comp rx=0 tx=0).stop 2024-01-31T05:35:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.842+0000 7fd39c876740 1 -- 172.21.15.45:0/2052195092 shutdown_connections 2024-01-31T05:35:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.842+0000 7fd39c876740 1 --2- 172.21.15.45:0/2052195092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc8420 0x55db81cd32a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.842+0000 7fd39c876740 1 --2- 172.21.15.45:0/2052195092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cd5b60 0x55db81ccfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.842+0000 7fd39c876740 1 --2- 172.21.15.45:0/2052195092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cc7b00 0x55db81cd5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.842+0000 7fd39c876740 1 -- 172.21.15.45:0/2052195092 >> 172.21.15.45:0/2052195092 conn(0x55db81cc2980 msgr2=0x55db81cc2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.842+0000 7fd39c876740 1 -- 172.21.15.45:0/2052195092 shutdown_connections 2024-01-31T05:35:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.842+0000 7fd39c876740 1 -- 172.21.15.45:0/2052195092 wait complete. 2024-01-31T05:35:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.842+0000 7fd39c876740 1 -- start start 2024-01-31T05:35:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.842+0000 7fd39c876740 1 Processor -- start 2024-01-31T05:35:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd39c876740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc7b00 0x55db81cdc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd39c876740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cc8420 0x55db81cdaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd39c876740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cd5b60 0x55db81cdafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd39c876740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db81cdcf50 con 0x55db81cc7b00 2024-01-31T05:35:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd39c876740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db81cdb520 con 0x55db81cd5b60 2024-01-31T05:35:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd39c876740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db81cdb6e0 con 0x55db81cc8420 2024-01-31T05:35:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd39c876740 10 client.0 _ll_register_callbacks cb 0x7fd384004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:35:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd3937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc7b00 0x55db81cdc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd392ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cc8420 0x55db81cdaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd3937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc7b00 0x55db81cdc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53994/0 (socket says 172.21.15.45:53994) 2024-01-31T05:35:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd3937fe640 1 -- 172.21.15.45:0/2548022020 learned_addr learned my addr 172.21.15.45:0/2548022020 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd393fff640 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cd5b60 0x55db81cdafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd392ffd640 1 -- 172.21.15.45:0/2548022020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cd5b60 msgr2=0x55db81cdafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd392ffd640 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cd5b60 0x55db81cdafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd392ffd640 1 -- 172.21.15.45:0/2548022020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc7b00 msgr2=0x55db81cdc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd392ffd640 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc7b00 0x55db81cdc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd392ffd640 1 -- 172.21.15.45:0/2548022020 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db81c2e6a0 con 0x55db81cc8420 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd3937fe640 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc7b00 0x55db81cdc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd392ffd640 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cc8420 0x55db81cdaa70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd388014010 tx=0x7fd388004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e18f89e9a71abe6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd372ffd640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd372ffd640 1 -- 172.21.15.45:0/2548022020 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388017040 con 0x55db81cc8420 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.843+0000 7fd372ffd640 1 -- 172.21.15.45:0/2548022020 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd388005410 con 0x55db81cc8420 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd39c876740 1 -- 172.21.15.45:0/2548022020 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3840c4980 con 0x55db81cc8420 2024-01-31T05:35:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd372ffd640 1 -- 172.21.15.45:0/2548022020 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd388005230 con 0x55db81cc8420 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd39c876740 10 client.25993 fetch_fsmap learned FSMap version 30 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd39c876740 1 -- 172.21.15.45:0/2548022020 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fd38c00fbd0 con 0x55db81cc8420 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd372ffd640 1 -- 172.21.15.45:0/2548022020 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3840c4980 con 0x55db81cc8420 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd372ffd640 1 -- 172.21.15.45:0/2548022020 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd388029070 con 0x55db81cc8420 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd39c876740 10 client.25993 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd39c876740 10 client.25993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd39c876740 20 client.25993 populate_metadata read hostname 'smithi045' 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd39c876740 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55db81d3cc50 0x55db81d3f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd39c876740 4 client.25993 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd39c876740 1 -- 172.21.15.45:0/2548022020 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55db8192f600 con 0x55db81d3cc50 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.844+0000 7fd3937fe640 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55db81d3cc50 0x55db81d3f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.845+0000 7fd372ffd640 10 client.25993 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.845+0000 7fd3937fe640 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55db81d3cc50 0x55db81d3f070 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa524dfc79bfcec server_cookie=efe7765b4a7cd12b in_seq=0 out_seq=0 2024-01-31T05:35:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd372ffd640 1 -- 172.21.15.45:0/2548022020 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55db8192f600 con 0x55db81d3cc50 2024-01-31T05:35:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd372ffd640 10 client.25993 handle_command_reply: tid=0 2024-01-31T05:35:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd39c876740 1 client.25993 shutdown 2024-01-31T05:35:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd39c876740 1 -- 172.21.15.45:0/2548022020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cc8420 msgr2=0x55db81cdaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd39c876740 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cc8420 0x55db81cdaa70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd388014010 tx=0x7fd388004610 comp rx=0 tx=0).stop 2024-01-31T05:35:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd39c876740 1 -- 172.21.15.45:0/2548022020 shutdown_connections 2024-01-31T05:35:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd39c876740 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55db81d3cc50 0x55db81d3f070 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd39c876740 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db81cd5b60 0x55db81cdafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd39c876740 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db81cc8420 0x55db81cdaa70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd372ffd640 0 client.25993 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd39c876740 1 --2- 172.21.15.45:0/2548022020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db81cc7b00 0x55db81cdc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd39c876740 1 -- 172.21.15.45:0/2548022020 >> 172.21.15.45:0/2548022020 conn(0x55db81cc2980 msgr2=0x55db81cd12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd39c876740 1 -- 172.21.15.45:0/2548022020 shutdown_connections 2024-01-31T05:35:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd39c876740 1 -- 172.21.15.45:0/2548022020 wait complete. 2024-01-31T05:35:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.857+0000 7fd39c876740 20 client.25993 trim_cache size 0 max 16384 2024-01-31T05:35:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:35:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:35:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:35:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:35:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.858+0000 7fd399ab8640 1 -- 172.21.15.45:0/2084249571 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd37c0a14f0 msgr2=0x7fd37c0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.858+0000 7fd399ab8640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd37c0a14f0 0x7fd37c0a39b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd38c005fd0 tx=0x7fd38c013b60 comp rx=0 tx=0).stop 2024-01-31T05:35:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.858+0000 7fd399ab8640 1 -- 172.21.15.45:0/2084249571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3940ffe40 msgr2=0x7fd3941972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.858+0000 7fd399ab8640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3940ffe40 0x7fd3941972d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd384005940 tx=0x7fd38402c670 comp rx=0 tx=0).stop 2024-01-31T05:35:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.858+0000 7fd399ab8640 1 -- 172.21.15.45:0/2084249571 shutdown_connections 2024-01-31T05:35:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.858+0000 7fd399ab8640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fd37c0a14f0 0x7fd37c0a39b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.858+0000 7fd399ab8640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd394104b90 0x7fd39419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.858+0000 7fd399ab8640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd394100800 0x7fd394197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.858+0000 7fd399ab8640 1 --2- 172.21.15.45:0/2084249571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3940ffe40 0x7fd3941972d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.858+0000 7fd399ab8640 1 -- 172.21.15.45:0/2084249571 >> 172.21.15.45:0/2084249571 conn(0x7fd394076a40 msgr2=0x7fd3940fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.858+0000 7fd399ab8640 1 -- 172.21.15.45:0/2084249571 shutdown_connections 2024-01-31T05:35:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:23.858+0000 7fd399ab8640 1 -- 172.21.15.45:0/2084249571 wait complete. 2024-01-31T05:35:24.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1341719 10.33 MB/sec execute 564 sec latency 69.829 ms 2024-01-31T05:35:24.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:24 smithi156 ceph-mon[35720]: pgmap v404: 97 pgs: 97 active+clean; 618 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 212 op/s 2024-01-31T05:35:25.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:24 smithi186 ceph-mon[35659]: pgmap v404: 97 pgs: 97 active+clean; 618 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 212 op/s 2024-01-31T05:35:25.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:24 smithi045 ceph-mon[31585]: pgmap v404: 97 pgs: 97 active+clean; 618 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 212 op/s 2024-01-31T05:35:25.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1342958 10.33 MB/sec execute 565 sec latency 17.317 ms 2024-01-31T05:35:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:25 smithi156 ceph-mon[35720]: pgmap v405: 97 pgs: 97 active+clean; 618 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 212 op/s 2024-01-31T05:35:26.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:25 smithi186 ceph-mon[35659]: pgmap v405: 97 pgs: 97 active+clean; 618 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 212 op/s 2024-01-31T05:35:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:25 smithi045 ceph-mon[31585]: pgmap v405: 97 pgs: 97 active+clean; 618 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 212 op/s 2024-01-31T05:35:26.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1345263 10.33 MB/sec execute 566 sec latency 18.105 ms 2024-01-31T05:35:27.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1346299 10.32 MB/sec execute 567 sec latency 72.082 ms 2024-01-31T05:35:28.803 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1348275 10.32 MB/sec execute 568 sec latency 26.440 ms 2024-01-31T05:35:28.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:28 smithi156 ceph-mon[35720]: pgmap v406: 97 pgs: 97 active+clean; 631 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 200 op/s 2024-01-31T05:35:29.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:28 smithi186 ceph-mon[35659]: pgmap v406: 97 pgs: 97 active+clean; 631 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 200 op/s 2024-01-31T05:35:29.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:28 smithi045 ceph-mon[31585]: pgmap v406: 97 pgs: 97 active+clean; 631 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 200 op/s 2024-01-31T05:35:29.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1349760 10.32 MB/sec execute 569 sec latency 13.716 ms 2024-01-31T05:35:29.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:29 smithi156 ceph-mon[35720]: pgmap v407: 97 pgs: 97 active+clean; 631 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-01-31T05:35:30.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:29 smithi186 ceph-mon[35659]: pgmap v407: 97 pgs: 97 active+clean; 631 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-01-31T05:35:30.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:29 smithi045 ceph-mon[31585]: pgmap v407: 97 pgs: 97 active+clean; 631 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-01-31T05:35:30.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1351092 10.31 MB/sec execute 570 sec latency 141.149 ms 2024-01-31T05:35:31.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1353105 10.32 MB/sec execute 571 sec latency 16.269 ms 2024-01-31T05:35:31.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:31 smithi156 ceph-mon[35720]: pgmap v408: 97 pgs: 97 active+clean; 631 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-01-31T05:35:32.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:31 smithi186 ceph-mon[35659]: pgmap v408: 97 pgs: 97 active+clean; 631 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-01-31T05:35:32.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:31 smithi045 ceph-mon[31585]: pgmap v408: 97 pgs: 97 active+clean; 631 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-01-31T05:35:32.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1354248 10.30 MB/sec execute 572 sec latency 59.905 ms 2024-01-31T05:35:33.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1356616 10.31 MB/sec execute 573 sec latency 15.679 ms 2024-01-31T05:35:33.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:33 smithi156 ceph-mon[35720]: pgmap v409: 97 pgs: 97 active+clean; 568 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 247 op/s 2024-01-31T05:35:34.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:33 smithi186 ceph-mon[35659]: pgmap v409: 97 pgs: 97 active+clean; 568 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 247 op/s 2024-01-31T05:35:34.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:33 smithi045 ceph-mon[31585]: pgmap v409: 97 pgs: 97 active+clean; 568 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 247 op/s 2024-01-31T05:35:34.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1357978 10.31 MB/sec execute 574 sec latency 19.704 ms 2024-01-31T05:35:35.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1360097 10.31 MB/sec execute 575 sec latency 102.856 ms 2024-01-31T05:35:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:35 smithi156 ceph-mon[35720]: pgmap v410: 97 pgs: 97 active+clean; 568 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 161 op/s 2024-01-31T05:35:36.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:35 smithi186 ceph-mon[35659]: pgmap v410: 97 pgs: 97 active+clean; 568 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 161 op/s 2024-01-31T05:35:36.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:35 smithi045 ceph-mon[31585]: pgmap v410: 97 pgs: 97 active+clean; 568 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 161 op/s 2024-01-31T05:35:36.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1361109 10.30 MB/sec execute 576 sec latency 24.163 ms 2024-01-31T05:35:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.950+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/82266560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c4106e00 msgr2=0x7fb7c41071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.950+0000 7fb7cbdd9640 1 --2- 172.21.15.45:0/82266560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c4106e00 0x7fb7c41071e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009900 tx=0x7fb7c002bc30 comp rx=0 tx=0).stop 2024-01-31T05:35:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.950+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/82266560 shutdown_connections 2024-01-31T05:35:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.950+0000 7fb7cbdd9640 1 --2- 172.21.15.45:0/82266560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7c4068d10 0x7fb7c410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.950+0000 7fb7cbdd9640 1 --2- 172.21.15.45:0/82266560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7c4068370 0x7fb7c40687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.950+0000 7fb7cbdd9640 1 --2- 172.21.15.45:0/82266560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c4106e00 0x7fb7c41071e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.950+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/82266560 >> 172.21.15.45:0/82266560 conn(0x7fb7c4076a40 msgr2=0x7fb7c4076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.950+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/82266560 shutdown_connections 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.950+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/82266560 wait complete. 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7cbdd9640 1 Processor -- start 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7cbdd9640 1 -- start start 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7cbdd9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7c4068370 0x7fb7c4197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7cbdd9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c4068d10 0x7fb7c41976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7cbdd9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7c4106e00 0x7fb7c419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7cbdd9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c419c000 con 0x7fb7c4068d10 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7cbdd9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c419c170 con 0x7fb7c4068370 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7cbdd9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c419c330 con 0x7fb7c4106e00 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7c9374640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c4068d10 0x7fb7c41976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7c9374640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c4068d10 0x7fb7c41976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45484/0 (socket says 172.21.15.45:45484) 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7c9374640 1 -- 172.21.15.45:0/2139328631 learned_addr learned my addr 172.21.15.45:0/2139328631 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7ca376640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7c4106e00 0x7fb7c419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7c9b75640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7c4068370 0x7fb7c4197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7c9374640 1 -- 172.21.15.45:0/2139328631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7c4106e00 msgr2=0x7fb7c419ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7c9374640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7c4106e00 0x7fb7c419ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7c9374640 1 -- 172.21.15.45:0/2139328631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7c4068370 msgr2=0x7fb7c4197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7c9374640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7c4068370 0x7fb7c4197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7c9374640 1 -- 172.21.15.45:0/2139328631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c0009560 con 0x7fb7c4068d10 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.951+0000 7fb7ca376640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7c4106e00 0x7fb7c419ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.952+0000 7fb7c9374640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c4068d10 0x7fb7c41976d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400e9f0 tx=0x7fb7b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.952+0000 7fb7baffd640 1 -- 172.21.15.45:0/2139328631 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4009700 con 0x7fb7c4068d10 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.952+0000 7fb7baffd640 1 -- 172.21.15.45:0/2139328631 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7b400cd80 con 0x7fb7c4068d10 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.952+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/2139328631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c419c610 con 0x7fb7c4068d10 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.952+0000 7fb7baffd640 1 -- 172.21.15.45:0/2139328631 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4018860 con 0x7fb7c4068d10 2024-01-31T05:35:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.952+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/2139328631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c41a3e90 con 0x7fb7c4068d10 2024-01-31T05:35:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.954+0000 7fb7baffd640 1 -- 172.21.15.45:0/2139328631 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fb7b40189c0 con 0x7fb7c4068d10 2024-01-31T05:35:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7ceb9b740 1 Processor -- start 2024-01-31T05:35:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7ceb9b740 1 -- start start 2024-01-31T05:35:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7baffd640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb7b00a14f0 0x7fb7b00a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7ceb9b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f060 0x55ed7b19f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7ceb9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f980 0x55ed7b1a8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7ceb9b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b1a8750 0x55ed7b1aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7ceb9b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7b10e040 con 0x55ed7b19f060 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7ceb9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7b10b730 con 0x55ed7b19f980 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7ceb9b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7b107020 con 0x55ed7b1a8750 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7ca376640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb7b00a14f0 0x7fb7b00a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7baffd640 1 -- 172.21.15.45:0/2139328631 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb7b4014070 con 0x7fb7c4068d10 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7c9374640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f980 0x55ed7b1a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7ca376640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b1a8750 0x55ed7b1aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7c9b75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f060 0x55ed7b19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7c9374640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f980 0x55ed7b1a8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:59478/0 (socket says 172.21.15.45:59478) 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.955+0000 7fb7c9374640 1 -- 172.21.15.45:0/2439090149 learned_addr learned my addr 172.21.15.45:0/2439090149 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.956+0000 7fb7c9374640 1 -- 172.21.15.45:0/2439090149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b1a8750 msgr2=0x55ed7b1aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.956+0000 7fb7c9374640 1 --2- 172.21.15.45:0/2439090149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b1a8750 0x55ed7b1aab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.956+0000 7fb7c9374640 1 -- 172.21.15.45:0/2439090149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f060 msgr2=0x55ed7b19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.956+0000 7fb7c9374640 1 --2- 172.21.15.45:0/2439090149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f060 0x55ed7b19f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.956+0000 7fb7c9374640 1 -- 172.21.15.45:0/2439090149 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed7b0fee40 con 0x55ed7b19f980 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.956+0000 7fb7ca376640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb7b00a14f0 0x7fb7b00a39b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009fd0 tx=0x7fb7bc011720 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.956+0000 7fb7c9374640 1 --2- 172.21.15.45:0/2439090149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f980 0x55ed7b1a8210 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009380 tx=0x7fb7b4099ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=941b26a4b4a1fe29 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.956+0000 7fb7affff640 1 -- 172.21.15.45:0/2439090149 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4017780 con 0x55ed7b19f980 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.956+0000 7fb7affff640 1 -- 172.21.15.45:0/2439090149 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7b409a7e0 con 0x55ed7b19f980 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.956+0000 7fb7affff640 1 -- 172.21.15.45:0/2439090149 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40a24e0 con 0x55ed7b19f980 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.957+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/2439090149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f980 msgr2=0x55ed7b1a8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.957+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/2439090149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f980 0x55ed7b1a8210 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009380 tx=0x7fb7b4099ce0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.957+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/2439090149 shutdown_connections 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.957+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/2439090149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b1a8750 0x55ed7b1aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.957+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/2439090149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f980 0x55ed7b1a8210 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.957+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/2439090149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f060 0x55ed7b19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.957+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/2439090149 >> 172.21.15.45:0/2439090149 conn(0x55ed7b199980 msgr2=0x55ed7b199d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.957+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/2439090149 shutdown_connections 2024-01-31T05:35:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.957+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/2439090149 wait complete. 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.957+0000 7fb7ceb9b740 1 -- start start 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.957+0000 7fb7ceb9b740 1 Processor -- start 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ceb9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f060 0x55ed7b1a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ceb9b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 0x55ed7b1a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ceb9b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b1a8750 0x55ed7b1a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ceb9b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7b0c01b0 con 0x55ed7b19f980 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ceb9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7b0bfe50 con 0x55ed7b19f060 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ceb9b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7b085090 con 0x55ed7b1a8750 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ceb9b740 10 client.0 _ll_register_callbacks cb 0x55ed7b0aaa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7c9374640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 0x55ed7b1a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7c9374640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 0x55ed7b1a5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45496/0 (socket says 172.21.15.45:45496) 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7c9374640 1 -- 172.21.15.45:0/782209481 learned_addr learned my addr 172.21.15.45:0/782209481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ca376640 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b1a8750 0x55ed7b1a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7c9b75640 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f060 0x55ed7b1a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7c9374640 1 -- 172.21.15.45:0/782209481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b1a8750 msgr2=0x55ed7b1a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7c9374640 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b1a8750 0x55ed7b1a61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7c9374640 1 -- 172.21.15.45:0/782209481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f060 msgr2=0x55ed7b1a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7c9374640 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f060 0x55ed7b1a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7c9374640 1 -- 172.21.15.45:0/782209481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c41a3e90 con 0x55ed7b19f980 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ca376640 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b1a8750 0x55ed7b1a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7c9b75640 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f060 0x55ed7b1a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7c9374640 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 0x55ed7b1a5ca0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4099de0 tx=0x7fb7b401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a828f81d92e549b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ad7fa640 1 -- 172.21.15.45:0/782209481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40caeb0 con 0x55ed7b19f980 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ad7fa640 1 -- 172.21.15.45:0/782209481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb7b409a0e0 con 0x55ed7b19f980 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.958+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/782209481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed7b100010 con 0x55ed7b19f980 2024-01-31T05:35:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7ad7fa640 1 -- 172.21.15.45:0/782209481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7b409a340 con 0x55ed7b19f980 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7ceb9b740 10 client.16443 fetch_fsmap learned FSMap version 30 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/782209481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ed7b1056a0 con 0x55ed7b19f980 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7ad7fa640 1 -- 172.21.15.45:0/782209481 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7b40a6080 con 0x55ed7b19f980 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7ad7fa640 1 -- 172.21.15.45:0/782209481 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb7b40b8020 con 0x55ed7b19f980 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7ceb9b740 10 client.16443 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7ceb9b740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7ceb9b740 20 client.16443 populate_metadata read hostname 'smithi045' 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ed7b1b8300 0x55ed7b1ba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7ceb9b740 4 client.16443 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/782209481 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed7b0ffd30 con 0x55ed7b1b8300 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7c9b75640 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ed7b1b8300 0x55ed7b1ba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7c9b75640 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ed7b1b8300 0x55ed7b1ba6f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a4a7f16189357c1 server_cookie=d3595ddd52e0e3cf in_seq=0 out_seq=0 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.959+0000 7fb7ad7fa640 10 client.16443 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ad7fa640 1 -- 172.21.15.45:0/782209481 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ed7b0ffd30 con 0x55ed7b1b8300 2024-01-31T05:35:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ad7fa640 10 client.16443 handle_command_reply: tid=0 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ceb9b740 1 client.16443 shutdown 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/782209481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 msgr2=0x55ed7b1a5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 0x55ed7b1a5ca0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4099de0 tx=0x7fb7b401f000 comp rx=0 tx=0).stop 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/782209481 shutdown_connections 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ed7b1b8300 0x55ed7b1ba6f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 0x55ed7b1a5ca0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b1a8750 0x55ed7b1a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ad7fa640 0 client.16443 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/782209481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b19f060 0x55ed7b1a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/782209481 >> 172.21.15.45:0/782209481 conn(0x55ed7b199980 msgr2=0x55ed7b19b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/782209481 shutdown_connections 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.960+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/782209481 wait complete. 2024-01-31T05:35:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.961+0000 7fb7ceb9b740 20 client.16443 trim_cache size 0 max 16384 2024-01-31T05:35:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7ceb9b740 1 Processor -- start 2024-01-31T05:35:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7ceb9b740 1 -- start start 2024-01-31T05:35:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7ceb9b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f060 0x55ed7b1a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7ceb9b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b19f980 0x55ed7b1a6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7ceb9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b1a8750 0x55ed7b1a4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7ceb9b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7b1b8100 con 0x55ed7b19f060 2024-01-31T05:35:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7ceb9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7b19ab40 con 0x55ed7b1a8750 2024-01-31T05:35:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7ceb9b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7b1bf900 con 0x55ed7b19f980 2024-01-31T05:35:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7c9b75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f060 0x55ed7b1a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7c9b75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f060 0x55ed7b1a7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45510/0 (socket says 172.21.15.45:45510) 2024-01-31T05:35:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7c9b75640 1 -- 172.21.15.45:0/2886223574 learned_addr learned my addr 172.21.15.45:0/2886223574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7c9374640 1 --2- 172.21.15.45:0/2886223574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b19f980 0x55ed7b1a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7c9b75640 1 -- 172.21.15.45:0/2886223574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b19f980 msgr2=0x55ed7b1a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7c9b75640 1 --2- 172.21.15.45:0/2886223574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b19f980 0x55ed7b1a6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7c9b75640 1 -- 172.21.15.45:0/2886223574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b1a8750 msgr2=0x55ed7b1a4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7c9b75640 1 --2- 172.21.15.45:0/2886223574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b1a8750 0x55ed7b1a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.974+0000 7fb7c9b75640 1 -- 172.21.15.45:0/2886223574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed7b0a2030 con 0x55ed7b19f060 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.975+0000 7fb7c9b75640 1 --2- 172.21.15.45:0/2886223574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f060 0x55ed7b1a7e90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009900 tx=0x7fb7c0004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b513024091a86f12 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.975+0000 7fb7af7fe640 1 -- 172.21.15.45:0/2886223574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0043040 con 0x55ed7b19f060 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.975+0000 7fb7af7fe640 1 -- 172.21.15.45:0/2886223574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c0045040 con 0x55ed7b19f060 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.975+0000 7fb7af7fe640 1 -- 172.21.15.45:0/2886223574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c004a690 con 0x55ed7b19f060 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.975+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/2886223574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f060 msgr2=0x55ed7b1a7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.975+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/2886223574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f060 0x55ed7b1a7e90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009900 tx=0x7fb7c0004dd0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.976+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/2886223574 shutdown_connections 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.976+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/2886223574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b1a8750 0x55ed7b1a4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.976+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/2886223574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b19f980 0x55ed7b1a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.976+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/2886223574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f060 0x55ed7b1a7e90 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.976+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/2886223574 >> 172.21.15.45:0/2886223574 conn(0x55ed7b199980 msgr2=0x55ed7b1a0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.976+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/2886223574 shutdown_connections 2024-01-31T05:35:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.976+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/2886223574 wait complete. 2024-01-31T05:35:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.976+0000 7fb7ceb9b740 1 -- start start 2024-01-31T05:35:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.976+0000 7fb7ceb9b740 1 Processor -- start 2024-01-31T05:35:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.976+0000 7fb7ceb9b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b19f060 0x55ed7b1af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.976+0000 7fb7ceb9b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 0x55ed7b1add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ceb9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b1a8750 0x55ed7b1ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ceb9b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7b1ae9f0 con 0x55ed7b19f980 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ceb9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7b1aeb60 con 0x55ed7b1a8750 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ceb9b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7b212c10 con 0x55ed7b19f060 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ceb9b740 10 client.0 _ll_register_callbacks cb 0x7fb7b401cf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7c9374640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 0x55ed7b1add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ca376640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b1a8750 0x55ed7b1ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7c9b75640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b19f060 0x55ed7b1af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7c9374640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 0x55ed7b1add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45518/0 (socket says 172.21.15.45:45518) 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7c9374640 1 -- 172.21.15.45:0/4031372566 learned_addr learned my addr 172.21.15.45:0/4031372566 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ca376640 1 -- 172.21.15.45:0/4031372566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b19f060 msgr2=0x55ed7b1af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ca376640 1 --2- 172.21.15.45:0/4031372566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b19f060 0x55ed7b1af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ca376640 1 -- 172.21.15.45:0/4031372566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 msgr2=0x55ed7b1add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ca376640 1 --2- 172.21.15.45:0/4031372566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 0x55ed7b1add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ca376640 1 -- 172.21.15.45:0/4031372566 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c410e480 con 0x55ed7b1a8750 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7c9374640 1 --2- 172.21.15.45:0/4031372566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 0x55ed7b1add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ca376640 1 --2- 172.21.15.45:0/4031372566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b1a8750 0x55ed7b1ae2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55ed7b19ada0 tx=0x7fb7bc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75609330d93e5fcf server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ad7fa640 1 -- 172.21.15.45:0/4031372566 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc016050 con 0x55ed7b1a8750 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/4031372566 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed7b195080 con 0x55ed7b1a8750 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.977+0000 7fb7ad7fa640 1 -- 172.21.15.45:0/4031372566 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb7bc004710 con 0x55ed7b1a8750 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.978+0000 7fb7ad7fa640 1 -- 172.21.15.45:0/4031372566 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7bc004b20 con 0x55ed7b1a8750 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.978+0000 7fb7ceb9b740 10 client.25943 fetch_fsmap learned FSMap version 30 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.978+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/4031372566 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ed7b0c7a50 con 0x55ed7b1a8750 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.978+0000 7fb7ad7fa640 1 -- 172.21.15.45:0/4031372566 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed7b195080 con 0x55ed7b1a8750 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.978+0000 7fb7ad7fa640 1 -- 172.21.15.45:0/4031372566 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb7bc030070 con 0x55ed7b1a8750 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.978+0000 7fb7ceb9b740 10 client.25943 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.978+0000 7fb7ceb9b740 10 client.25943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.978+0000 7fb7ceb9b740 20 client.25943 populate_metadata read hostname 'smithi045' 2024-01-31T05:35:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.978+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/4031372566 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ed7b214470 0x55ed7b216890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.978+0000 7fb7ceb9b740 4 client.25943 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:35:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.978+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/4031372566 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fb7b409a7e0 con 0x55ed7b214470 2024-01-31T05:35:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.978+0000 7fb7c9b75640 1 --2- 172.21.15.45:0/4031372566 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ed7b214470 0x55ed7b216890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.979+0000 7fb7ad7fa640 10 client.25943 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.979+0000 7fb7c9b75640 1 --2- 172.21.15.45:0/4031372566 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ed7b214470 0x55ed7b216890 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a08f5659c2a7dd8 server_cookie=c872a7961ce99175 in_seq=0 out_seq=0 2024-01-31T05:35:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.991+0000 7fb7ad7fa640 1 -- 172.21.15.45:0/4031372566 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fb7b409a7e0 con 0x55ed7b214470 2024-01-31T05:35:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.991+0000 7fb7ad7fa640 10 client.25943 handle_command_reply: tid=0 2024-01-31T05:35:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.991+0000 7fb7ceb9b740 1 client.25943 shutdown 2024-01-31T05:35:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/4031372566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b1a8750 msgr2=0x55ed7b1ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/4031372566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b1a8750 0x55ed7b1ae2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55ed7b19ada0 tx=0x7fb7bc004a40 comp rx=0 tx=0).stop 2024-01-31T05:35:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/4031372566 shutdown_connections 2024-01-31T05:35:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/4031372566 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ed7b214470 0x55ed7b216890 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/4031372566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed7b19f980 0x55ed7b1add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7ad7fa640 0 client.25943 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/4031372566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed7b1a8750 0x55ed7b1ae2f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7ceb9b740 1 --2- 172.21.15.45:0/4031372566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed7b19f060 0x55ed7b1af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/4031372566 >> 172.21.15.45:0/4031372566 conn(0x55ed7b199980 msgr2=0x55ed7b1a3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/4031372566 shutdown_connections 2024-01-31T05:35:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7ceb9b740 1 -- 172.21.15.45:0/4031372566 wait complete. 2024-01-31T05:35:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7ceb9b740 20 client.25943 trim_cache size 0 max 16384 2024-01-31T05:35:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:35:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:35:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:35:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:35:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/2139328631 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb7b00a14f0 msgr2=0x7fb7b00a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7cbdd9640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb7b00a14f0 0x7fb7b00a39b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009fd0 tx=0x7fb7bc011720 comp rx=0 tx=0).stop 2024-01-31T05:35:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/2139328631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c4068d10 msgr2=0x7fb7c41976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7cbdd9640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c4068d10 0x7fb7c41976d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400e9f0 tx=0x7fb7b400eeb0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/2139328631 shutdown_connections 2024-01-31T05:35:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7cbdd9640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fb7b00a14f0 0x7fb7b00a39b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7cbdd9640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c4068d10 0x7fb7c41976d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7cbdd9640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7c4106e00 0x7fb7c419ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7cbdd9640 1 --2- 172.21.15.45:0/2139328631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7c4068370 0x7fb7c4197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.992+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/2139328631 >> 172.21.15.45:0/2139328631 conn(0x7fb7c4076a40 msgr2=0x7fb7c40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.993+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/2139328631 shutdown_connections 2024-01-31T05:35:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:36.993+0000 7fb7cbdd9640 1 -- 172.21.15.45:0/2139328631 wait complete. 2024-01-31T05:35:37.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1363266 10.30 MB/sec execute 577 sec latency 88.781 ms 2024-01-31T05:35:38.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1364356 10.29 MB/sec execute 578 sec latency 18.942 ms 2024-01-31T05:35:38.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:38 smithi156 ceph-mon[35720]: pgmap v411: 97 pgs: 97 active+clean; 582 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 243 op/s 2024-01-31T05:35:39.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:38 smithi186 ceph-mon[35659]: pgmap v411: 97 pgs: 97 active+clean; 582 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 243 op/s 2024-01-31T05:35:39.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:38 smithi045 ceph-mon[31585]: pgmap v411: 97 pgs: 97 active+clean; 582 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 243 op/s 2024-01-31T05:35:39.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1366829 10.30 MB/sec execute 579 sec latency 14.414 ms 2024-01-31T05:35:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:39 smithi156 ceph-mon[35720]: pgmap v412: 97 pgs: 97 active+clean; 582 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 163 op/s 2024-01-31T05:35:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:39 smithi186 ceph-mon[35659]: pgmap v412: 97 pgs: 97 active+clean; 582 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 163 op/s 2024-01-31T05:35:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:39 smithi045 ceph-mon[31585]: pgmap v412: 97 pgs: 97 active+clean; 582 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 163 op/s 2024-01-31T05:35:40.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1367785 10.29 MB/sec execute 580 sec latency 73.630 ms 2024-01-31T05:35:41.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1369901 10.29 MB/sec execute 581 sec latency 30.873 ms 2024-01-31T05:35:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:41 smithi156 ceph-mon[35720]: pgmap v413: 97 pgs: 97 active+clean; 582 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 163 op/s 2024-01-31T05:35:42.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:41 smithi186 ceph-mon[35659]: pgmap v413: 97 pgs: 97 active+clean; 582 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 163 op/s 2024-01-31T05:35:42.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:41 smithi045 ceph-mon[31585]: pgmap v413: 97 pgs: 97 active+clean; 582 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 163 op/s 2024-01-31T05:35:42.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1371112 10.29 MB/sec execute 582 sec latency 94.243 ms 2024-01-31T05:35:43.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1373095 10.28 MB/sec execute 583 sec latency 20.360 ms 2024-01-31T05:35:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:43 smithi156 ceph-mon[35720]: pgmap v414: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 246 op/s 2024-01-31T05:35:44.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:43 smithi186 ceph-mon[35659]: pgmap v414: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 246 op/s 2024-01-31T05:35:44.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:43 smithi045 ceph-mon[31585]: pgmap v414: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 246 op/s 2024-01-31T05:35:44.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1374554 10.28 MB/sec execute 584 sec latency 26.127 ms 2024-01-31T05:35:45.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1375389 10.29 MB/sec execute 585 sec latency 88.980 ms 2024-01-31T05:35:45.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:45 smithi156 ceph-mon[35720]: pgmap v415: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 165 op/s 2024-01-31T05:35:46.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:45 smithi186 ceph-mon[35659]: pgmap v415: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 165 op/s 2024-01-31T05:35:46.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:45 smithi045 ceph-mon[31585]: pgmap v415: 97 pgs: 97 active+clean; 595 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 165 op/s 2024-01-31T05:35:46.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1377438 10.30 MB/sec execute 586 sec latency 18.972 ms 2024-01-31T05:35:47.807 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1379065 10.30 MB/sec execute 587 sec latency 78.681 ms 2024-01-31T05:35:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:47 smithi156 ceph-mon[35720]: pgmap v416: 97 pgs: 97 active+clean; 605 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.2 MiB/s wr, 244 op/s 2024-01-31T05:35:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:47 smithi186 ceph-mon[35659]: pgmap v416: 97 pgs: 97 active+clean; 605 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.2 MiB/s wr, 244 op/s 2024-01-31T05:35:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:47 smithi045 ceph-mon[31585]: pgmap v416: 97 pgs: 97 active+clean; 605 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.2 MiB/s wr, 244 op/s 2024-01-31T05:35:48.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1381356 10.30 MB/sec execute 588 sec latency 16.677 ms 2024-01-31T05:35:49.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1383057 10.30 MB/sec execute 589 sec latency 18.873 ms 2024-01-31T05:35:49.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:49 smithi156 ceph-mon[35720]: pgmap v417: 97 pgs: 97 active+clean; 605 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 161 op/s 2024-01-31T05:35:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.081+0000 7f5690798640 1 -- 172.21.15.45:0/3746714264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688106e00 msgr2=0x7f56881071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.081+0000 7f5690798640 1 --2- 172.21.15.45:0/3746714264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688106e00 0x7f56881071e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5684009900 tx=0x7f568402bc30 comp rx=0 tx=0).stop 2024-01-31T05:35:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.081+0000 7f5690798640 1 -- 172.21.15.45:0/3746714264 shutdown_connections 2024-01-31T05:35:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.081+0000 7f5690798640 1 --2- 172.21.15.45:0/3746714264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5688068d10 0x7f568810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.081+0000 7f5690798640 1 --2- 172.21.15.45:0/3746714264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5688068370 0x7f56880687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.081+0000 7f5690798640 1 --2- 172.21.15.45:0/3746714264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688106e00 0x7f56881071e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.081+0000 7f5690798640 1 -- 172.21.15.45:0/3746714264 >> 172.21.15.45:0/3746714264 conn(0x7f5688076a40 msgr2=0x7f5688076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.081+0000 7f5690798640 1 -- 172.21.15.45:0/3746714264 shutdown_connections 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.081+0000 7f5690798640 1 -- 172.21.15.45:0/3746714264 wait complete. 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f5690798640 1 Processor -- start 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f5690798640 1 -- start start 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f5690798640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5688068370 0x7f56880fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f5690798640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688068d10 0x7f5688100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f5690798640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5688106e00 0x7f5688100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f5690798640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56881a54c0 con 0x7f5688068d10 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f5690798640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56881a5630 con 0x7f5688106e00 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f5690798640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56881a57f0 con 0x7f5688068370 2024-01-31T05:35:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f568dd33640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688068d10 0x7f5688100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f568e534640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5688068370 0x7f56880fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f568dd33640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688068d10 0x7f5688100450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35922/0 (socket says 172.21.15.45:35922) 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f568dd33640 1 -- 172.21.15.45:0/741542311 learned_addr learned my addr 172.21.15.45:0/741542311 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f568e534640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5688068370 0x7f56880fff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:60388/0 (socket says 172.21.15.45:60388) 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f568ed35640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5688106e00 0x7f5688100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f568dd33640 1 -- 172.21.15.45:0/741542311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5688068370 msgr2=0x7f56880fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f568dd33640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5688068370 0x7f56880fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f568dd33640 1 -- 172.21.15.45:0/741542311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5688106e00 msgr2=0x7f5688100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f568dd33640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5688106e00 0x7f5688100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f568dd33640 1 -- 172.21.15.45:0/741542311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684009560 con 0x7f5688068d10 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.082+0000 7f568dd33640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688068d10 0x7f5688100450 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f567800d950 tx=0x7f567800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.083+0000 7f567f7fe640 1 -- 172.21.15.45:0/741542311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678009a30 con 0x7f5688068d10 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.083+0000 7f567f7fe640 1 -- 172.21.15.45:0/741542311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5678009b90 con 0x7f5688068d10 2024-01-31T05:35:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.083+0000 7f567f7fe640 1 -- 172.21.15.45:0/741542311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56780047a0 con 0x7f5688068d10 2024-01-31T05:35:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.083+0000 7f5690798640 1 -- 172.21.15.45:0/741542311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56881a5ad0 con 0x7f5688068d10 2024-01-31T05:35:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.083+0000 7f5690798640 1 -- 172.21.15.45:0/741542311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56881a5f90 con 0x7f5688068d10 2024-01-31T05:35:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.085+0000 7f567f7fe640 1 -- 172.21.15.45:0/741542311 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f5678009d00 con 0x7f5688068d10 2024-01-31T05:35:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.085+0000 7f5693633740 1 Processor -- start 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.085+0000 7f5693633740 1 -- start start 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f5693633740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f5693633740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f5693633740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cbbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f5693633740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d4c1f040 con 0x55d8d4cb9750 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f5693633740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d4c1c730 con 0x55d8d4cb0980 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f5693633740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d4c18020 con 0x55d8d4cb0060 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f567f7fe640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f56740a1520 0x7f56740a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568e534640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568ed35640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568dd33640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cbbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568e534640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:38002/0 (socket says 172.21.15.45:38002) 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568e534640 1 -- 172.21.15.45:0/3431734705 learned_addr learned my addr 172.21.15.45:0/3431734705 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568ed35640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:60396/0 (socket says 172.21.15.45:60396) 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568ed35640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f56740a1520 0x7f56740a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f567f7fe640 1 -- 172.21.15.45:0/741542311 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f56780cc990 con 0x7f5688068d10 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568e534640 1 -- 172.21.15.45:0/3431734705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 msgr2=0x55d8d4cb0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568e534640 1 --2- 172.21.15.45:0/3431734705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568e534640 1 -- 172.21.15.45:0/3431734705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 msgr2=0x55d8d4cbbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568e534640 1 --2- 172.21.15.45:0/3431734705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cbbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568e534640 1 -- 172.21.15.45:0/3431734705 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8d4c0fe40 con 0x55d8d4cb0980 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568e534640 1 --2- 172.21.15.45:0/3431734705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb9210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5684000c00 tx=0x7f56840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ff68a7ce650070 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.086+0000 7f568ed35640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f56740a1520 0x7f56740a39e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5680005fd0 tx=0x7f5680009500 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.087+0000 7f567cff9640 1 -- 172.21.15.45:0/3431734705 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56840048b0 con 0x55d8d4cb0980 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.087+0000 7f567cff9640 1 -- 172.21.15.45:0/3431734705 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5684004a10 con 0x55d8d4cb0980 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.087+0000 7f567cff9640 1 -- 172.21.15.45:0/3431734705 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684003850 con 0x55d8d4cb0980 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.087+0000 7f5693633740 1 -- 172.21.15.45:0/3431734705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 msgr2=0x55d8d4cb9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.087+0000 7f5693633740 1 --2- 172.21.15.45:0/3431734705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb9210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5684000c00 tx=0x7f56840046c0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.087+0000 7f5693633740 1 -- 172.21.15.45:0/3431734705 shutdown_connections 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.087+0000 7f5693633740 1 --2- 172.21.15.45:0/3431734705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cbbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.087+0000 7f5693633740 1 --2- 172.21.15.45:0/3431734705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb9210 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.087+0000 7f5693633740 1 --2- 172.21.15.45:0/3431734705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.087+0000 7f5693633740 1 -- 172.21.15.45:0/3431734705 >> 172.21.15.45:0/3431734705 conn(0x55d8d4caa980 msgr2=0x55d8d4caad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.087+0000 7f5693633740 1 -- 172.21.15.45:0/3431734705 shutdown_connections 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.087+0000 7f5693633740 1 -- 172.21.15.45:0/3431734705 wait complete. 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f5693633740 1 -- start start 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f5693633740 1 Processor -- start 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f5693633740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f5693633740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f5693633740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cb7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f5693633740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d4bd11b0 con 0x55d8d4cb9750 2024-01-31T05:35:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f5693633740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d4bd0e50 con 0x55d8d4cb0980 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f5693633740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d4b96090 con 0x55d8d4cb0060 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f5693633740 10 client.0 _ll_register_callbacks cb 0x55d8d4bbba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f568e534640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f568ed35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cb7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f568dd33640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f568e534640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:60404/0 (socket says 172.21.15.45:60404) 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f568e534640 1 -- 172.21.15.45:0/2561059666 learned_addr learned my addr 172.21.15.45:0/2561059666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f568ed35640 1 -- 172.21.15.45:0/2561059666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 msgr2=0x55d8d4cb66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f568ed35640 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f568ed35640 1 -- 172.21.15.45:0/2561059666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 msgr2=0x55d8d4cb6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.088+0000 7f568ed35640 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f568ed35640 1 -- 172.21.15.45:0/2561059666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568810e480 con 0x55d8d4cb9750 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f568dd33640 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:35:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f568e534640 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f568ed35640 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cb7160 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f568000ff20 tx=0x7f5680004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154b70cfc289639f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f5671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f5671ffb640 1 -- 172.21.15.45:0/2561059666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680024070 con 0x55d8d4cb9750 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f5671ffb640 1 -- 172.21.15.45:0/2561059666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f568001fde0 con 0x55d8d4cb9750 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f5693633740 1 -- 172.21.15.45:0/2561059666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8d4c11010 con 0x55d8d4cb9750 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f5671ffb640 1 -- 172.21.15.45:0/2561059666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f568002a420 con 0x55d8d4cb9750 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f5693633740 10 client.16461 fetch_fsmap learned FSMap version 30 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f5693633740 1 -- 172.21.15.45:0/2561059666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d8d4c166a0 con 0x55d8d4cb9750 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f5671ffb640 1 -- 172.21.15.45:0/2561059666 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5680037020 con 0x55d8d4cb9750 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f5671ffb640 1 -- 172.21.15.45:0/2561059666 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5680032440 con 0x55d8d4cb9750 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.089+0000 7f5693633740 10 client.16461 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.090+0000 7f5693633740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.090+0000 7f5693633740 20 client.16461 populate_metadata read hostname 'smithi045' 2024-01-31T05:35:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.090+0000 7f5693633740 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d8d4cc9560 0x55d8d4ccb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.090+0000 7f5693633740 4 client.16461 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.090+0000 7f5693633740 1 -- 172.21.15.45:0/2561059666 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8d4c10d30 con 0x55d8d4cc9560 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.090+0000 7f568e534640 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d8d4cc9560 0x55d8d4ccb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.090+0000 7f568e534640 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d8d4cc9560 0x55d8d4ccb980 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a8ddc4f44bfc0f7 server_cookie=790ed6ae96271e5c in_seq=0 out_seq=0 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.090+0000 7f5671ffb640 10 client.16461 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5671ffb640 1 -- 172.21.15.45:0/2561059666 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d8d4c10d30 con 0x55d8d4cc9560 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5671ffb640 10 client.16461 handle_command_reply: tid=0 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5693633740 1 client.16461 shutdown 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5693633740 1 -- 172.21.15.45:0/2561059666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 msgr2=0x55d8d4cb7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5693633740 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cb7160 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f568000ff20 tx=0x7f5680004a40 comp rx=0 tx=0).stop 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5693633740 1 -- 172.21.15.45:0/2561059666 shutdown_connections 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5693633740 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d8d4cc9560 0x55d8d4ccb980 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5693633740 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cb7160 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5693633740 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5693633740 1 --2- 172.21.15.45:0/2561059666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5671ffb640 0 client.16461 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5693633740 1 -- 172.21.15.45:0/2561059666 >> 172.21.15.45:0/2561059666 conn(0x55d8d4caa980 msgr2=0x55d8d4cac6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5693633740 1 -- 172.21.15.45:0/2561059666 shutdown_connections 2024-01-31T05:35:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5693633740 1 -- 172.21.15.45:0/2561059666 wait complete. 2024-01-31T05:35:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.091+0000 7f5693633740 20 client.16461 trim_cache size 0 max 16384 2024-01-31T05:35:50.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:49 smithi186 ceph-mon[35659]: pgmap v417: 97 pgs: 97 active+clean; 605 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 161 op/s 2024-01-31T05:35:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.104+0000 7f5693633740 1 Processor -- start 2024-01-31T05:35:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.104+0000 7f5693633740 1 -- start start 2024-01-31T05:35:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f5693633740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f5693633740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f5693633740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb9750 0x55d8d4cb19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f5693633740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d4c10d30 con 0x55d8d4cb0060 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f5693633740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d4cc5fd0 con 0x55d8d4cb0980 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f5693633740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d4cbbb70 con 0x55d8d4cb9750 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f568e534640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f568e534640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35936/0 (socket says 172.21.15.45:35936) 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f568e534640 1 -- 172.21.15.45:0/3952260929 learned_addr learned my addr 172.21.15.45:0/3952260929 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f568dd33640 1 --2- 172.21.15.45:0/3952260929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f568ed35640 1 --2- 172.21.15.45:0/3952260929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb9750 0x55d8d4cb19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f568e534640 1 -- 172.21.15.45:0/3952260929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb9750 msgr2=0x55d8d4cb19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f568e534640 1 --2- 172.21.15.45:0/3952260929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb9750 0x55d8d4cb19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f568e534640 1 -- 172.21.15.45:0/3952260929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 msgr2=0x55d8d4cb6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f568e534640 1 --2- 172.21.15.45:0/3952260929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f568e534640 1 -- 172.21.15.45:0/3952260929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8d4bb3030 con 0x55d8d4cb0060 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f568ed35640 1 --2- 172.21.15.45:0/3952260929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb9750 0x55d8d4cb19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f568e534640 1 --2- 172.21.15.45:0/3952260929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb8e10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5684009900 tx=0x7f568402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e04f3fd965b834 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.105+0000 7f5673fff640 1 -- 172.21.15.45:0/3952260929 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684052be0 con 0x55d8d4cb0060 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.106+0000 7f5673fff640 1 -- 172.21.15.45:0/3952260929 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56840044f0 con 0x55d8d4cb0060 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.106+0000 7f5673fff640 1 -- 172.21.15.45:0/3952260929 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56840039b0 con 0x55d8d4cb0060 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.106+0000 7f5693633740 1 -- 172.21.15.45:0/3952260929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb0060 msgr2=0x55d8d4cb8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.106+0000 7f5693633740 1 --2- 172.21.15.45:0/3952260929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb8e10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5684009900 tx=0x7f568402bc30 comp rx=0 tx=0).stop 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.106+0000 7f5693633740 1 -- 172.21.15.45:0/3952260929 shutdown_connections 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.106+0000 7f5693633740 1 --2- 172.21.15.45:0/3952260929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb9750 0x55d8d4cb19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.106+0000 7f5693633740 1 --2- 172.21.15.45:0/3952260929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cb6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.106+0000 7f5693633740 1 --2- 172.21.15.45:0/3952260929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb0060 0x55d8d4cb8e10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.106+0000 7f5693633740 1 -- 172.21.15.45:0/3952260929 >> 172.21.15.45:0/3952260929 conn(0x55d8d4caa980 msgr2=0x55d8d4cd0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.106+0000 7f5693633740 1 -- 172.21.15.45:0/3952260929 shutdown_connections 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.106+0000 7f5693633740 1 -- 172.21.15.45:0/3952260929 wait complete. 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f5693633740 1 -- start start 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f5693633740 1 Processor -- start 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f5693633740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cc03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f5693633740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cc0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f5693633740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cbea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f5693633740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d4cbf0f0 con 0x55d8d4cb9750 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f5693633740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d4cbf260 con 0x55d8d4cb0980 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f5693633740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d4cbf420 con 0x55d8d4cb0060 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f5693633740 10 client.0 _ll_register_callbacks cb 0x7f5684009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f568ed35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cbea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f568dd33640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cc0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f568ed35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cbea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35938/0 (socket says 172.21.15.45:35938) 2024-01-31T05:35:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f568e534640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cc03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f568ed35640 1 -- 172.21.15.45:0/887438137 learned_addr learned my addr 172.21.15.45:0/887438137 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f568ed35640 1 -- 172.21.15.45:0/887438137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 msgr2=0x55d8d4cc03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f568ed35640 1 --2- 172.21.15.45:0/887438137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cc03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f568ed35640 1 -- 172.21.15.45:0/887438137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 msgr2=0x55d8d4cc0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f568ed35640 1 --2- 172.21.15.45:0/887438137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cc0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.107+0000 7f568ed35640 1 -- 172.21.15.45:0/887438137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8d4c0fe40 con 0x55d8d4cb9750 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f568ed35640 1 --2- 172.21.15.45:0/887438137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cbea20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5680032770 tx=0x7f5680032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc88dc60888aec1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5671ffb640 1 -- 172.21.15.45:0/887438137 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680024070 con 0x55d8d4cb9750 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5671ffb640 1 -- 172.21.15.45:0/887438137 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5680038080 con 0x55d8d4cb9750 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5693633740 1 -- 172.21.15.45:0/887438137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5684004a10 con 0x55d8d4cb9750 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5671ffb640 1 -- 172.21.15.45:0/887438137 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56800119a0 con 0x55d8d4cb9750 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5693633740 10 client.16467 fetch_fsmap learned FSMap version 30 2024-01-31T05:35:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5693633740 1 -- 172.21.15.45:0/887438137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d8d4bd8a50 con 0x55d8d4cb9750 2024-01-31T05:35:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5671ffb640 1 -- 172.21.15.45:0/887438137 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5684004a10 con 0x55d8d4cb9750 2024-01-31T05:35:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5671ffb640 1 -- 172.21.15.45:0/887438137 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5680003be0 con 0x55d8d4cb9750 2024-01-31T05:35:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5693633740 10 client.16467 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:35:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5693633740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:35:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5693633740 20 client.16467 populate_metadata read hostname 'smithi045' 2024-01-31T05:35:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5693633740 1 --2- 172.21.15.45:0/887438137 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d8d4d20d80 0x55d8d4d231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:35:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5693633740 4 client.16467 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:35:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f5693633740 1 -- 172.21.15.45:0/887438137 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d8d4917600 con 0x55d8d4d20d80 2024-01-31T05:35:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.108+0000 7f568e534640 1 --2- 172.21.15.45:0/887438137 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d8d4d20d80 0x55d8d4d231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:35:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.109+0000 7f5671ffb640 10 client.16467 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.109+0000 7f568e534640 1 --2- 172.21.15.45:0/887438137 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d8d4d20d80 0x55d8d4d231a0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4839aa9362f619d2 server_cookie=c9d2bfb82f09f2b2 in_seq=0 out_seq=0 2024-01-31T05:35:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.121+0000 7f5671ffb640 1 -- 172.21.15.45:0/887438137 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d8d4917600 con 0x55d8d4d20d80 2024-01-31T05:35:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.121+0000 7f5671ffb640 10 client.16467 handle_command_reply: tid=0 2024-01-31T05:35:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.121+0000 7f5693633740 1 client.16467 shutdown 2024-01-31T05:35:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5693633740 1 -- 172.21.15.45:0/887438137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 msgr2=0x55d8d4cbea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5693633740 1 --2- 172.21.15.45:0/887438137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cbea20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5680032770 tx=0x7f5680032f70 comp rx=0 tx=0).stop 2024-01-31T05:35:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5693633740 1 -- 172.21.15.45:0/887438137 shutdown_connections 2024-01-31T05:35:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5693633740 1 --2- 172.21.15.45:0/887438137 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d8d4d20d80 0x55d8d4d231a0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5693633740 1 --2- 172.21.15.45:0/887438137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8d4cb9750 0x55d8d4cbea20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5693633740 1 --2- 172.21.15.45:0/887438137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d4cb0980 0x55d8d4cc0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5671ffb640 0 client.16467 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:35:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5693633740 1 --2- 172.21.15.45:0/887438137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8d4cb0060 0x55d8d4cc03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5693633740 1 -- 172.21.15.45:0/887438137 >> 172.21.15.45:0/887438137 conn(0x55d8d4caa980 msgr2=0x55d8d4cb19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5693633740 1 -- 172.21.15.45:0/887438137 shutdown_connections 2024-01-31T05:35:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5693633740 1 -- 172.21.15.45:0/887438137 wait complete. 2024-01-31T05:35:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5693633740 20 client.16467 trim_cache size 0 max 16384 2024-01-31T05:35:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:35:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:35:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:35:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:35:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5690798640 1 -- 172.21.15.45:0/741542311 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f56740a1520 msgr2=0x7f56740a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5690798640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f56740a1520 0x7f56740a39e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5680005fd0 tx=0x7f5680009500 comp rx=0 tx=0).stop 2024-01-31T05:35:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5690798640 1 -- 172.21.15.45:0/741542311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688068d10 msgr2=0x7f5688100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:35:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5690798640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688068d10 0x7f5688100450 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f567800d950 tx=0x7f567800de10 comp rx=0 tx=0).stop 2024-01-31T05:35:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5690798640 1 -- 172.21.15.45:0/741542311 shutdown_connections 2024-01-31T05:35:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5690798640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f56740a1520 0x7f56740a39e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5690798640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688068d10 0x7f5688100450 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5690798640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5688106e00 0x7f5688100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5690798640 1 --2- 172.21.15.45:0/741542311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5688068370 0x7f56880fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:35:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.122+0000 7f5690798640 1 -- 172.21.15.45:0/741542311 >> 172.21.15.45:0/741542311 conn(0x7f5688076a40 msgr2=0x7f56880feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:35:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.123+0000 7f5690798640 1 -- 172.21.15.45:0/741542311 shutdown_connections 2024-01-31T05:35:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:35:50.123+0000 7f5690798640 1 -- 172.21.15.45:0/741542311 wait complete. 2024-01-31T05:35:50.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:49 smithi045 ceph-mon[31585]: pgmap v417: 97 pgs: 97 active+clean; 605 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 161 op/s 2024-01-31T05:35:50.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1385643 10.31 MB/sec execute 590 sec latency 16.464 ms 2024-01-31T05:35:51.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1387134 10.30 MB/sec execute 591 sec latency 17.371 ms 2024-01-31T05:35:51.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:51 smithi156 ceph-mon[35720]: pgmap v418: 97 pgs: 97 active+clean; 605 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 161 op/s 2024-01-31T05:35:52.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:51 smithi186 ceph-mon[35659]: pgmap v418: 97 pgs: 97 active+clean; 605 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 161 op/s 2024-01-31T05:35:52.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:51 smithi045 ceph-mon[31585]: pgmap v418: 97 pgs: 97 active+clean; 605 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 161 op/s 2024-01-31T05:35:52.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1389033 10.30 MB/sec execute 592 sec latency 279.144 ms 2024-01-31T05:35:53.804 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1390479 10.30 MB/sec execute 593 sec latency 54.169 ms 2024-01-31T05:35:53.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:53 smithi156 ceph-mon[35720]: pgmap v419: 97 pgs: 97 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 246 op/s 2024-01-31T05:35:54.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:53 smithi186 ceph-mon[35659]: pgmap v419: 97 pgs: 97 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 246 op/s 2024-01-31T05:35:54.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:53 smithi045 ceph-mon[31585]: pgmap v419: 97 pgs: 97 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 246 op/s 2024-01-31T05:35:54.805 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1392879 10.31 MB/sec execute 594 sec latency 18.833 ms 2024-01-31T05:35:55.805 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1394295 10.30 MB/sec execute 595 sec latency 19.924 ms 2024-01-31T05:35:55.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:55 smithi156 ceph-mon[35720]: pgmap v420: 97 pgs: 97 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 163 op/s 2024-01-31T05:35:56.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:55 smithi186 ceph-mon[35659]: pgmap v420: 97 pgs: 97 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 163 op/s 2024-01-31T05:35:56.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:55 smithi045 ceph-mon[31585]: pgmap v420: 97 pgs: 97 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 163 op/s 2024-01-31T05:35:56.805 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1396574 10.31 MB/sec execute 596 sec latency 17.313 ms 2024-01-31T05:35:57.805 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1397886 10.30 MB/sec execute 597 sec latency 182.475 ms 2024-01-31T05:35:57.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:57 smithi156 ceph-mon[35720]: pgmap v421: 97 pgs: 97 active+clean; 625 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 238 op/s 2024-01-31T05:35:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:57 smithi186 ceph-mon[35659]: pgmap v421: 97 pgs: 97 active+clean; 625 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 238 op/s 2024-01-31T05:35:58.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:57 smithi045 ceph-mon[31585]: pgmap v421: 97 pgs: 97 active+clean; 625 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 238 op/s 2024-01-31T05:35:58.805 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1400049 10.30 MB/sec execute 598 sec latency 221.701 ms 2024-01-31T05:35:59.805 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1401599 10.30 MB/sec execute 599 sec latency 63.983 ms 2024-01-31T05:35:59.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:35:59 smithi156 ceph-mon[35720]: pgmap v422: 97 pgs: 97 active+clean; 625 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-01-31T05:36:00.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:35:59 smithi186 ceph-mon[35659]: pgmap v422: 97 pgs: 97 active+clean; 625 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-01-31T05:36:00.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:35:59 smithi045 ceph-mon[31585]: pgmap v422: 97 pgs: 97 active+clean; 625 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-01-31T05:36:00.805 INFO:tasks.workunit.client.0.smithi045.stdout: 1 cleanup 600 sec 2024-01-31T05:36:01.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:00 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:36:01.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:00 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:36:01.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:00 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:36:01.539 INFO:tasks.workunit.client.0.smithi045.stdout: 0 cleanup 601 sec 2024-01-31T05:36:01.540 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-01-31T05:36:01.540 INFO:tasks.workunit.client.0.smithi045.stdout: Operation Count AvgLat MaxLat 2024-01-31T05:36:01.540 INFO:tasks.workunit.client.0.smithi045.stdout: ---------------------------------------- 2024-01-31T05:36:01.540 INFO:tasks.workunit.client.0.smithi045.stdout: NTCreateX 196220 1.146 1254.429 2024-01-31T05:36:01.540 INFO:tasks.workunit.client.0.smithi045.stdout: Close 144178 0.003 0.130 2024-01-31T05:36:01.540 INFO:tasks.workunit.client.0.smithi045.stdout: Rename 8305 5.184 131.189 2024-01-31T05:36:01.540 INFO:tasks.workunit.client.0.smithi045.stdout: Unlink 39589 1.789 281.398 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: Deltree 6 91.578 125.539 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: Mkdir 3 0.678 1.061 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: Qpathinfo 177863 0.962 310.649 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: Qfileinfo 31203 0.002 0.028 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: Qfsinfo 32581 0.214 19.103 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: Sfileinfo 15981 0.020 1.273 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: Find 68749 0.409 15148.039 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: WriteX 97999 0.043 76.847 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: ReadX 307417 0.017 26.015 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: LockX 638 1.204 88.386 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: UnlockX 638 0.782 4.664 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: Flush 13743 2.983 288.043 2024-01-31T05:36:01.541 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-01-31T05:36:01.542 INFO:tasks.workunit.client.0.smithi045.stdout:Throughput 10.2951 MB/sec 1 clients 1 procs max_latency=15148.050 ms 2024-01-31T05:36:01.542 INFO:tasks.workunit.client.0.smithi045.stderr:+ dbench 10 2024-01-31T05:36:01.542 INFO:tasks.workunit.client.0.smithi045.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-01-31T05:36:01.542 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-01-31T05:36:01.543 INFO:tasks.workunit.client.0.smithi045.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-01-31T05:36:01.543 INFO:tasks.workunit.client.0.smithi045.stdout:8 of 10 processes prepared for launch 0 sec 2024-01-31T05:36:01.641 INFO:tasks.workunit.client.0.smithi045.stdout:10 of 10 processes prepared for launch 0 sec 2024-01-31T05:36:01.641 INFO:tasks.workunit.client.0.smithi045.stdout:releasing clients 2024-01-31T05:36:02.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:01 smithi156 ceph-mon[35720]: pgmap v423: 97 pgs: 97 active+clean; 625 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-01-31T05:36:02.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:01 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:36:02.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:01 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:36:02.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:01 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:36:02.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:01 smithi186 ceph-mon[35659]: pgmap v423: 97 pgs: 97 active+clean; 625 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-01-31T05:36:02.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:01 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:36:02.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:01 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:36:02.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:01 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:36:02.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:01 smithi045 ceph-mon[31585]: pgmap v423: 97 pgs: 97 active+clean; 625 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-01-31T05:36:02.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:01 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:36:02.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:01 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:36:02.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:01 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:36:02.641 INFO:tasks.workunit.client.0.smithi045.stdout: 10 343 123.95 MB/sec warmup 1 sec latency 139.983 ms 2024-01-31T05:36:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.212+0000 7f48efd89640 1 -- 172.21.15.45:0/4205549245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e80ffbb0 msgr2=0x7f48e810db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.212+0000 7f48efd89640 1 --2- 172.21.15.45:0/4205549245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e80ffbb0 0x7f48e810db80 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009930 tx=0x7f48e002bc30 comp rx=0 tx=0).stop 2024-01-31T05:36:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.212+0000 7f48efd89640 1 -- 172.21.15.45:0/4205549245 shutdown_connections 2024-01-31T05:36:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.212+0000 7f48efd89640 1 --2- 172.21.15.45:0/4205549245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e80ffbb0 0x7f48e810db80 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.212+0000 7f48efd89640 1 --2- 172.21.15.45:0/4205549245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e80ff1f0 0x7f48e80ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.212+0000 7f48efd89640 1 --2- 172.21.15.45:0/4205549245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48e8105cd0 0x7f48e81060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.212+0000 7f48efd89640 1 -- 172.21.15.45:0/4205549245 >> 172.21.15.45:0/4205549245 conn(0x7f48e8076180 msgr2=0x7f48e80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.212+0000 7f48efd89640 1 -- 172.21.15.45:0/4205549245 shutdown_connections 2024-01-31T05:36:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.212+0000 7f48efd89640 1 -- 172.21.15.45:0/4205549245 wait complete. 2024-01-31T05:36:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48efd89640 1 Processor -- start 2024-01-31T05:36:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48efd89640 1 -- start start 2024-01-31T05:36:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48efd89640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48e80ff1f0 0x7f48e81003d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48efd89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e80ffbb0 0x7f48e8100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48efd89640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8105cd0 0x7f48e8100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48efd89640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e81a5570 con 0x7f48e80ffbb0 2024-01-31T05:36:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48efd89640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e81a56e0 con 0x7f48e8105cd0 2024-01-31T05:36:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48efd89640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e81a58a0 con 0x7f48e80ff1f0 2024-01-31T05:36:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48ed324640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e80ffbb0 0x7f48e8100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48edb25640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48e80ff1f0 0x7f48e81003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48edb25640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48e80ff1f0 0x7f48e81003d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:40722/0 (socket says 172.21.15.45:40722) 2024-01-31T05:36:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48edb25640 1 -- 172.21.15.45:0/1430819343 learned_addr learned my addr 172.21.15.45:0/1430819343 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48ee326640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8105cd0 0x7f48e8100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48ed324640 1 -- 172.21.15.45:0/1430819343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48e80ff1f0 msgr2=0x7f48e81003d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48ed324640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48e80ff1f0 0x7f48e81003d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48ed324640 1 -- 172.21.15.45:0/1430819343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8105cd0 msgr2=0x7f48e8100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48ed324640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8105cd0 0x7f48e8100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.213+0000 7f48ed324640 1 -- 172.21.15.45:0/1430819343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e0009560 con 0x7f48e80ffbb0 2024-01-31T05:36:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.214+0000 7f48ed324640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e80ffbb0 0x7f48e8100910 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f48d8005fd0 tx=0x7f48d800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.214+0000 7f48deffd640 1 -- 172.21.15.45:0/1430819343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8009c30 con 0x7f48e80ffbb0 2024-01-31T05:36:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.214+0000 7f48efd89640 1 -- 172.21.15.45:0/1430819343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e81a5b80 con 0x7f48e80ffbb0 2024-01-31T05:36:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.214+0000 7f48deffd640 1 -- 172.21.15.45:0/1430819343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d80044f0 con 0x7f48e80ffbb0 2024-01-31T05:36:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.214+0000 7f48deffd640 1 -- 172.21.15.45:0/1430819343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8003db0 con 0x7f48e80ffbb0 2024-01-31T05:36:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.214+0000 7f48efd89640 1 -- 172.21.15.45:0/1430819343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e81a60e0 con 0x7f48e80ffbb0 2024-01-31T05:36:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.216+0000 7f48deffd640 1 -- 172.21.15.45:0/1430819343 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f48d8003f10 con 0x7f48e80ffbb0 2024-01-31T05:36:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48deffd640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f48d40a15a0 0x7f48d40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48f2c28740 1 Processor -- start 2024-01-31T05:36:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48edb25640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f48d40a15a0 0x7f48d40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48deffd640 1 -- 172.21.15.45:0/1430819343 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f48d80cc680 con 0x7f48e80ffbb0 2024-01-31T05:36:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48f2c28740 1 -- start start 2024-01-31T05:36:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48f2c28740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d0060 0x5627887d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48f2c28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887d9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48f2c28740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d9750 0x5627887dbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48f2c28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278873f040 con 0x5627887d0980 2024-01-31T05:36:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48f2c28740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278873c730 con 0x5627887d0060 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48f2c28740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562788738020 con 0x5627887d9750 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48ed324640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887d9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.217+0000 7f48edb25640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f48d40a15a0 0x7f48d40a3a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48e40049d0 tx=0x7f48e4009500 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48ed324640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887d9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53386/0 (socket says 172.21.15.45:53386) 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48ed324640 1 -- 172.21.15.45:0/530927415 learned_addr learned my addr 172.21.15.45:0/530927415 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48ee326640 1 --2- 172.21.15.45:0/530927415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d9750 0x5627887dbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48edb25640 1 --2- 172.21.15.45:0/530927415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d0060 0x5627887d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48ed324640 1 -- 172.21.15.45:0/530927415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d9750 msgr2=0x5627887dbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48ed324640 1 --2- 172.21.15.45:0/530927415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d9750 0x5627887dbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48ed324640 1 -- 172.21.15.45:0/530927415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d0060 msgr2=0x5627887d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48ed324640 1 --2- 172.21.15.45:0/530927415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d0060 0x5627887d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48ed324640 1 -- 172.21.15.45:0/530927415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56278872fe40 con 0x5627887d0980 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48ed324640 1 --2- 172.21.15.45:0/530927415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887d9210 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f48d80a1280 tx=0x7f48d80958e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7502aa7e3005c456 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48d3fff640 1 -- 172.21.15.45:0/530927415 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80994d0 con 0x5627887d0980 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48d3fff640 1 -- 172.21.15.45:0/530927415 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d809d570 con 0x5627887d0980 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.218+0000 7f48d3fff640 1 -- 172.21.15.45:0/530927415 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80c9ff0 con 0x5627887d0980 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.219+0000 7f48f2c28740 1 -- 172.21.15.45:0/530927415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 msgr2=0x5627887d9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.219+0000 7f48f2c28740 1 --2- 172.21.15.45:0/530927415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887d9210 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f48d80a1280 tx=0x7f48d80958e0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.219+0000 7f48f2c28740 1 -- 172.21.15.45:0/530927415 shutdown_connections 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.219+0000 7f48f2c28740 1 --2- 172.21.15.45:0/530927415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887d9210 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.219+0000 7f48f2c28740 1 --2- 172.21.15.45:0/530927415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d9750 0x5627887dbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.219+0000 7f48f2c28740 1 --2- 172.21.15.45:0/530927415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d0060 0x5627887d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.219+0000 7f48f2c28740 1 -- 172.21.15.45:0/530927415 >> 172.21.15.45:0/530927415 conn(0x5627887ca980 msgr2=0x5627887cad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.219+0000 7f48f2c28740 1 -- 172.21.15.45:0/530927415 shutdown_connections 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.219+0000 7f48f2c28740 1 -- 172.21.15.45:0/530927415 wait complete. 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.219+0000 7f48f2c28740 1 -- start start 2024-01-31T05:36:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.219+0000 7f48f2c28740 1 Processor -- start 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48f2c28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0060 0x5627887d6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48f2c28740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d0980 0x5627887d6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48f2c28740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d9750 0x5627887d71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48f2c28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627886f11b0 con 0x5627887d0060 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48f2c28740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627886f0e50 con 0x5627887d9750 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48f2c28740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627886b6090 con 0x5627887d0980 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48f2c28740 10 client.0 _ll_register_callbacks cb 0x5627886dba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48edb25640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0060 0x5627887d6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48edb25640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0060 0x5627887d6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53402/0 (socket says 172.21.15.45:53402) 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48edb25640 1 -- 172.21.15.45:0/4213449841 learned_addr learned my addr 172.21.15.45:0/4213449841 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48ed324640 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d0980 0x5627887d6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48ee326640 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d9750 0x5627887d71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48edb25640 1 -- 172.21.15.45:0/4213449841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d0980 msgr2=0x5627887d6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48edb25640 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d0980 0x5627887d6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48edb25640 1 -- 172.21.15.45:0/4213449841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d9750 msgr2=0x5627887d71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48edb25640 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d9750 0x5627887d71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48edb25640 1 -- 172.21.15.45:0/4213449841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e81a60e0 con 0x5627887d0060 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48ed324640 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d0980 0x5627887d6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48edb25640 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0060 0x5627887d6740 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f48e400fe50 tx=0x7f48e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b584d0c173633 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48d17fa640 1 -- 172.21.15.45:0/4213449841 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4022070 con 0x5627887d0060 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48f2c28740 1 -- 172.21.15.45:0/4213449841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562788731010 con 0x5627887d0060 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.220+0000 7f48d17fa640 1 -- 172.21.15.45:0/4213449841 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f48e401db20 con 0x5627887d0060 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.221+0000 7f48d17fa640 1 -- 172.21.15.45:0/4213449841 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48e401dce0 con 0x5627887d0060 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.221+0000 7f48f2c28740 10 client.16479 fetch_fsmap learned FSMap version 30 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.221+0000 7f48f2c28740 1 -- 172.21.15.45:0/4213449841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5627887366a0 con 0x5627887d0060 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.221+0000 7f48d17fa640 1 -- 172.21.15.45:0/4213449841 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48e4033080 con 0x5627887d0060 2024-01-31T05:36:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.221+0000 7f48d17fa640 1 -- 172.21.15.45:0/4213449841 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f48e402d030 con 0x5627887d0060 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.221+0000 7f48f2c28740 10 client.16479 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.221+0000 7f48f2c28740 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.221+0000 7f48f2c28740 20 client.16479 populate_metadata read hostname 'smithi045' 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.221+0000 7f48f2c28740 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5627887e95f0 0x5627887eba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.221+0000 7f48f2c28740 4 client.16479 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.221+0000 7f48f2c28740 1 -- 172.21.15.45:0/4213449841 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562788730d30 con 0x5627887e95f0 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.221+0000 7f48ed324640 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5627887e95f0 0x5627887eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.222+0000 7f48ed324640 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5627887e95f0 0x5627887eba10 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b77df71c7ff5160 server_cookie=f5122ed4253bd2cc in_seq=0 out_seq=0 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.222+0000 7f48d17fa640 10 client.16479 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.222+0000 7f48d17fa640 1 -- 172.21.15.45:0/4213449841 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562788730d30 con 0x5627887e95f0 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.222+0000 7f48d17fa640 10 client.16479 handle_command_reply: tid=0 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.222+0000 7f48f2c28740 1 client.16479 shutdown 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.223+0000 7f48f2c28740 1 -- 172.21.15.45:0/4213449841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0060 msgr2=0x5627887d6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.223+0000 7f48f2c28740 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0060 0x5627887d6740 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f48e400fe50 tx=0x7f48e4004ab0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.223+0000 7f48f2c28740 1 -- 172.21.15.45:0/4213449841 shutdown_connections 2024-01-31T05:36:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.223+0000 7f48f2c28740 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5627887e95f0 0x5627887eba10 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.223+0000 7f48f2c28740 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d9750 0x5627887d71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.223+0000 7f48f2c28740 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d0980 0x5627887d6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.223+0000 7f48f2c28740 1 --2- 172.21.15.45:0/4213449841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0060 0x5627887d6740 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.223+0000 7f48d17fa640 0 client.16479 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.223+0000 7f48f2c28740 1 -- 172.21.15.45:0/4213449841 >> 172.21.15.45:0/4213449841 conn(0x5627887ca980 msgr2=0x5627887cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.223+0000 7f48f2c28740 1 -- 172.21.15.45:0/4213449841 shutdown_connections 2024-01-31T05:36:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.223+0000 7f48f2c28740 1 -- 172.21.15.45:0/4213449841 wait complete. 2024-01-31T05:36:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.223+0000 7f48f2c28740 20 client.16479 trim_cache size 0 max 16384 2024-01-31T05:36:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.236+0000 7f48f2c28740 1 Processor -- start 2024-01-31T05:36:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.236+0000 7f48f2c28740 1 -- start start 2024-01-31T05:36:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48f2c28740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d0060 0x5627887d8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48f2c28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887d6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48f2c28740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d9750 0x5627887d5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48f2c28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562788730d30 con 0x5627887d0980 2024-01-31T05:36:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48f2c28740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627887c4ee0 con 0x5627887d9750 2024-01-31T05:36:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48f2c28740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627887ca7f0 con 0x5627887d0060 2024-01-31T05:36:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48ed324640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887d6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48edb25640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d0060 0x5627887d8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48ed324640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887d6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53416/0 (socket says 172.21.15.45:53416) 2024-01-31T05:36:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48ed324640 1 -- 172.21.15.45:0/751099552 learned_addr learned my addr 172.21.15.45:0/751099552 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48ee326640 1 --2- 172.21.15.45:0/751099552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d9750 0x5627887d5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48ed324640 1 -- 172.21.15.45:0/751099552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d0060 msgr2=0x5627887d8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48ed324640 1 --2- 172.21.15.45:0/751099552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d0060 0x5627887d8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48ed324640 1 -- 172.21.15.45:0/751099552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d9750 msgr2=0x5627887d5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48ed324640 1 --2- 172.21.15.45:0/751099552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d9750 0x5627887d5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48ed324640 1 -- 172.21.15.45:0/751099552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627886d3030 con 0x5627887d0980 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48ed324640 1 --2- 172.21.15.45:0/751099552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887d6fe0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f48d80959d0 tx=0x7f48d8005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caae79705e970c69 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48d37fe640 1 -- 172.21.15.45:0/751099552 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80a1830 con 0x5627887d0980 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48d37fe640 1 -- 172.21.15.45:0/751099552 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d80994d0 con 0x5627887d0980 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.237+0000 7f48d37fe640 1 -- 172.21.15.45:0/751099552 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80c9ff0 con 0x5627887d0980 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.238+0000 7f48f2c28740 1 -- 172.21.15.45:0/751099552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 msgr2=0x5627887d6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.238+0000 7f48f2c28740 1 --2- 172.21.15.45:0/751099552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887d6fe0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f48d80959d0 tx=0x7f48d8005ab0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.238+0000 7f48f2c28740 1 -- 172.21.15.45:0/751099552 shutdown_connections 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.238+0000 7f48f2c28740 1 --2- 172.21.15.45:0/751099552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887d6fe0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.238+0000 7f48f2c28740 1 --2- 172.21.15.45:0/751099552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d9750 0x5627887d5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.238+0000 7f48f2c28740 1 --2- 172.21.15.45:0/751099552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d0060 0x5627887d8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.238+0000 7f48f2c28740 1 -- 172.21.15.45:0/751099552 >> 172.21.15.45:0/751099552 conn(0x5627887ca980 msgr2=0x5627887d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.238+0000 7f48f2c28740 1 -- 172.21.15.45:0/751099552 shutdown_connections 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.238+0000 7f48f2c28740 1 -- 172.21.15.45:0/751099552 wait complete. 2024-01-31T05:36:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48f2c28740 1 -- start start 2024-01-31T05:36:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48f2c28740 1 Processor -- start 2024-01-31T05:36:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48f2c28740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d0060 0x5627887de960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48f2c28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887deea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48f2c28740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d9750 0x562788838bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48f2c28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562788839100 con 0x5627887d0980 2024-01-31T05:36:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48f2c28740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562788839240 con 0x5627887d0060 2024-01-31T05:36:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48f2c28740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562788839400 con 0x5627887d9750 2024-01-31T05:36:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48f2c28740 10 client.0 _ll_register_callbacks cb 0x7f48d8095150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:36:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48ed324640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887deea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48ed324640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887deea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53428/0 (socket says 172.21.15.45:53428) 2024-01-31T05:36:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48ed324640 1 -- 172.21.15.45:0/882681508 learned_addr learned my addr 172.21.15.45:0/882681508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48edb25640 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d0060 0x5627887de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48ee326640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d9750 0x562788838bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48ed324640 1 -- 172.21.15.45:0/882681508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d9750 msgr2=0x562788838bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48ed324640 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d9750 0x562788838bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48ed324640 1 -- 172.21.15.45:0/882681508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d0060 msgr2=0x5627887de960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48ed324640 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d0060 0x5627887de960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48ed324640 1 -- 172.21.15.45:0/882681508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56278872fe40 con 0x5627887d0980 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.239+0000 7f48edb25640 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d0060 0x5627887de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48ed324640 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887deea0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009610 tx=0x7f48d8005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adef7f93d5865c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48d17fa640 1 -- 172.21.15.45:0/882681508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80a1830 con 0x5627887d0980 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48d17fa640 1 -- 172.21.15.45:0/882681508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f48d8095be0 con 0x5627887d0980 2024-01-31T05:36:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48f2c28740 1 -- 172.21.15.45:0/882681508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48d809d570 con 0x5627887d0980 2024-01-31T05:36:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48d17fa640 1 -- 172.21.15.45:0/882681508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d80c9ff0 con 0x5627887d0980 2024-01-31T05:36:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48f2c28740 10 client.16485 fetch_fsmap learned FSMap version 30 2024-01-31T05:36:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48f2c28740 1 -- 172.21.15.45:0/882681508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5627886f8a50 con 0x5627887d0980 2024-01-31T05:36:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48d17fa640 1 -- 172.21.15.45:0/882681508 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d809d570 con 0x5627887d0980 2024-01-31T05:36:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48d17fa640 1 -- 172.21.15.45:0/882681508 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f48d80b7420 con 0x5627887d0980 2024-01-31T05:36:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48f2c28740 10 client.16485 fetch_fsmap finished waiting for FSMap version 30 2024-01-31T05:36:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48f2c28740 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:36:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48f2c28740 20 client.16485 populate_metadata read hostname 'smithi045' 2024-01-31T05:36:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48f2c28740 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562788840820 0x562788842c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48f2c28740 4 client.16485 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:36:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.240+0000 7f48f2c28740 1 -- 172.21.15.45:0/882681508 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562788437600 con 0x562788840820 2024-01-31T05:36:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.241+0000 7f48edb25640 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562788840820 0x562788842c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.241+0000 7f48d17fa640 10 client.16485 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.241+0000 7f48edb25640 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562788840820 0x562788842c40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40c2b6a3dbbfbec9 server_cookie=b2eb9e20ca095d42 in_seq=0 out_seq=0 2024-01-31T05:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.257+0000 7f48d17fa640 1 -- 172.21.15.45:0/882681508 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562788437600 con 0x562788840820 2024-01-31T05:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.257+0000 7f48d17fa640 10 client.16485 handle_command_reply: tid=0 2024-01-31T05:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.257+0000 7f48f2c28740 1 client.16485 shutdown 2024-01-31T05:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48f2c28740 1 -- 172.21.15.45:0/882681508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 msgr2=0x5627887deea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48f2c28740 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887deea0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009610 tx=0x7f48d8005d80 comp rx=0 tx=0).stop 2024-01-31T05:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48f2c28740 1 -- 172.21.15.45:0/882681508 shutdown_connections 2024-01-31T05:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48f2c28740 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x562788840820 0x562788842c40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48f2c28740 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627887d0980 0x5627887deea0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48f2c28740 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5627887d9750 0x562788838bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48d17fa640 0 client.16485 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48f2c28740 1 --2- 172.21.15.45:0/882681508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627887d0060 0x5627887de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48f2c28740 1 -- 172.21.15.45:0/882681508 >> 172.21.15.45:0/882681508 conn(0x5627887ca980 msgr2=0x5627887d4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48f2c28740 1 -- 172.21.15.45:0/882681508 shutdown_connections 2024-01-31T05:36:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48f2c28740 1 -- 172.21.15.45:0/882681508 wait complete. 2024-01-31T05:36:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48f2c28740 20 client.16485 trim_cache size 0 max 16384 2024-01-31T05:36:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:36:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:36:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:36:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:36:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48efd89640 1 -- 172.21.15.45:0/1430819343 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f48d40a15a0 msgr2=0x7f48d40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48efd89640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f48d40a15a0 0x7f48d40a3a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48e40049d0 tx=0x7f48e4009500 comp rx=0 tx=0).stop 2024-01-31T05:36:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48efd89640 1 -- 172.21.15.45:0/1430819343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e80ffbb0 msgr2=0x7f48e8100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48efd89640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e80ffbb0 0x7f48e8100910 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f48d8005fd0 tx=0x7f48d800dde0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48efd89640 1 -- 172.21.15.45:0/1430819343 shutdown_connections 2024-01-31T05:36:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48efd89640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f48d40a15a0 0x7f48d40a3a60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48efd89640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e80ffbb0 0x7f48e8100910 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48efd89640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8105cd0 0x7f48e8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48efd89640 1 --2- 172.21.15.45:0/1430819343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48e80ff1f0 0x7f48e81003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.258+0000 7f48efd89640 1 -- 172.21.15.45:0/1430819343 >> 172.21.15.45:0/1430819343 conn(0x7f48e8076180 msgr2=0x7f48e8103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.259+0000 7f48efd89640 1 -- 172.21.15.45:0/1430819343 shutdown_connections 2024-01-31T05:36:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:03.259+0000 7f48efd89640 1 -- 172.21.15.45:0/1430819343 wait complete. 2024-01-31T05:36:03.641 INFO:tasks.workunit.client.0.smithi045.stdout: 10 642 100.45 MB/sec warmup 2 sec latency 310.963 ms 2024-01-31T05:36:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:03 smithi156 ceph-mon[35720]: pgmap v424: 97 pgs: 97 active+clean; 632 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.7 MiB/s wr, 243 op/s 2024-01-31T05:36:04.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:03 smithi186 ceph-mon[35659]: pgmap v424: 97 pgs: 97 active+clean; 632 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.7 MiB/s wr, 243 op/s 2024-01-31T05:36:04.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:03 smithi045 ceph-mon[31585]: pgmap v424: 97 pgs: 97 active+clean; 632 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.7 MiB/s wr, 243 op/s 2024-01-31T05:36:04.641 INFO:tasks.workunit.client.0.smithi045.stdout: 10 957 71.09 MB/sec warmup 3 sec latency 232.212 ms 2024-01-31T05:36:05.641 INFO:tasks.workunit.client.0.smithi045.stdout: 10 1444 62.42 MB/sec warmup 4 sec latency 202.348 ms 2024-01-31T05:36:06.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:06 smithi186 ceph-mon[35659]: osdmap e110: 12 total, 12 up, 12 in 2024-01-31T05:36:06.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:06 smithi186 ceph-mon[35659]: pgmap v426: 97 pgs: 97 active+clean; 632 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 190 op/s 2024-01-31T05:36:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:06 smithi045 ceph-mon[31585]: osdmap e110: 12 total, 12 up, 12 in 2024-01-31T05:36:06.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:06 smithi045 ceph-mon[31585]: pgmap v426: 97 pgs: 97 active+clean; 632 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 190 op/s 2024-01-31T05:36:06.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:06 smithi156 ceph-mon[35720]: osdmap e110: 12 total, 12 up, 12 in 2024-01-31T05:36:06.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:06 smithi156 ceph-mon[35720]: pgmap v426: 97 pgs: 97 active+clean; 632 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 190 op/s 2024-01-31T05:36:06.641 INFO:tasks.workunit.client.0.smithi045.stdout: 10 2193 53.60 MB/sec warmup 5 sec latency 183.157 ms 2024-01-31T05:36:07.641 INFO:tasks.workunit.client.0.smithi045.stdout: 10 2664 52.68 MB/sec warmup 6 sec latency 271.753 ms 2024-01-31T05:36:08.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:08 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:36:08.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:08 smithi186 ceph-mon[35659]: pgmap v427: 97 pgs: 97 active+clean; 567 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 244 op/s 2024-01-31T05:36:08.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:08 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:36:08.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:08 smithi045 ceph-mon[31585]: pgmap v427: 97 pgs: 97 active+clean; 567 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 244 op/s 2024-01-31T05:36:08.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:08 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:36:08.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:08 smithi156 ceph-mon[35720]: pgmap v427: 97 pgs: 97 active+clean; 567 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 244 op/s 2024-01-31T05:36:08.641 INFO:tasks.workunit.client.0.smithi045.stdout: 10 3244 50.48 MB/sec warmup 7 sec latency 158.481 ms 2024-01-31T05:36:09.641 INFO:tasks.workunit.client.0.smithi045.stdout: 10 3664 48.09 MB/sec warmup 8 sec latency 164.006 ms 2024-01-31T05:36:09.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:09 smithi156 ceph-mon[35720]: pgmap v428: 97 pgs: 97 active+clean; 567 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 244 op/s 2024-01-31T05:36:10.094 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:09 smithi186 ceph-mon[35659]: pgmap v428: 97 pgs: 97 active+clean; 567 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 244 op/s 2024-01-31T05:36:10.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:09 smithi045 ceph-mon[31585]: pgmap v428: 97 pgs: 97 active+clean; 567 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 244 op/s 2024-01-31T05:36:10.641 INFO:tasks.workunit.client.0.smithi045.stdout: 10 4044 44.31 MB/sec warmup 9 sec latency 214.565 ms 2024-01-31T05:36:11.641 INFO:tasks.workunit.client.0.smithi045.stdout: 10 4339 41.21 MB/sec warmup 10 sec latency 213.899 ms 2024-01-31T05:36:11.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:11 smithi156 ceph-mon[35720]: pgmap v429: 97 pgs: 97 active+clean; 567 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 244 op/s 2024-01-31T05:36:12.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:11 smithi186 ceph-mon[35659]: pgmap v429: 97 pgs: 97 active+clean; 567 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 244 op/s 2024-01-31T05:36:12.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:11 smithi045 ceph-mon[31585]: pgmap v429: 97 pgs: 97 active+clean; 567 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 244 op/s 2024-01-31T05:36:12.641 INFO:tasks.workunit.client.0.smithi045.stdout: 10 4719 38.94 MB/sec warmup 11 sec latency 218.478 ms 2024-01-31T05:36:13.641 INFO:tasks.workunit.client.0.smithi045.stdout: 10 5117 36.99 MB/sec warmup 12 sec latency 272.262 ms 2024-01-31T05:36:13.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:13 smithi156 ceph-mon[35720]: osdmap e111: 12 total, 12 up, 12 in 2024-01-31T05:36:13.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:13 smithi156 ceph-mon[35720]: pgmap v431: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 354 op/s 2024-01-31T05:36:14.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:13 smithi186 ceph-mon[35659]: osdmap e111: 12 total, 12 up, 12 in 2024-01-31T05:36:14.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:13 smithi186 ceph-mon[35659]: pgmap v431: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 354 op/s 2024-01-31T05:36:14.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:13 smithi045 ceph-mon[31585]: osdmap e111: 12 total, 12 up, 12 in 2024-01-31T05:36:14.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:13 smithi045 ceph-mon[31585]: pgmap v431: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 354 op/s 2024-01-31T05:36:14.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 5684 36.57 MB/sec warmup 13 sec latency 252.993 ms 2024-01-31T05:36:15.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 6413 35.59 MB/sec warmup 14 sec latency 144.276 ms 2024-01-31T05:36:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:15 smithi156 ceph-mon[35720]: pgmap v432: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 303 op/s 2024-01-31T05:36:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:15 smithi186 ceph-mon[35659]: pgmap v432: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 303 op/s 2024-01-31T05:36:16.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:15 smithi045 ceph-mon[31585]: pgmap v432: 97 pgs: 97 active+clean; 583 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 303 op/s 2024-01-31T05:36:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 -- 172.21.15.45:0/3162795783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba8100800 msgr2=0x7feba810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 --2- 172.21.15.45:0/3162795783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba8100800 0x7feba810ca20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7feba0009ca0 tx=0x7feba002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 -- 172.21.15.45:0/3162795783 shutdown_connections 2024-01-31T05:36:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 --2- 172.21.15.45:0/3162795783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba8100800 0x7feba810ca20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 --2- 172.21.15.45:0/3162795783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feba80ffe40 0x7feba81002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 --2- 172.21.15.45:0/3162795783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba8104b90 0x7feba8104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 -- 172.21.15.45:0/3162795783 >> 172.21.15.45:0/3162795783 conn(0x7feba8076a40 msgr2=0x7feba8076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 -- 172.21.15.45:0/3162795783 shutdown_connections 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 -- 172.21.15.45:0/3162795783 wait complete. 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 Processor -- start 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 -- start start 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba80ffe40 0x7feba81971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba8100800 0x7feba8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feba8104b90 0x7feba819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba819c080 con 0x7feba80ffe40 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba819c1f0 con 0x7feba8104b90 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.364+0000 7febaf9a7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba819c3b0 con 0x7feba8100800 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febad743640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba80ffe40 0x7feba81971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febad743640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba80ffe40 0x7feba81971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41894/0 (socket says 172.21.15.45:41894) 2024-01-31T05:36:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febad743640 1 -- 172.21.15.45:0/3900388834 learned_addr learned my addr 172.21.15.45:0/3900388834 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febacf42640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba8100800 0x7feba8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febadf44640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feba8104b90 0x7feba819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febad743640 1 -- 172.21.15.45:0/3900388834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba8100800 msgr2=0x7feba8197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febad743640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba8100800 0x7feba8197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febad743640 1 -- 172.21.15.45:0/3900388834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feba8104b90 msgr2=0x7feba819ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febad743640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feba8104b90 0x7feba819ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febad743640 1 -- 172.21.15.45:0/3900388834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c009490 con 0x7feba80ffe40 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febad743640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba80ffe40 0x7feba81971c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7feba400e9f0 tx=0x7feba400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7feb9a7fc640 1 -- 172.21.15.45:0/3900388834 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4009720 con 0x7feba80ffe40 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7feb9a7fc640 1 -- 172.21.15.45:0/3900388834 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feba400cd80 con 0x7feba80ffe40 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febaf9a7640 1 -- 172.21.15.45:0/3900388834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba0009900 con 0x7feba80ffe40 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7feb9a7fc640 1 -- 172.21.15.45:0/3900388834 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4010430 con 0x7feba80ffe40 2024-01-31T05:36:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.365+0000 7febaf9a7640 1 -- 172.21.15.45:0/3900388834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba81a3d30 con 0x7feba80ffe40 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.368+0000 7feb9a7fc640 1 -- 172.21.15.45:0/3900388834 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7feba4005020 con 0x7feba80ffe40 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.368+0000 7febb2764740 1 Processor -- start 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.368+0000 7febb2764740 1 -- start start 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.368+0000 7febb2764740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 0x5599bdadd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.368+0000 7febb2764740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd980 0x5599bdae6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.368+0000 7febb2764740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdae8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.368+0000 7febb2764740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bda4c040 con 0x5599bdadd060 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.368+0000 7febb2764740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bda49730 con 0x5599bdae6750 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.368+0000 7febb2764740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bda45020 con 0x5599bdadd980 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.368+0000 7feb9a7fc640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7feb940a14f0 0x7feb940a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febadf44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 0x5599bdadd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febad743640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd980 0x5599bdae6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febadf44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 0x5599bdadd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41908/0 (socket says 172.21.15.45:41908) 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febadf44640 1 -- 172.21.15.45:0/3474888595 learned_addr learned my addr 172.21.15.45:0/3474888595 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febacf42640 1 --2- 172.21.15.45:0/3474888595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdae8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febadf44640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7feb940a14f0 0x7feb940a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7feb9a7fc640 1 -- 172.21.15.45:0/3900388834 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7feba4014070 con 0x7feba80ffe40 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febadf44640 1 -- 172.21.15.45:0/3474888595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd980 msgr2=0x5599bdae6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febadf44640 1 --2- 172.21.15.45:0/3474888595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd980 0x5599bdae6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febadf44640 1 -- 172.21.15.45:0/3474888595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 msgr2=0x5599bdae8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febadf44640 1 --2- 172.21.15.45:0/3474888595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdae8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febadf44640 1 -- 172.21.15.45:0/3474888595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599bda3ce40 con 0x5599bdadd060 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febadf44640 1 --2- 172.21.15.45:0/3474888595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 0x5599bdadd440 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7feba0037040 tx=0x7feba0034790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b28ec224acef2c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7febadf44640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7feb940a14f0 0x7feb940a39b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feba0009ca0 tx=0x7feba0049000 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7feb937fe640 1 -- 172.21.15.45:0/3474888595 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0048070 con 0x5599bdadd060 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7feb937fe640 1 -- 172.21.15.45:0/3474888595 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feba0032a90 con 0x5599bdadd060 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.369+0000 7feb937fe640 1 -- 172.21.15.45:0/3474888595 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0003ca0 con 0x5599bdadd060 2024-01-31T05:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.370+0000 7febb2764740 1 -- 172.21.15.45:0/3474888595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 msgr2=0x5599bdadd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.370+0000 7febb2764740 1 --2- 172.21.15.45:0/3474888595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 0x5599bdadd440 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7feba0037040 tx=0x7feba0034790 comp rx=0 tx=0).stop 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.370+0000 7febb2764740 1 -- 172.21.15.45:0/3474888595 shutdown_connections 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.370+0000 7febb2764740 1 --2- 172.21.15.45:0/3474888595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdae8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.370+0000 7febb2764740 1 --2- 172.21.15.45:0/3474888595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd980 0x5599bdae6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.370+0000 7febb2764740 1 --2- 172.21.15.45:0/3474888595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 0x5599bdadd440 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.370+0000 7febb2764740 1 -- 172.21.15.45:0/3474888595 >> 172.21.15.45:0/3474888595 conn(0x5599bdad7980 msgr2=0x5599bdad7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.370+0000 7febb2764740 1 -- 172.21.15.45:0/3474888595 shutdown_connections 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.370+0000 7febb2764740 1 -- 172.21.15.45:0/3474888595 wait complete. 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.370+0000 7febb2764740 1 -- start start 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febb2764740 1 Processor -- start 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febb2764740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd060 0x5599bdae3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febb2764740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd980 0x5599bdae3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febb2764740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdae4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febb2764740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bd9fe1b0 con 0x5599bdadd980 2024-01-31T05:36:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febb2764740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bd9fde50 con 0x5599bdae6750 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febb2764740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bd9c3090 con 0x5599bdadd060 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febb2764740 10 client.0 _ll_register_callbacks cb 0x5599bd9e8a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febacf42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd980 0x5599bdae3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febad743640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd060 0x5599bdae3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febacf42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd980 0x5599bdae3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41918/0 (socket says 172.21.15.45:41918) 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febacf42640 1 -- 172.21.15.45:0/2325169655 learned_addr learned my addr 172.21.15.45:0/2325169655 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febadf44640 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdae4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febacf42640 1 -- 172.21.15.45:0/2325169655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd060 msgr2=0x5599bdae3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febacf42640 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd060 0x5599bdae3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febacf42640 1 -- 172.21.15.45:0/2325169655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 msgr2=0x5599bdae4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febacf42640 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdae4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febacf42640 1 -- 172.21.15.45:0/2325169655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba80686f0 con 0x5599bdadd980 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febad743640 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd060 0x5599bdae3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.371+0000 7febadf44640 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdae4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:36:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7febacf42640 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd980 0x5599bdae3c50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7feb9c012820 tx=0x7feb9c012ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8226f3890de3d1a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7feb90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7feb90ff9640 1 -- 172.21.15.45:0/2325169655 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c017070 con 0x5599bdadd980 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7feb90ff9640 1 -- 172.21.15.45:0/2325169655 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7feb9c00f8c0 con 0x5599bdadd980 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7febb2764740 1 -- 172.21.15.45:0/2325169655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599bda3e010 con 0x5599bdadd980 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7feb90ff9640 1 -- 172.21.15.45:0/2325169655 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb9c00fa80 con 0x5599bdadd980 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7febb2764740 10 client.16497 fetch_fsmap learned FSMap version 31 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7febb2764740 1 -- 172.21.15.45:0/2325169655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5599bda436a0 con 0x5599bdadd980 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7feb90ff9640 1 -- 172.21.15.45:0/2325169655 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb9c028080 con 0x5599bdadd980 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7feb90ff9640 1 -- 172.21.15.45:0/2325169655 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feb9c022030 con 0x5599bdadd980 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7febb2764740 10 client.16497 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7febb2764740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7febb2764740 20 client.16497 populate_metadata read hostname 'smithi045' 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7febb2764740 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5599bdaf6600 0x5599bdaf8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7febb2764740 4 client.16497 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.372+0000 7febb2764740 1 -- 172.21.15.45:0/2325169655 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599bda3dd30 con 0x5599bdaf6600 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.373+0000 7febad743640 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5599bdaf6600 0x5599bdaf8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.373+0000 7febad743640 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5599bdaf6600 0x5599bdaf8a40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d556a52501aabbe server_cookie=eb9e5e6aa61b8e1b in_seq=0 out_seq=0 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.373+0000 7feb90ff9640 10 client.16497 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7feb90ff9640 1 -- 172.21.15.45:0/2325169655 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599bda3dd30 con 0x5599bdaf6600 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7feb90ff9640 10 client.16497 handle_command_reply: tid=0 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7febb2764740 1 client.16497 shutdown 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7febb2764740 1 -- 172.21.15.45:0/2325169655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd980 msgr2=0x5599bdae3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7febb2764740 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd980 0x5599bdae3c50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7feb9c012820 tx=0x7feb9c012ce0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7febb2764740 1 -- 172.21.15.45:0/2325169655 shutdown_connections 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7febb2764740 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5599bdaf6600 0x5599bdaf8a40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7febb2764740 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd980 0x5599bdae3c50 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7feb90ff9640 0 client.16497 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7febb2764740 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdae4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7febb2764740 1 --2- 172.21.15.45:0/2325169655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd060 0x5599bdae3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7febb2764740 1 -- 172.21.15.45:0/2325169655 >> 172.21.15.45:0/2325169655 conn(0x5599bdad7980 msgr2=0x5599bdad96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7febb2764740 1 -- 172.21.15.45:0/2325169655 shutdown_connections 2024-01-31T05:36:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7febb2764740 1 -- 172.21.15.45:0/2325169655 wait complete. 2024-01-31T05:36:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.374+0000 7febb2764740 20 client.16497 trim_cache size 0 max 16384 2024-01-31T05:36:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.387+0000 7febb2764740 1 Processor -- start 2024-01-31T05:36:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.387+0000 7febb2764740 1 -- start start 2024-01-31T05:36:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febb2764740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd060 0x5599bdae5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febb2764740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdadd980 0x5599bdae3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febb2764740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdae6750 0x5599bdae0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febb2764740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bdad74b0 con 0x5599bdae6750 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febb2764740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bda3dd30 con 0x5599bdadd980 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febb2764740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bdad8780 con 0x5599bdadd060 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febadf44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdae6750 0x5599bdae0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febadf44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdae6750 0x5599bdae0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41928/0 (socket says 172.21.15.45:41928) 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febadf44640 1 -- 172.21.15.45:0/2271121342 learned_addr learned my addr 172.21.15.45:0/2271121342 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febad743640 1 --2- 172.21.15.45:0/2271121342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd060 0x5599bdae5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febacf42640 1 --2- 172.21.15.45:0/2271121342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdadd980 0x5599bdae3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febadf44640 1 -- 172.21.15.45:0/2271121342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd060 msgr2=0x5599bdae5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febadf44640 1 --2- 172.21.15.45:0/2271121342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd060 0x5599bdae5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febadf44640 1 -- 172.21.15.45:0/2271121342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdadd980 msgr2=0x5599bdae3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febadf44640 1 --2- 172.21.15.45:0/2271121342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdadd980 0x5599bdae3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febadf44640 1 -- 172.21.15.45:0/2271121342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599bd9e0030 con 0x5599bdae6750 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7febadf44640 1 --2- 172.21.15.45:0/2271121342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdae6750 0x5599bdae0970 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7feba002c6d0 tx=0x7feba002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147bccc3459191ad server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7feb92ffd640 1 -- 172.21.15.45:0/2271121342 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0048040 con 0x5599bdae6750 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7feb92ffd640 1 -- 172.21.15.45:0/2271121342 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feba0032360 con 0x5599bdae6750 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.388+0000 7feb92ffd640 1 -- 172.21.15.45:0/2271121342 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba00043c0 con 0x5599bdae6750 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.389+0000 7febb2764740 1 -- 172.21.15.45:0/2271121342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdae6750 msgr2=0x5599bdae0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.389+0000 7febb2764740 1 --2- 172.21.15.45:0/2271121342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdae6750 0x5599bdae0970 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7feba002c6d0 tx=0x7feba002c890 comp rx=0 tx=0).stop 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.389+0000 7febb2764740 1 -- 172.21.15.45:0/2271121342 shutdown_connections 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.389+0000 7febb2764740 1 --2- 172.21.15.45:0/2271121342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdae6750 0x5599bdae0970 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.389+0000 7febb2764740 1 --2- 172.21.15.45:0/2271121342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdadd980 0x5599bdae3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.389+0000 7febb2764740 1 --2- 172.21.15.45:0/2271121342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd060 0x5599bdae5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.389+0000 7febb2764740 1 -- 172.21.15.45:0/2271121342 >> 172.21.15.45:0/2271121342 conn(0x5599bdad7980 msgr2=0x5599bdae1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.389+0000 7febb2764740 1 -- 172.21.15.45:0/2271121342 shutdown_connections 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.389+0000 7febb2764740 1 -- 172.21.15.45:0/2271121342 wait complete. 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.389+0000 7febb2764740 1 -- start start 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febb2764740 1 Processor -- start 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febb2764740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 0x5599bdaed830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febb2764740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd980 0x5599bdaebbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febb2764740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdaec250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febb2764740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bdaec920 con 0x5599bdadd060 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febb2764740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bdaeca90 con 0x5599bdae6750 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febb2764740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bdb4cbc0 con 0x5599bdadd980 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febb2764740 10 client.0 _ll_register_callbacks cb 0x7feba0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febad743640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 0x5599bdaed830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febad743640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 0x5599bdaed830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41932/0 (socket says 172.21.15.45:41932) 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febad743640 1 -- 172.21.15.45:0/2312172012 learned_addr learned my addr 172.21.15.45:0/2312172012 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febacf42640 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd980 0x5599bdaebbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febadf44640 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdaec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febad743640 1 -- 172.21.15.45:0/2312172012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd980 msgr2=0x5599bdaebbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febad743640 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd980 0x5599bdaebbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febad743640 1 -- 172.21.15.45:0/2312172012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 msgr2=0x5599bdaec250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febad743640 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdaec250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febad743640 1 -- 172.21.15.45:0/2312172012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599bda3ce40 con 0x5599bdadd060 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.390+0000 7febadf44640 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdaec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7febad743640 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 0x5599bdaed830 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7feba4004a60 tx=0x7feba4069790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3199196fb1a39 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7feb90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7feb90ff9640 1 -- 172.21.15.45:0/2312172012 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba409e1d0 con 0x5599bdadd060 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7feb90ff9640 1 -- 172.21.15.45:0/2312172012 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7feba40cd040 con 0x5599bdadd060 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7febb2764740 1 -- 172.21.15.45:0/2312172012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feba0032a90 con 0x5599bdadd060 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7feb90ff9640 1 -- 172.21.15.45:0/2312172012 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7feba40a2410 con 0x5599bdadd060 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7febb2764740 10 client.16503 fetch_fsmap learned FSMap version 31 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7febb2764740 1 -- 172.21.15.45:0/2312172012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5599bda05a50 con 0x5599bdadd060 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7feb90ff9640 1 -- 172.21.15.45:0/2312172012 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feba0032a90 con 0x5599bdadd060 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7feb90ff9640 1 -- 172.21.15.45:0/2312172012 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feba401f060 con 0x5599bdadd060 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7febb2764740 10 client.16503 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:36:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7febb2764740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:36:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7febb2764740 20 client.16503 populate_metadata read hostname 'smithi045' 2024-01-31T05:36:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7febb2764740 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5599bdb4e3d0 0x5599bdb507c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7febb2764740 4 client.16503 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:36:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.391+0000 7febb2764740 1 -- 172.21.15.45:0/2312172012 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5599bd744600 con 0x5599bdb4e3d0 2024-01-31T05:36:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.392+0000 7febacf42640 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5599bdb4e3d0 0x5599bdb507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.392+0000 7febacf42640 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5599bdb4e3d0 0x5599bdb507c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899e684fd72d2e2f server_cookie=14ca2e5bf82ad192 in_seq=0 out_seq=0 2024-01-31T05:36:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.392+0000 7feb90ff9640 10 client.16503 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7feb90ff9640 1 -- 172.21.15.45:0/2312172012 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5599bd744600 con 0x5599bdb4e3d0 2024-01-31T05:36:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7feb90ff9640 10 client.16503 handle_command_reply: tid=0 2024-01-31T05:36:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7febb2764740 1 client.16503 shutdown 2024-01-31T05:36:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7febb2764740 1 -- 172.21.15.45:0/2312172012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 msgr2=0x5599bdaed830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7febb2764740 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 0x5599bdaed830 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7feba4004a60 tx=0x7feba4069790 comp rx=0 tx=0).stop 2024-01-31T05:36:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7febb2764740 1 -- 172.21.15.45:0/2312172012 shutdown_connections 2024-01-31T05:36:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7febb2764740 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5599bdb4e3d0 0x5599bdb507c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7febb2764740 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599bdae6750 0x5599bdaec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7febb2764740 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5599bdadd980 0x5599bdaebbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7feb90ff9640 0 client.16503 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7febb2764740 1 --2- 172.21.15.45:0/2312172012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599bdadd060 0x5599bdaed830 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7febb2764740 1 -- 172.21.15.45:0/2312172012 >> 172.21.15.45:0/2312172012 conn(0x5599bdad7980 msgr2=0x5599bdad8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7febb2764740 1 -- 172.21.15.45:0/2312172012 shutdown_connections 2024-01-31T05:36:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7febb2764740 1 -- 172.21.15.45:0/2312172012 wait complete. 2024-01-31T05:36:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.489+0000 7febb2764740 20 client.16503 trim_cache size 0 max 16384 2024-01-31T05:36:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:36:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:36:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:36:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:36:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.490+0000 7febaf9a7640 1 -- 172.21.15.45:0/3900388834 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7feb940a14f0 msgr2=0x7feb940a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.490+0000 7febaf9a7640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7feb940a14f0 0x7feb940a39b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feba0009ca0 tx=0x7feba0049000 comp rx=0 tx=0).stop 2024-01-31T05:36:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.490+0000 7febaf9a7640 1 -- 172.21.15.45:0/3900388834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba80ffe40 msgr2=0x7feba81971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.490+0000 7febaf9a7640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba80ffe40 0x7feba81971c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7feba400e9f0 tx=0x7feba400edf0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.490+0000 7febaf9a7640 1 -- 172.21.15.45:0/3900388834 shutdown_connections 2024-01-31T05:36:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.490+0000 7febaf9a7640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7feb940a14f0 0x7feb940a39b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.490+0000 7febaf9a7640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feba8104b90 0x7feba819ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.490+0000 7febaf9a7640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba8100800 0x7feba8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.490+0000 7febaf9a7640 1 --2- 172.21.15.45:0/3900388834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba80ffe40 0x7feba81971c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.490+0000 7febaf9a7640 1 -- 172.21.15.45:0/3900388834 >> 172.21.15.45:0/3900388834 conn(0x7feba8076a40 msgr2=0x7feba80fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.490+0000 7febaf9a7640 1 -- 172.21.15.45:0/3900388834 shutdown_connections 2024-01-31T05:36:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:16.490+0000 7febaf9a7640 1 -- 172.21.15.45:0/3900388834 wait complete. 2024-01-31T05:36:16.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 6955 36.54 MB/sec warmup 15 sec latency 150.395 ms 2024-01-31T05:36:17.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 7547 36.68 MB/sec warmup 16 sec latency 170.755 ms 2024-01-31T05:36:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:17 smithi156 ceph-mon[35720]: pgmap v433: 97 pgs: 97 active+clean; 615 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 360 op/s 2024-01-31T05:36:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:17 smithi186 ceph-mon[35659]: pgmap v433: 97 pgs: 97 active+clean; 615 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 360 op/s 2024-01-31T05:36:18.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:17 smithi045 ceph-mon[31585]: pgmap v433: 97 pgs: 97 active+clean; 615 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 360 op/s 2024-01-31T05:36:18.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 8030 36.34 MB/sec warmup 17 sec latency 252.941 ms 2024-01-31T05:36:19.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 8409 35.61 MB/sec warmup 18 sec latency 195.714 ms 2024-01-31T05:36:19.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:19 smithi156 ceph-mon[35720]: pgmap v434: 97 pgs: 97 active+clean; 615 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 360 op/s 2024-01-31T05:36:20.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:19 smithi186 ceph-mon[35659]: pgmap v434: 97 pgs: 97 active+clean; 615 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 360 op/s 2024-01-31T05:36:20.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:19 smithi045 ceph-mon[31585]: pgmap v434: 97 pgs: 97 active+clean; 615 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 360 op/s 2024-01-31T05:36:20.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 8648 34.15 MB/sec warmup 19 sec latency 238.949 ms 2024-01-31T05:36:21.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 9007 32.89 MB/sec warmup 20 sec latency 83.631 ms 2024-01-31T05:36:21.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:21 smithi156 ceph-mon[35720]: pgmap v435: 97 pgs: 97 active+clean; 615 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 360 op/s 2024-01-31T05:36:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:21 smithi186 ceph-mon[35659]: pgmap v435: 97 pgs: 97 active+clean; 615 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 360 op/s 2024-01-31T05:36:22.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:21 smithi045 ceph-mon[31585]: pgmap v435: 97 pgs: 97 active+clean; 615 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 360 op/s 2024-01-31T05:36:22.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 9548 32.47 MB/sec warmup 21 sec latency 67.393 ms 2024-01-31T05:36:23.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 10087 32.64 MB/sec warmup 22 sec latency 155.816 ms 2024-01-31T05:36:23.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:23 smithi156 ceph-mon[35720]: pgmap v436: 97 pgs: 97 active+clean; 631 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 339 op/s 2024-01-31T05:36:24.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:23 smithi186 ceph-mon[35659]: pgmap v436: 97 pgs: 97 active+clean; 631 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 339 op/s 2024-01-31T05:36:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:23 smithi045 ceph-mon[31585]: pgmap v436: 97 pgs: 97 active+clean; 631 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 339 op/s 2024-01-31T05:36:24.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 10804 32.21 MB/sec warmup 23 sec latency 76.875 ms 2024-01-31T05:36:25.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 11333 32.38 MB/sec warmup 24 sec latency 180.277 ms 2024-01-31T05:36:25.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:25 smithi156 ceph-mon[35720]: pgmap v437: 97 pgs: 97 active+clean; 631 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 311 op/s 2024-01-31T05:36:26.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:25 smithi186 ceph-mon[35659]: pgmap v437: 97 pgs: 97 active+clean; 631 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 311 op/s 2024-01-31T05:36:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:25 smithi045 ceph-mon[31585]: pgmap v437: 97 pgs: 97 active+clean; 631 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 311 op/s 2024-01-31T05:36:26.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 12071 33.23 MB/sec warmup 25 sec latency 177.917 ms 2024-01-31T05:36:27.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 12494 33.14 MB/sec warmup 26 sec latency 189.113 ms 2024-01-31T05:36:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:27 smithi156 ceph-mon[35720]: pgmap v438: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.0 MiB/s wr, 456 op/s 2024-01-31T05:36:28.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:27 smithi186 ceph-mon[35659]: pgmap v438: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.0 MiB/s wr, 456 op/s 2024-01-31T05:36:28.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:27 smithi045 ceph-mon[31585]: pgmap v438: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.0 MiB/s wr, 456 op/s 2024-01-31T05:36:28.642 INFO:tasks.workunit.client.0.smithi045.stdout: 10 12780 32.23 MB/sec warmup 27 sec latency 301.810 ms 2024-01-31T05:36:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.587+0000 7f6e914d6640 1 -- 172.21.15.45:0/3921497251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e8c100f30 msgr2=0x7f6e8c101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.587+0000 7f6e914d6640 1 --2- 172.21.15.45:0/3921497251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e8c100f30 0x7f6e8c101310 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009930 tx=0x7f6e7c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:36:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.587+0000 7f6e914d6640 1 -- 172.21.15.45:0/3921497251 shutdown_connections 2024-01-31T05:36:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.587+0000 7f6e914d6640 1 --2- 172.21.15.45:0/3921497251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c106000 0x7f6e8c10dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.587+0000 7f6e914d6640 1 --2- 172.21.15.45:0/3921497251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e8c1018e0 0x7f6e8c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.587+0000 7f6e914d6640 1 --2- 172.21.15.45:0/3921497251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e8c100f30 0x7f6e8c101310 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.587+0000 7f6e914d6640 1 -- 172.21.15.45:0/3921497251 >> 172.21.15.45:0/3921497251 conn(0x7f6e8c0fc7d0 msgr2=0x7f6e8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.587+0000 7f6e914d6640 1 -- 172.21.15.45:0/3921497251 shutdown_connections 2024-01-31T05:36:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.587+0000 7f6e914d6640 1 -- 172.21.15.45:0/3921497251 wait complete. 2024-01-31T05:36:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e914d6640 1 Processor -- start 2024-01-31T05:36:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e914d6640 1 -- start start 2024-01-31T05:36:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e914d6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e8c100f30 0x7f6e8c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e914d6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c106000 0x7f6e8c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e914d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e8c19dcb0 0x7f6e8c19e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e914d6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c19e6a0 con 0x7f6e8c19dcb0 2024-01-31T05:36:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e914d6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c19e810 con 0x7f6e8c106000 2024-01-31T05:36:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e914d6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c19e980 con 0x7f6e8c100f30 2024-01-31T05:36:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e827fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c106000 0x7f6e8c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e8affd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e8c100f30 0x7f6e8c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e827fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c106000 0x7f6e8c199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:45540/0 (socket says 172.21.15.45:45540) 2024-01-31T05:36:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e8affd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e8c100f30 0x7f6e8c1993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:48124/0 (socket says 172.21.15.45:48124) 2024-01-31T05:36:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e827fc640 1 -- 172.21.15.45:0/1731668115 learned_addr learned my addr 172.21.15.45:0/1731668115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.588+0000 7f6e8b7fe640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e8c19dcb0 0x7f6e8c19e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.589+0000 7f6e827fc640 1 -- 172.21.15.45:0/1731668115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e8c100f30 msgr2=0x7f6e8c1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.589+0000 7f6e827fc640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e8c100f30 0x7f6e8c1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.589+0000 7f6e827fc640 1 -- 172.21.15.45:0/1731668115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e8c19dcb0 msgr2=0x7f6e8c19e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.589+0000 7f6e827fc640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e8c19dcb0 0x7f6e8c19e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.589+0000 7f6e827fc640 1 -- 172.21.15.45:0/1731668115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c009560 con 0x7f6e8c106000 2024-01-31T05:36:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.589+0000 7f6e827fc640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c106000 0x7f6e8c199930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400e9f0 tx=0x7f6e7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.589+0000 7f6e88ff9640 1 -- 172.21.15.45:0/1731668115 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e740096c0 con 0x7f6e8c106000 2024-01-31T05:36:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.589+0000 7f6e88ff9640 1 -- 172.21.15.45:0/1731668115 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e740054e0 con 0x7f6e8c106000 2024-01-31T05:36:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.589+0000 7f6e914d6640 1 -- 172.21.15.45:0/1731668115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8c1a5f60 con 0x7f6e8c106000 2024-01-31T05:36:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.589+0000 7f6e88ff9640 1 -- 172.21.15.45:0/1731668115 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74010430 con 0x7f6e8c106000 2024-01-31T05:36:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.589+0000 7f6e914d6640 1 -- 172.21.15.45:0/1731668115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8c1a6380 con 0x7f6e8c106000 2024-01-31T05:36:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.592+0000 7f6e88ff9640 1 -- 172.21.15.45:0/1731668115 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f6e74005020 con 0x7f6e8c106000 2024-01-31T05:36:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.592+0000 7f6e94375740 1 Processor -- start 2024-01-31T05:36:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.592+0000 7f6e94375740 1 -- start start 2024-01-31T05:36:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e88ff9640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f6e780a14f0 0x7f6e780a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e94375740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0447ff060 0x55b0447ff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e94375740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044808210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e94375740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b044808750 0x55b04480ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e94375740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04476e040 con 0x55b0447ff060 2024-01-31T05:36:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e94375740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04476b730 con 0x55b0447ff980 2024-01-31T05:36:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e94375740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b044767020 con 0x55b044808750 2024-01-31T05:36:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e8affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0447ff060 0x55b0447ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e8b7fe640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f6e780a14f0 0x7f6e780a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e8affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0447ff060 0x55b0447ff440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53176/0 (socket says 172.21.15.45:53176) 2024-01-31T05:36:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e8affd640 1 -- 172.21.15.45:0/4200123417 learned_addr learned my addr 172.21.15.45:0/4200123417 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e827fc640 1 --2- 172.21.15.45:0/4200123417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044808210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e88ff9640 1 -- 172.21.15.45:0/1731668115 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6e74014070 con 0x7f6e8c106000 2024-01-31T05:36:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e8b7fe640 1 --2- 172.21.15.45:0/4200123417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b044808750 0x55b04480ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e8affd640 1 -- 172.21.15.45:0/4200123417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b044808750 msgr2=0x55b04480ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e8affd640 1 --2- 172.21.15.45:0/4200123417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b044808750 0x55b04480ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e8affd640 1 -- 172.21.15.45:0/4200123417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 msgr2=0x55b044808210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e8affd640 1 --2- 172.21.15.45:0/4200123417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044808210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e8affd640 1 -- 172.21.15.45:0/4200123417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b04475ee40 con 0x55b0447ff060 2024-01-31T05:36:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.593+0000 7f6e827fc640 1 --2- 172.21.15.45:0/4200123417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044808210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.594+0000 7f6e8affd640 1 --2- 172.21.15.45:0/4200123417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0447ff060 0x55b0447ff440 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c036040 tx=0x7f6e7c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556636de7da11603 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.594+0000 7f6e8b7fe640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f6e780a14f0 0x7f6e780a39b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55b0447fada0 tx=0x7f6e84009a90 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.594+0000 7f6e817fa640 1 -- 172.21.15.45:0/4200123417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0046c0 con 0x55b0447ff060 2024-01-31T05:36:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.594+0000 7f6e817fa640 1 -- 172.21.15.45:0/4200123417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e7c004820 con 0x55b0447ff060 2024-01-31T05:36:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.594+0000 7f6e817fa640 1 -- 172.21.15.45:0/4200123417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c005200 con 0x55b0447ff060 2024-01-31T05:36:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.594+0000 7f6e94375740 1 -- 172.21.15.45:0/4200123417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0447ff060 msgr2=0x55b0447ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.594+0000 7f6e94375740 1 --2- 172.21.15.45:0/4200123417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0447ff060 0x55b0447ff440 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c036040 tx=0x7f6e7c005d50 comp rx=0 tx=0).stop 2024-01-31T05:36:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 -- 172.21.15.45:0/4200123417 shutdown_connections 2024-01-31T05:36:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 --2- 172.21.15.45:0/4200123417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b044808750 0x55b04480ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 --2- 172.21.15.45:0/4200123417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044808210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 --2- 172.21.15.45:0/4200123417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0447ff060 0x55b0447ff440 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 -- 172.21.15.45:0/4200123417 >> 172.21.15.45:0/4200123417 conn(0x55b0447f9980 msgr2=0x55b0447f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 -- 172.21.15.45:0/4200123417 shutdown_connections 2024-01-31T05:36:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 -- 172.21.15.45:0/4200123417 wait complete. 2024-01-31T05:36:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 -- start start 2024-01-31T05:36:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 Processor -- start 2024-01-31T05:36:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff060 0x55b044805650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044805b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b0448060d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0447201b0 con 0x55b044808750 2024-01-31T05:36:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04471fe50 con 0x55b0447ff980 2024-01-31T05:36:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0446e5090 con 0x55b0447ff060 2024-01-31T05:36:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e94375740 10 client.0 _ll_register_callbacks cb 0x55b04470aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:36:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b0448060d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b0448060d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53184/0 (socket says 172.21.15.45:53184) 2024-01-31T05:36:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e8b7fe640 1 -- 172.21.15.45:0/3875801721 learned_addr learned my addr 172.21.15.45:0/3875801721 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e8affd640 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff060 0x55b044805650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.595+0000 7f6e827fc640 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044805b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e8b7fe640 1 -- 172.21.15.45:0/3875801721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff060 msgr2=0x55b044805650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e8b7fe640 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff060 0x55b044805650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e8b7fe640 1 -- 172.21.15.45:0/3875801721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 msgr2=0x55b044805b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e8b7fe640 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044805b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e8b7fe640 1 -- 172.21.15.45:0/3875801721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8c10e3e0 con 0x55b044808750 2024-01-31T05:36:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e827fc640 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044805b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e8b7fe640 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b0448060d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400fc30 tx=0x7f6e84009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2eaaac7ad4f228 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e727fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:36:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e727fc640 1 -- 172.21.15.45:0/3875801721 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84010e00 con 0x55b044808750 2024-01-31T05:36:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e94375740 1 -- 172.21.15.45:0/3875801721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b044760010 con 0x55b044808750 2024-01-31T05:36:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e727fc640 1 -- 172.21.15.45:0/3875801721 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6e84021480 con 0x55b044808750 2024-01-31T05:36:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e727fc640 1 -- 172.21.15.45:0/3875801721 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e84021640 con 0x55b044808750 2024-01-31T05:36:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e94375740 10 client.16515 fetch_fsmap learned FSMap version 31 2024-01-31T05:36:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e94375740 1 -- 172.21.15.45:0/3875801721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b0447656a0 con 0x55b044808750 2024-01-31T05:36:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.596+0000 7f6e727fc640 1 -- 172.21.15.45:0/3875801721 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b044760010 con 0x55b044808750 2024-01-31T05:36:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.597+0000 7f6e727fc640 1 -- 172.21.15.45:0/3875801721 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6e84020780 con 0x55b044808750 2024-01-31T05:36:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.597+0000 7f6e94375740 10 client.16515 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:36:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.597+0000 7f6e94375740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:36:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.597+0000 7f6e94375740 20 client.16515 populate_metadata read hostname 'smithi045' 2024-01-31T05:36:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.597+0000 7f6e94375740 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b0448184c0 0x55b04481a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.597+0000 7f6e94375740 4 client.16515 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:36:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.597+0000 7f6e94375740 1 -- 172.21.15.45:0/3875801721 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b04475fd30 con 0x55b0448184c0 2024-01-31T05:36:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.597+0000 7f6e8affd640 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b0448184c0 0x55b04481a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.597+0000 7f6e8affd640 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b0448184c0 0x55b04481a8b0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f2453a2774eb2d8 server_cookie=da5c6a48691d42ee in_seq=0 out_seq=0 2024-01-31T05:36:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.597+0000 7f6e727fc640 10 client.16515 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e727fc640 1 -- 172.21.15.45:0/3875801721 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b04475fd30 con 0x55b0448184c0 2024-01-31T05:36:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e727fc640 10 client.16515 handle_command_reply: tid=0 2024-01-31T05:36:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e94375740 1 client.16515 shutdown 2024-01-31T05:36:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e94375740 1 -- 172.21.15.45:0/3875801721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 msgr2=0x55b0448060d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e94375740 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b0448060d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400fc30 tx=0x7f6e84009500 comp rx=0 tx=0).stop 2024-01-31T05:36:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e94375740 1 -- 172.21.15.45:0/3875801721 shutdown_connections 2024-01-31T05:36:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e94375740 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b0448184c0 0x55b04481a8b0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e94375740 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b0448060d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e727fc640 0 client.16515 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e94375740 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044805b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e94375740 1 --2- 172.21.15.45:0/3875801721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff060 0x55b044805650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e94375740 1 -- 172.21.15.45:0/3875801721 >> 172.21.15.45:0/3875801721 conn(0x55b0447f9980 msgr2=0x55b0447fb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e94375740 1 -- 172.21.15.45:0/3875801721 shutdown_connections 2024-01-31T05:36:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e94375740 1 -- 172.21.15.45:0/3875801721 wait complete. 2024-01-31T05:36:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.598+0000 7f6e94375740 20 client.16515 trim_cache size 0 max 16384 2024-01-31T05:36:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e94375740 1 Processor -- start 2024-01-31T05:36:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e94375740 1 -- start start 2024-01-31T05:36:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e94375740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff060 0x55b044807d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e94375740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044805ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e94375740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b0448028d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e94375740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b044760010 con 0x55b044808750 2024-01-31T05:36:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e94375740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0447fe340 con 0x55b0447ff980 2024-01-31T05:36:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e94375740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04480aec0 con 0x55b0447ff060 2024-01-31T05:36:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b0448028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e8affd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff060 0x55b044807d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e827fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044805ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b0448028d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53188/0 (socket says 172.21.15.45:53188) 2024-01-31T05:36:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.615+0000 7f6e8b7fe640 1 -- 172.21.15.45:0/1774228336 learned_addr learned my addr 172.21.15.45:0/1774228336 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.616+0000 7f6e827fc640 1 -- 172.21.15.45:0/1774228336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff060 msgr2=0x55b044807d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.616+0000 7f6e827fc640 1 --2- 172.21.15.45:0/1774228336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff060 0x55b044807d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.616+0000 7f6e827fc640 1 -- 172.21.15.45:0/1774228336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 msgr2=0x55b0448028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.616+0000 7f6e827fc640 1 --2- 172.21.15.45:0/1774228336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b0448028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.616+0000 7f6e827fc640 1 -- 172.21.15.45:0/1774228336 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b044702030 con 0x55b0447ff980 2024-01-31T05:36:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.616+0000 7f6e8affd640 1 --2- 172.21.15.45:0/1774228336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff060 0x55b044807d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.616+0000 7f6e8b7fe640 1 --2- 172.21.15.45:0/1774228336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b0448028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.616+0000 7f6e827fc640 1 --2- 172.21.15.45:0/1774228336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044805ef0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6e740caa00 tx=0x7f6e740964b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9548ccbd680e8db server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.616+0000 7f6e80ff9640 1 -- 172.21.15.45:0/1774228336 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74063cc0 con 0x55b0447ff980 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.616+0000 7f6e80ff9640 1 -- 172.21.15.45:0/1774228336 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e7409a7b0 con 0x55b0447ff980 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.616+0000 7f6e80ff9640 1 -- 172.21.15.45:0/1774228336 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e740046c0 con 0x55b0447ff980 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.617+0000 7f6e94375740 1 -- 172.21.15.45:0/1774228336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 msgr2=0x55b044805ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.617+0000 7f6e94375740 1 --2- 172.21.15.45:0/1774228336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044805ef0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6e740caa00 tx=0x7f6e740964b0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.617+0000 7f6e94375740 1 -- 172.21.15.45:0/1774228336 shutdown_connections 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.617+0000 7f6e94375740 1 --2- 172.21.15.45:0/1774228336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b0448028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.617+0000 7f6e94375740 1 --2- 172.21.15.45:0/1774228336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff980 0x55b044805ef0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.617+0000 7f6e94375740 1 --2- 172.21.15.45:0/1774228336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff060 0x55b044807d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.617+0000 7f6e94375740 1 -- 172.21.15.45:0/1774228336 >> 172.21.15.45:0/1774228336 conn(0x55b0447f9980 msgr2=0x55b044800950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.617+0000 7f6e94375740 1 -- 172.21.15.45:0/1774228336 shutdown_connections 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.617+0000 7f6e94375740 1 -- 172.21.15.45:0/1774228336 wait complete. 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.617+0000 7f6e94375740 1 -- start start 2024-01-31T05:36:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.617+0000 7f6e94375740 1 Processor -- start 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e94375740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff060 0x55b04480d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e94375740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff980 0x55b04480dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e94375740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b04486d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e94375740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04480e330 con 0x55b044808750 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e94375740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04486df20 con 0x55b0447ff060 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e94375740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04486e0e0 con 0x55b0447ff980 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e94375740 10 client.0 _ll_register_callbacks cb 0x7f6e74005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e8affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff060 0x55b04480d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b04486d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e827fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff980 0x55b04480dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e8affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff060 0x55b04480d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:45588/0 (socket says 172.21.15.45:45588) 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e8affd640 1 -- 172.21.15.45:0/4111968306 learned_addr learned my addr 172.21.15.45:0/4111968306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e827fc640 1 -- 172.21.15.45:0/4111968306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff060 msgr2=0x55b04480d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e827fc640 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff060 0x55b04480d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e827fc640 1 -- 172.21.15.45:0/4111968306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 msgr2=0x55b04486d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e827fc640 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b04486d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e827fc640 1 -- 172.21.15.45:0/4111968306 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b04475ee40 con 0x55b0447ff980 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e8b7fe640 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b04486d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e8affd640 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff060 0x55b04480d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e827fc640 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff980 0x55b04480dbb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e74099d60 tx=0x7f6e7409a920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f779439316a59179 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e727fc640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.618+0000 7f6e727fc640 1 -- 172.21.15.45:0/4111968306 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74063cc0 con 0x55b0447ff980 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e94375740 1 -- 172.21.15.45:0/4111968306 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0447f5080 con 0x55b0447ff980 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e727fc640 1 -- 172.21.15.45:0/4111968306 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6e7409e200 con 0x55b0447ff980 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e727fc640 1 -- 172.21.15.45:0/4111968306 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e740a2440 con 0x55b0447ff980 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e94375740 10 client.26083 fetch_fsmap learned FSMap version 31 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e94375740 1 -- 172.21.15.45:0/4111968306 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b0447656a0 con 0x55b0447ff980 2024-01-31T05:36:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e727fc640 1 -- 172.21.15.45:0/4111968306 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0447f5080 con 0x55b0447ff980 2024-01-31T05:36:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e727fc640 1 -- 172.21.15.45:0/4111968306 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6e7401f060 con 0x55b0447ff980 2024-01-31T05:36:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e94375740 10 client.26083 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:36:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e94375740 10 client.26083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:36:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e94375740 20 client.26083 populate_metadata read hostname 'smithi045' 2024-01-31T05:36:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e94375740 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b0448184c0 0x55b044871600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e94375740 4 client.26083 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:36:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e94375740 1 -- 172.21.15.45:0/4111968306 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f6e7c004820 con 0x55b0448184c0 2024-01-31T05:36:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.619+0000 7f6e8affd640 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b0448184c0 0x55b044871600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.620+0000 7f6e727fc640 10 client.26083 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.620+0000 7f6e8affd640 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b0448184c0 0x55b044871600 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dabba76140efa41 server_cookie=62275467ff914afd in_seq=0 out_seq=0 2024-01-31T05:36:29.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 13149 31.58 MB/sec warmup 28 sec latency 223.075 ms 2024-01-31T05:36:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.671+0000 7f6e727fc640 1 -- 172.21.15.45:0/4111968306 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f6e7c004820 con 0x55b0448184c0 2024-01-31T05:36:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.671+0000 7f6e727fc640 10 client.26083 handle_command_reply: tid=0 2024-01-31T05:36:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e94375740 1 client.26083 shutdown 2024-01-31T05:36:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e94375740 1 -- 172.21.15.45:0/4111968306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff980 msgr2=0x55b04480dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e94375740 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff980 0x55b04480dbb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e74099d60 tx=0x7f6e7409a920 comp rx=0 tx=0).stop 2024-01-31T05:36:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e94375740 1 -- 172.21.15.45:0/4111968306 shutdown_connections 2024-01-31T05:36:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e94375740 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55b0448184c0 0x55b044871600 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e94375740 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b044808750 0x55b04486d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e94375740 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b0447ff980 0x55b04480dbb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e727fc640 0 client.26083 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e94375740 1 --2- 172.21.15.45:0/4111968306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0447ff060 0x55b04480d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e94375740 1 -- 172.21.15.45:0/4111968306 >> 172.21.15.45:0/4111968306 conn(0x55b0447f9980 msgr2=0x55b044805f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e94375740 1 -- 172.21.15.45:0/4111968306 shutdown_connections 2024-01-31T05:36:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e94375740 1 -- 172.21.15.45:0/4111968306 wait complete. 2024-01-31T05:36:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.672+0000 7f6e94375740 20 client.26083 trim_cache size 0 max 16384 2024-01-31T05:36:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:36:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:36:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:36:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:36:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.673+0000 7f6e914d6640 1 -- 172.21.15.45:0/1731668115 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f6e780a14f0 msgr2=0x7f6e780a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.673+0000 7f6e914d6640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f6e780a14f0 0x7f6e780a39b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55b0447fada0 tx=0x7f6e84009a90 comp rx=0 tx=0).stop 2024-01-31T05:36:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.673+0000 7f6e914d6640 1 -- 172.21.15.45:0/1731668115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c106000 msgr2=0x7f6e8c199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.673+0000 7f6e914d6640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c106000 0x7f6e8c199930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400e9f0 tx=0x7f6e7400eeb0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.673+0000 7f6e914d6640 1 -- 172.21.15.45:0/1731668115 shutdown_connections 2024-01-31T05:36:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.673+0000 7f6e914d6640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f6e780a14f0 0x7f6e780a39b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.673+0000 7f6e914d6640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e8c19dcb0 0x7f6e8c19e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.673+0000 7f6e914d6640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c106000 0x7f6e8c199930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.673+0000 7f6e914d6640 1 --2- 172.21.15.45:0/1731668115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e8c100f30 0x7f6e8c1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.673+0000 7f6e914d6640 1 -- 172.21.15.45:0/1731668115 >> 172.21.15.45:0/1731668115 conn(0x7f6e8c0fc7d0 msgr2=0x7f6e8c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.673+0000 7f6e914d6640 1 -- 172.21.15.45:0/1731668115 shutdown_connections 2024-01-31T05:36:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:29.673+0000 7f6e914d6640 1 -- 172.21.15.45:0/1731668115 wait complete. 2024-01-31T05:36:29.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:29 smithi156 ceph-mon[35720]: pgmap v439: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-01-31T05:36:30.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:29 smithi186 ceph-mon[35659]: pgmap v439: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-01-31T05:36:30.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:29 smithi045 ceph-mon[31585]: pgmap v439: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-01-31T05:36:30.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 13584 31.11 MB/sec warmup 29 sec latency 177.069 ms 2024-01-31T05:36:31.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 14100 31.13 MB/sec warmup 30 sec latency 136.932 ms 2024-01-31T05:36:31.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:31 smithi156 ceph-mon[35720]: pgmap v440: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-01-31T05:36:32.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:31 smithi186 ceph-mon[35659]: pgmap v440: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-01-31T05:36:32.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:31 smithi045 ceph-mon[31585]: pgmap v440: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-01-31T05:36:32.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 14762 31.05 MB/sec warmup 31 sec latency 63.537 ms 2024-01-31T05:36:33.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 15428 31.05 MB/sec warmup 32 sec latency 183.917 ms 2024-01-31T05:36:33.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:33 smithi156 ceph-mon[35720]: pgmap v441: 97 pgs: 97 active+clean; 662 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.6 MiB/s wr, 464 op/s 2024-01-31T05:36:34.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:33 smithi186 ceph-mon[35659]: pgmap v441: 97 pgs: 97 active+clean; 662 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.6 MiB/s wr, 464 op/s 2024-01-31T05:36:34.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:33 smithi045 ceph-mon[31585]: pgmap v441: 97 pgs: 97 active+clean; 662 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.6 MiB/s wr, 464 op/s 2024-01-31T05:36:34.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 15767 30.75 MB/sec warmup 33 sec latency 572.656 ms 2024-01-31T05:36:35.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 16364 31.23 MB/sec warmup 34 sec latency 195.048 ms 2024-01-31T05:36:35.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:35 smithi156 ceph-mon[35720]: pgmap v442: 97 pgs: 97 active+clean; 662 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 320 op/s 2024-01-31T05:36:36.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:35 smithi186 ceph-mon[35659]: pgmap v442: 97 pgs: 97 active+clean; 662 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 320 op/s 2024-01-31T05:36:36.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:35 smithi045 ceph-mon[31585]: pgmap v442: 97 pgs: 97 active+clean; 662 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 320 op/s 2024-01-31T05:36:36.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 16759 31.09 MB/sec warmup 35 sec latency 238.084 ms 2024-01-31T05:36:37.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 17129 30.86 MB/sec warmup 36 sec latency 57.042 ms 2024-01-31T05:36:37.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:37 smithi156 ceph-mon[35720]: pgmap v443: 97 pgs: 97 active+clean; 848 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 20 MiB/s wr, 500 op/s 2024-01-31T05:36:38.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:37 smithi186 ceph-mon[35659]: pgmap v443: 97 pgs: 97 active+clean; 848 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 20 MiB/s wr, 500 op/s 2024-01-31T05:36:38.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:37 smithi045 ceph-mon[31585]: pgmap v443: 97 pgs: 97 active+clean; 848 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 20 MiB/s wr, 500 op/s 2024-01-31T05:36:38.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 17411 30.29 MB/sec warmup 37 sec latency 56.693 ms 2024-01-31T05:36:39.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 17616 29.70 MB/sec warmup 38 sec latency 483.933 ms 2024-01-31T05:36:39.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:39 smithi156 ceph-mon[35720]: pgmap v444: 97 pgs: 97 active+clean; 848 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 18 MiB/s wr, 355 op/s 2024-01-31T05:36:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:39 smithi186 ceph-mon[35659]: pgmap v444: 97 pgs: 97 active+clean; 848 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 18 MiB/s wr, 355 op/s 2024-01-31T05:36:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:39 smithi045 ceph-mon[31585]: pgmap v444: 97 pgs: 97 active+clean; 848 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 18 MiB/s wr, 355 op/s 2024-01-31T05:36:40.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 17945 29.16 MB/sec warmup 39 sec latency 153.550 ms 2024-01-31T05:36:41.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 18433 29.10 MB/sec warmup 40 sec latency 68.635 ms 2024-01-31T05:36:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:41 smithi156 ceph-mon[35720]: pgmap v445: 97 pgs: 97 active+clean; 848 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 18 MiB/s wr, 355 op/s 2024-01-31T05:36:42.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:41 smithi186 ceph-mon[35659]: pgmap v445: 97 pgs: 97 active+clean; 848 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 18 MiB/s wr, 355 op/s 2024-01-31T05:36:42.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:41 smithi045 ceph-mon[31585]: pgmap v445: 97 pgs: 97 active+clean; 848 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 18 MiB/s wr, 355 op/s 2024-01-31T05:36:42.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 18936 29.17 MB/sec warmup 41 sec latency 165.565 ms 2024-01-31T05:36:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.776+0000 7f5dbd460640 1 -- 172.21.15.45:0/1783779947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db8100f40 msgr2=0x7f5db8101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.776+0000 7f5dbd460640 1 --2- 172.21.15.45:0/1783779947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db8100f40 0x7f5db8101320 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009930 tx=0x7f5da802bc30 comp rx=0 tx=0).stop 2024-01-31T05:36:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.776+0000 7f5dbd460640 1 -- 172.21.15.45:0/1783779947 shutdown_connections 2024-01-31T05:36:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.776+0000 7f5dbd460640 1 --2- 172.21.15.45:0/1783779947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db8106010 0x7f5db810dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.776+0000 7f5dbd460640 1 --2- 172.21.15.45:0/1783779947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db81018f0 0x7f5db8105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.776+0000 7f5dbd460640 1 --2- 172.21.15.45:0/1783779947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db8100f40 0x7f5db8101320 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.776+0000 7f5dbd460640 1 -- 172.21.15.45:0/1783779947 >> 172.21.15.45:0/1783779947 conn(0x7f5db80fc7a0 msgr2=0x7f5db80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.776+0000 7f5dbd460640 1 -- 172.21.15.45:0/1783779947 shutdown_connections 2024-01-31T05:36:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.776+0000 7f5dbd460640 1 -- 172.21.15.45:0/1783779947 wait complete. 2024-01-31T05:36:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5dbd460640 1 Processor -- start 2024-01-31T05:36:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5dbd460640 1 -- start start 2024-01-31T05:36:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5dbd460640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db8100f40 0x7f5db81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5dbd460640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db8106010 0x7f5db8199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5dbd460640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db819dc90 0x7f5db819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5dbd460640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db819e680 con 0x7f5db8100f40 2024-01-31T05:36:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5dbd460640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db819e7f0 con 0x7f5db8106010 2024-01-31T05:36:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5dbd460640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db819e960 con 0x7f5db819dc90 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5db6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db8100f40 0x7f5db81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5db6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db8100f40 0x7f5db81993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44708/0 (socket says 172.21.15.45:44708) 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5db6ffd640 1 -- 172.21.15.45:0/2146409442 learned_addr learned my addr 172.21.15.45:0/2146409442 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5db67fc640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db8106010 0x7f5db8199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.777+0000 7f5db77fe640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db819dc90 0x7f5db819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.778+0000 7f5db6ffd640 1 -- 172.21.15.45:0/2146409442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db819dc90 msgr2=0x7f5db819e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.778+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db819dc90 0x7f5db819e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.778+0000 7f5db6ffd640 1 -- 172.21.15.45:0/2146409442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db8106010 msgr2=0x7f5db8199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.778+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db8106010 0x7f5db8199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.778+0000 7f5db6ffd640 1 -- 172.21.15.45:0/2146409442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac009490 con 0x7f5db8100f40 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.778+0000 7f5db67fc640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db8106010 0x7f5db8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.778+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db8100f40 0x7f5db81993d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5da8005940 tx=0x7f5da8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.778+0000 7f5d9ffff640 1 -- 172.21.15.45:0/2146409442 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8004e50 con 0x7f5db8100f40 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.778+0000 7f5d9ffff640 1 -- 172.21.15.45:0/2146409442 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5da80054c0 con 0x7f5db8100f40 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.778+0000 7f5dbd460640 1 -- 172.21.15.45:0/2146409442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da8009560 con 0x7f5db8100f40 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.778+0000 7f5d9ffff640 1 -- 172.21.15.45:0/2146409442 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8003b20 con 0x7f5db8100f40 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.778+0000 7f5dbd460640 1 -- 172.21.15.45:0/2146409442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db81a61d0 con 0x7f5db8100f40 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.780+0000 7f5d9ffff640 1 -- 172.21.15.45:0/2146409442 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f5da8003c80 con 0x7f5db8100f40 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.780+0000 7f5dc0218740 1 Processor -- start 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5dc0218740 1 -- start start 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5dc0218740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 0x55ba243ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5dc0218740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce980 0x55ba243d7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5dc0218740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243d9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5dc0218740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2433d040 con 0x55ba243ce060 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5dc0218740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2433a730 con 0x55ba243ce980 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5dc0218740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba24336020 con 0x55ba243d7750 2024-01-31T05:36:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5d9ffff640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5da00a14f0 0x7f5da00a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 0x55ba243ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce980 0x55ba243d7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db67fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243d9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 0x55ba243ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44716/0 (socket says 172.21.15.45:44716) 2024-01-31T05:36:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db77fe640 1 -- 172.21.15.45:0/656511574 learned_addr learned my addr 172.21.15.45:0/656511574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db77fe640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5da00a14f0 0x7f5da00a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5d9ffff640 1 -- 172.21.15.45:0/2146409442 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5da80ef5a0 con 0x7f5db8100f40 2024-01-31T05:36:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db77fe640 1 -- 172.21.15.45:0/656511574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 msgr2=0x55ba243d9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db77fe640 1 --2- 172.21.15.45:0/656511574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243d9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db77fe640 1 -- 172.21.15.45:0/656511574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce980 msgr2=0x55ba243d7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db77fe640 1 --2- 172.21.15.45:0/656511574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce980 0x55ba243d7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db77fe640 1 -- 172.21.15.45:0/656511574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba2432de40 con 0x55ba243ce060 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/656511574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce980 0x55ba243d7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.781+0000 7f5db67fc640 1 --2- 172.21.15.45:0/656511574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.782+0000 7f5db77fe640 1 --2- 172.21.15.45:0/656511574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 0x55ba243ce440 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5db0013da0 tx=0x7f5db00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef0ec733aa665d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.782+0000 7f5db77fe640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5da00a14f0 0x7f5da00a39b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5db0011cb0 tx=0x7f5db0015000 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.782+0000 7f5d9d7fa640 1 -- 172.21.15.45:0/656511574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0013560 con 0x55ba243ce060 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.782+0000 7f5d9d7fa640 1 -- 172.21.15.45:0/656511574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db0003e00 con 0x55ba243ce060 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.782+0000 7f5d9d7fa640 1 -- 172.21.15.45:0/656511574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db001fba0 con 0x55ba243ce060 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.782+0000 7f5dc0218740 1 -- 172.21.15.45:0/656511574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 msgr2=0x55ba243ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.782+0000 7f5dc0218740 1 --2- 172.21.15.45:0/656511574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 0x55ba243ce440 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5db0013da0 tx=0x7f5db00117b0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 -- 172.21.15.45:0/656511574 shutdown_connections 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 --2- 172.21.15.45:0/656511574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 --2- 172.21.15.45:0/656511574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce980 0x55ba243d7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 --2- 172.21.15.45:0/656511574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 0x55ba243ce440 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 -- 172.21.15.45:0/656511574 >> 172.21.15.45:0/656511574 conn(0x55ba243c8980 msgr2=0x55ba243c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 -- 172.21.15.45:0/656511574 shutdown_connections 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 -- 172.21.15.45:0/656511574 wait complete. 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 -- start start 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 Processor -- start 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce060 0x55ba243d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 0x55ba243d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba242ef1b0 con 0x55ba243ce980 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba242eee50 con 0x55ba243ce060 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba242b4090 con 0x55ba243d7750 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.783+0000 7f5dc0218740 10 client.0 _ll_register_callbacks cb 0x55ba242d9a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5db67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 0x55ba243d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5db67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 0x55ba243d4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44722/0 (socket says 172.21.15.45:44722) 2024-01-31T05:36:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5db67fc640 1 -- 172.21.15.45:0/1902300626 learned_addr learned my addr 172.21.15.45:0/1902300626 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce060 0x55ba243d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5db77fe640 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5db67fc640 1 -- 172.21.15.45:0/1902300626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 msgr2=0x55ba243d5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5db67fc640 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243d5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5db67fc640 1 -- 172.21.15.45:0/1902300626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce060 msgr2=0x55ba243d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5db67fc640 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce060 0x55ba243d4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5db67fc640 1 -- 172.21.15.45:0/1902300626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db001b040 con 0x55ba243ce980 2024-01-31T05:36:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5db67fc640 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 0x55ba243d4c40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5dac012d50 tx=0x7f5dac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc3d3038f96cdd0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5d9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:36:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5d9aff5640 1 -- 172.21.15.45:0/1902300626 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac017070 con 0x55ba243ce980 2024-01-31T05:36:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5dc0218740 1 -- 172.21.15.45:0/1902300626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba2432f010 con 0x55ba243ce980 2024-01-31T05:36:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.784+0000 7f5d9aff5640 1 -- 172.21.15.45:0/1902300626 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5dac012e40 con 0x55ba243ce980 2024-01-31T05:36:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.785+0000 7f5d9aff5640 1 -- 172.21.15.45:0/1902300626 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dac0122e0 con 0x55ba243ce980 2024-01-31T05:36:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.785+0000 7f5dc0218740 10 client.16533 fetch_fsmap learned FSMap version 31 2024-01-31T05:36:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.785+0000 7f5dc0218740 1 -- 172.21.15.45:0/1902300626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ba243346a0 con 0x55ba243ce980 2024-01-31T05:36:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.785+0000 7f5d9aff5640 1 -- 172.21.15.45:0/1902300626 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dac028080 con 0x55ba243ce980 2024-01-31T05:36:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.785+0000 7f5d9aff5640 1 -- 172.21.15.45:0/1902300626 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5dac022030 con 0x55ba243ce980 2024-01-31T05:36:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.785+0000 7f5dc0218740 10 client.16533 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:36:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.785+0000 7f5dc0218740 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:36:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.785+0000 7f5dc0218740 20 client.16533 populate_metadata read hostname 'smithi045' 2024-01-31T05:36:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.785+0000 7f5dc0218740 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba243e7620 0x55ba243e9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.785+0000 7f5dc0218740 4 client.16533 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:36:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.785+0000 7f5dc0218740 1 -- 172.21.15.45:0/1902300626 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba2432ed30 con 0x55ba243e7620 2024-01-31T05:36:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.785+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba243e7620 0x55ba243e9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.786+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba243e7620 0x55ba243e9a40 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20da5632b9396718 server_cookie=7259f459960c2be9 in_seq=0 out_seq=0 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.786+0000 7f5d9aff5640 10 client.16533 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5d9aff5640 1 -- 172.21.15.45:0/1902300626 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba2432ed30 con 0x55ba243e7620 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5d9aff5640 10 client.16533 handle_command_reply: tid=0 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5dc0218740 1 client.16533 shutdown 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5dc0218740 1 -- 172.21.15.45:0/1902300626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 msgr2=0x55ba243d4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5dc0218740 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 0x55ba243d4c40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5dac012d50 tx=0x7f5dac00f790 comp rx=0 tx=0).stop 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5dc0218740 1 -- 172.21.15.45:0/1902300626 shutdown_connections 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5dc0218740 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba243e7620 0x55ba243e9a40 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5dc0218740 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 0x55ba243d4c40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5dc0218740 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243d5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5dc0218740 1 --2- 172.21.15.45:0/1902300626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce060 0x55ba243d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5dc0218740 1 -- 172.21.15.45:0/1902300626 >> 172.21.15.45:0/1902300626 conn(0x55ba243c8980 msgr2=0x55ba243ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5d9aff5640 0 client.16533 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5dc0218740 1 -- 172.21.15.45:0/1902300626 shutdown_connections 2024-01-31T05:36:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5dc0218740 1 -- 172.21.15.45:0/1902300626 wait complete. 2024-01-31T05:36:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.787+0000 7f5dc0218740 20 client.16533 trim_cache size 0 max 16384 2024-01-31T05:36:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.801+0000 7f5dc0218740 1 Processor -- start 2024-01-31T05:36:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.801+0000 7f5dc0218740 1 -- start start 2024-01-31T05:36:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.801+0000 7f5dc0218740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 0x55ba243d6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.801+0000 7f5dc0218740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243ce980 0x55ba243ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.801+0000 7f5dc0218740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243d7750 0x55ba243d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.801+0000 7f5dc0218740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba243cd340 con 0x55ba243ce060 2024-01-31T05:36:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.801+0000 7f5dc0218740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba243e4030 con 0x55ba243d7750 2024-01-31T05:36:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.801+0000 7f5dc0218740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba243c9980 con 0x55ba243ce980 2024-01-31T05:36:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5db6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 0x55ba243d6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5db6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 0x55ba243d6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44724/0 (socket says 172.21.15.45:44724) 2024-01-31T05:36:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5db77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243d7750 0x55ba243d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5db6ffd640 1 -- 172.21.15.45:0/3479128602 learned_addr learned my addr 172.21.15.45:0/3479128602 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5db67fc640 1 --2- 172.21.15.45:0/3479128602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243ce980 0x55ba243ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5db6ffd640 1 -- 172.21.15.45:0/3479128602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243ce980 msgr2=0x55ba243ee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/3479128602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243ce980 0x55ba243ee3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5db6ffd640 1 -- 172.21.15.45:0/3479128602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243d7750 msgr2=0x55ba243d2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/3479128602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243d7750 0x55ba243d2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5db6ffd640 1 -- 172.21.15.45:0/3479128602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba242d1030 con 0x55ba243ce060 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5db77fe640 1 --2- 172.21.15.45:0/3479128602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243d7750 0x55ba243d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/3479128602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 0x55ba243d6e30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5da80ece00 tx=0x7f5da80f19b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661b87a59a60976 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5d9cff9640 1 -- 172.21.15.45:0/3479128602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da80bcb50 con 0x55ba243ce060 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5d9cff9640 1 -- 172.21.15.45:0/3479128602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5da80e0e90 con 0x55ba243ce060 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.802+0000 7f5d9cff9640 1 -- 172.21.15.45:0/3479128602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da80b8b60 con 0x55ba243ce060 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.803+0000 7f5dc0218740 1 -- 172.21.15.45:0/3479128602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 msgr2=0x55ba243d6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.803+0000 7f5dc0218740 1 --2- 172.21.15.45:0/3479128602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 0x55ba243d6e30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5da80ece00 tx=0x7f5da80f19b0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.803+0000 7f5dc0218740 1 -- 172.21.15.45:0/3479128602 shutdown_connections 2024-01-31T05:36:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.803+0000 7f5dc0218740 1 --2- 172.21.15.45:0/3479128602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243d7750 0x55ba243d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.803+0000 7f5dc0218740 1 --2- 172.21.15.45:0/3479128602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243ce980 0x55ba243ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.803+0000 7f5dc0218740 1 --2- 172.21.15.45:0/3479128602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce060 0x55ba243d6e30 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.803+0000 7f5dc0218740 1 -- 172.21.15.45:0/3479128602 >> 172.21.15.45:0/3479128602 conn(0x55ba243c8980 msgr2=0x55ba243cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.803+0000 7f5dc0218740 1 -- 172.21.15.45:0/3479128602 shutdown_connections 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.803+0000 7f5dc0218740 1 -- 172.21.15.45:0/3479128602 wait complete. 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.803+0000 7f5dc0218740 1 -- start start 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.803+0000 7f5dc0218740 1 Processor -- start 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5dc0218740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce060 0x55ba243de4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5dc0218740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 0x55ba243dc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5dc0218740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243dcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5dc0218740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba243dd460 con 0x55ba243ce980 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5dc0218740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba243dd5d0 con 0x55ba243ce060 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5dc0218740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba243dd790 con 0x55ba243d7750 2024-01-31T05:36:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5dc0218740 10 client.0 _ll_register_callbacks cb 0x7f5db0009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 0x55ba243dc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce060 0x55ba243de4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce060 0x55ba243de4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:38930/0 (socket says 172.21.15.45:38930) 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 0x55ba243dc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44732/0 (socket says 172.21.15.45:44732) 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db6ffd640 1 -- 172.21.15.45:0/536321661 learned_addr learned my addr 172.21.15.45:0/536321661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db77fe640 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243dcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db6ffd640 1 -- 172.21.15.45:0/536321661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 msgr2=0x55ba243dcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243dcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db6ffd640 1 -- 172.21.15.45:0/536321661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 msgr2=0x55ba243dc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 0x55ba243dc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db6ffd640 1 -- 172.21.15.45:0/536321661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba2432de40 con 0x55ba243ce060 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db67fc640 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 0x55ba243dc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5db6ffd640 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce060 0x55ba243de4f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009430 tx=0x7f5da8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=755aeb4d72b26d86 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.804+0000 7f5d9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5d9aff5640 1 -- 172.21.15.45:0/536321661 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da80c05a0 con 0x55ba243ce060 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5d9aff5640 1 -- 172.21.15.45:0/536321661 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5da80d3030 con 0x55ba243ce060 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5dc0218740 1 -- 172.21.15.45:0/536321661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5db0003e00 con 0x55ba243ce060 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5d9aff5640 1 -- 172.21.15.45:0/536321661 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5da80bcb50 con 0x55ba243ce060 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5dc0218740 10 client.26030 fetch_fsmap learned FSMap version 31 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5dc0218740 1 -- 172.21.15.45:0/536321661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f5da8009560 con 0x55ba243ce060 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5d9aff5640 1 -- 172.21.15.45:0/536321661 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db0003e00 con 0x55ba243ce060 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5d9aff5640 1 -- 172.21.15.45:0/536321661 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5da803c070 con 0x55ba243ce060 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5dc0218740 10 client.26030 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5dc0218740 10 client.26030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5dc0218740 20 client.26030 populate_metadata read hostname 'smithi045' 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5dc0218740 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba2443ebb0 0x55ba24440fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5dc0218740 4 client.26030 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:36:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5dc0218740 1 -- 172.21.15.45:0/536321661 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ba24035600 con 0x55ba2443ebb0 2024-01-31T05:36:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.805+0000 7f5db67fc640 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba2443ebb0 0x55ba24440fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.806+0000 7f5d9aff5640 10 client.26030 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.806+0000 7f5db67fc640 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba2443ebb0 0x55ba24440fd0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b20fa72f7b40b7 server_cookie=a29980428d03ceb3 in_seq=0 out_seq=0 2024-01-31T05:36:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5d9aff5640 1 -- 172.21.15.45:0/536321661 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ba24035600 con 0x55ba2443ebb0 2024-01-31T05:36:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5d9aff5640 10 client.26030 handle_command_reply: tid=0 2024-01-31T05:36:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5dc0218740 1 client.26030 shutdown 2024-01-31T05:36:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5dc0218740 1 -- 172.21.15.45:0/536321661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce060 msgr2=0x55ba243de4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5dc0218740 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce060 0x55ba243de4f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009430 tx=0x7f5da8005fd0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5dc0218740 1 -- 172.21.15.45:0/536321661 shutdown_connections 2024-01-31T05:36:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5dc0218740 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ba2443ebb0 0x55ba24440fd0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5dc0218740 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243ce980 0x55ba243dc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5dc0218740 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba243d7750 0x55ba243dcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5d9aff5640 0 client.26030 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5dc0218740 1 --2- 172.21.15.45:0/536321661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba243ce060 0x55ba243de4f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5dc0218740 1 -- 172.21.15.45:0/536321661 >> 172.21.15.45:0/536321661 conn(0x55ba243c8980 msgr2=0x55ba243ee3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5dc0218740 1 -- 172.21.15.45:0/536321661 shutdown_connections 2024-01-31T05:36:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.849+0000 7f5dc0218740 1 -- 172.21.15.45:0/536321661 wait complete. 2024-01-31T05:36:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dc0218740 20 client.26030 trim_cache size 0 max 16384 2024-01-31T05:36:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:36:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:36:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:36:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:36:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dbd460640 1 -- 172.21.15.45:0/2146409442 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5da00a14f0 msgr2=0x7f5da00a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dbd460640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5da00a14f0 0x7f5da00a39b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5db0011cb0 tx=0x7f5db0015000 comp rx=0 tx=0).stop 2024-01-31T05:36:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dbd460640 1 -- 172.21.15.45:0/2146409442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db8100f40 msgr2=0x7f5db81993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dbd460640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db8100f40 0x7f5db81993d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5da8005940 tx=0x7f5da8005d50 comp rx=0 tx=0).stop 2024-01-31T05:36:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dbd460640 1 -- 172.21.15.45:0/2146409442 shutdown_connections 2024-01-31T05:36:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dbd460640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5da00a14f0 0x7f5da00a39b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dbd460640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5db819dc90 0x7f5db819e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dbd460640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db8106010 0x7f5db8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dbd460640 1 --2- 172.21.15.45:0/2146409442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db8100f40 0x7f5db81993d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dbd460640 1 -- 172.21.15.45:0/2146409442 >> 172.21.15.45:0/2146409442 conn(0x7f5db80fc7a0 msgr2=0x7f5db80fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dbd460640 1 -- 172.21.15.45:0/2146409442 shutdown_connections 2024-01-31T05:36:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:42.850+0000 7f5dbd460640 1 -- 172.21.15.45:0/2146409442 wait complete. 2024-01-31T05:36:43.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 19695 29.07 MB/sec warmup 42 sec latency 151.338 ms 2024-01-31T05:36:43.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:43 smithi156 ceph-mon[35720]: pgmap v446: 97 pgs: 97 active+clean; 887 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 24 MiB/s wr, 554 op/s 2024-01-31T05:36:44.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:43 smithi186 ceph-mon[35659]: pgmap v446: 97 pgs: 97 active+clean; 887 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 24 MiB/s wr, 554 op/s 2024-01-31T05:36:44.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:43 smithi045 ceph-mon[31585]: pgmap v446: 97 pgs: 97 active+clean; 887 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 24 MiB/s wr, 554 op/s 2024-01-31T05:36:44.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 20128 29.03 MB/sec warmup 43 sec latency 168.935 ms 2024-01-31T05:36:45.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 20735 29.32 MB/sec warmup 44 sec latency 178.145 ms 2024-01-31T05:36:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:45 smithi156 ceph-mon[35720]: pgmap v447: 97 pgs: 97 active+clean; 887 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 22 MiB/s wr, 379 op/s 2024-01-31T05:36:46.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:45 smithi186 ceph-mon[35659]: pgmap v447: 97 pgs: 97 active+clean; 887 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 22 MiB/s wr, 379 op/s 2024-01-31T05:36:46.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:45 smithi045 ceph-mon[31585]: pgmap v447: 97 pgs: 97 active+clean; 887 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 22 MiB/s wr, 379 op/s 2024-01-31T05:36:46.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 21203 29.50 MB/sec warmup 45 sec latency 228.592 ms 2024-01-31T05:36:47.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 21501 29.20 MB/sec warmup 46 sec latency 89.044 ms 2024-01-31T05:36:47.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:47 smithi156 ceph-mon[35720]: pgmap v448: 97 pgs: 97 active+clean; 813 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 24 MiB/s wr, 568 op/s 2024-01-31T05:36:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:47 smithi186 ceph-mon[35659]: pgmap v448: 97 pgs: 97 active+clean; 813 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 24 MiB/s wr, 568 op/s 2024-01-31T05:36:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:47 smithi045 ceph-mon[31585]: pgmap v448: 97 pgs: 97 active+clean; 813 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 24 MiB/s wr, 568 op/s 2024-01-31T05:36:48.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 21860 28.88 MB/sec warmup 47 sec latency 45.171 ms 2024-01-31T05:36:49.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 22076 28.46 MB/sec warmup 48 sec latency 345.363 ms 2024-01-31T05:36:49.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:49 smithi156 ceph-mon[35720]: pgmap v449: 97 pgs: 97 active+clean; 813 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.9 MiB/s wr, 388 op/s 2024-01-31T05:36:50.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:49 smithi186 ceph-mon[35659]: pgmap v449: 97 pgs: 97 active+clean; 813 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.9 MiB/s wr, 388 op/s 2024-01-31T05:36:50.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:49 smithi045 ceph-mon[31585]: pgmap v449: 97 pgs: 97 active+clean; 813 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.9 MiB/s wr, 388 op/s 2024-01-31T05:36:50.643 INFO:tasks.workunit.client.0.smithi045.stdout: 10 22455 28.25 MB/sec warmup 49 sec latency 291.132 ms 2024-01-31T05:36:51.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 22900 28.12 MB/sec warmup 50 sec latency 210.730 ms 2024-01-31T05:36:51.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:51 smithi156 ceph-mon[35720]: pgmap v450: 97 pgs: 97 active+clean; 813 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.9 MiB/s wr, 388 op/s 2024-01-31T05:36:52.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:51 smithi186 ceph-mon[35659]: pgmap v450: 97 pgs: 97 active+clean; 813 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.9 MiB/s wr, 388 op/s 2024-01-31T05:36:52.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:51 smithi045 ceph-mon[31585]: pgmap v450: 97 pgs: 97 active+clean; 813 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.9 MiB/s wr, 388 op/s 2024-01-31T05:36:52.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 23549 28.20 MB/sec warmup 51 sec latency 173.573 ms 2024-01-31T05:36:53.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 24320 28.21 MB/sec warmup 52 sec latency 151.895 ms 2024-01-31T05:36:53.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:53 smithi156 ceph-mon[35720]: pgmap v451: 97 pgs: 97 active+clean; 828 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 12 MiB/s wr, 598 op/s 2024-01-31T05:36:54.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:53 smithi186 ceph-mon[35659]: pgmap v451: 97 pgs: 97 active+clean; 828 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 12 MiB/s wr, 598 op/s 2024-01-31T05:36:54.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:53 smithi045 ceph-mon[31585]: pgmap v451: 97 pgs: 97 active+clean; 828 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 12 MiB/s wr, 598 op/s 2024-01-31T05:36:54.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 24770 28.41 MB/sec warmup 53 sec latency 364.572 ms 2024-01-31T05:36:55.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 25345 28.71 MB/sec warmup 54 sec latency 137.928 ms 2024-01-31T05:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.943+0000 7f2d24071640 1 -- 172.21.15.45:0/2508753854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d1c068370 msgr2=0x7f2d1c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.943+0000 7f2d24071640 1 --2- 172.21.15.45:0/2508753854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d1c068370 0x7f2d1c0687d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009900 tx=0x7f2d0c02bcd0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.943+0000 7f2d24071640 1 -- 172.21.15.45:0/2508753854 shutdown_connections 2024-01-31T05:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.943+0000 7f2d24071640 1 --2- 172.21.15.45:0/2508753854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d1c068370 0x7f2d1c0687d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.943+0000 7f2d24071640 1 --2- 172.21.15.45:0/2508753854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d1c068d10 0x7f2d1c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.943+0000 7f2d24071640 1 --2- 172.21.15.45:0/2508753854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d1c106de0 0x7f2d1c1071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.943+0000 7f2d24071640 1 -- 172.21.15.45:0/2508753854 >> 172.21.15.45:0/2508753854 conn(0x7f2d1c076a40 msgr2=0x7f2d1c076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.943+0000 7f2d24071640 1 -- 172.21.15.45:0/2508753854 shutdown_connections 2024-01-31T05:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.943+0000 7f2d24071640 1 -- 172.21.15.45:0/2508753854 wait complete. 2024-01-31T05:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.944+0000 7f2d24071640 1 Processor -- start 2024-01-31T05:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.944+0000 7f2d24071640 1 -- start start 2024-01-31T05:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.944+0000 7f2d24071640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d1c068370 0x7f2d1c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.944+0000 7f2d24071640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d1c068d10 0x7f2d1c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.944+0000 7f2d24071640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d1c106de0 0x7f2d1c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.944+0000 7f2d24071640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c19c050 con 0x7f2d1c068370 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.944+0000 7f2d24071640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c19c1c0 con 0x7f2d1c068d10 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.944+0000 7f2d24071640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c19c380 con 0x7f2d1c106de0 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d21e0d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d1c068370 0x7f2d1c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d2160c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d1c068d10 0x7f2d1c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d21e0d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d1c068370 0x7f2d1c1971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57016/0 (socket says 172.21.15.45:57016) 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d21e0d640 1 -- 172.21.15.45:0/3443808133 learned_addr learned my addr 172.21.15.45:0/3443808133 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d2260e640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d1c106de0 0x7f2d1c19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d2160c640 1 -- 172.21.15.45:0/3443808133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d1c106de0 msgr2=0x7f2d1c19ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d2160c640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d1c106de0 0x7f2d1c19ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d2160c640 1 -- 172.21.15.45:0/3443808133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d1c068370 msgr2=0x7f2d1c1971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d2160c640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d1c068370 0x7f2d1c1971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d2160c640 1 -- 172.21.15.45:0/3443808133 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d14009840 con 0x7f2d1c068d10 2024-01-31T05:36:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d21e0d640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d1c068370 0x7f2d1c1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d2160c640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d1c068d10 0x7f2d1c197720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c036040 tx=0x7f2d0c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.945+0000 7f2d12ffd640 1 -- 172.21.15.45:0/3443808133 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c02ca30 con 0x7f2d1c068d10 2024-01-31T05:36:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.946+0000 7f2d24071640 1 -- 172.21.15.45:0/3443808133 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c009560 con 0x7f2d1c068d10 2024-01-31T05:36:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.946+0000 7f2d12ffd640 1 -- 172.21.15.45:0/3443808133 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d0c0054c0 con 0x7f2d1c068d10 2024-01-31T05:36:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.946+0000 7f2d12ffd640 1 -- 172.21.15.45:0/3443808133 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c003a80 con 0x7f2d1c068d10 2024-01-31T05:36:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.946+0000 7f2d24071640 1 -- 172.21.15.45:0/3443808133 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c0ffe40 con 0x7f2d1c068d10 2024-01-31T05:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.948+0000 7f2d12ffd640 1 -- 172.21.15.45:0/3443808133 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f2d0c02cb90 con 0x7f2d1c068d10 2024-01-31T05:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d26e35740 1 Processor -- start 2024-01-31T05:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d26e35740 1 -- start start 2024-01-31T05:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d26e35740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f3346e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d26e35740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 0x559f33477210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d26e35740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f33477750 0x559f33479b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d26e35740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f333dd040 con 0x559f33477750 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d26e35740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f333da730 con 0x559f3346e980 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d26e35740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f333d6020 con 0x559f3346e060 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d2160c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f33477750 0x559f33479b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d12ffd640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2d080a14f0 0x7f2d080a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d2260e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f3346e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d2260e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f3346e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:55480/0 (socket says 172.21.15.45:55480) 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d2260e640 1 -- 172.21.15.45:0/4271563616 learned_addr learned my addr 172.21.15.45:0/4271563616 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d21e0d640 1 --2- 172.21.15.45:0/4271563616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 0x559f33477210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d2260e640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2d080a14f0 0x7f2d080a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.949+0000 7f2d12ffd640 1 -- 172.21.15.45:0/3443808133 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2d0c0eef50 con 0x7f2d1c068d10 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.950+0000 7f2d2160c640 1 -- 172.21.15.45:0/4271563616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 msgr2=0x559f3346e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.950+0000 7f2d2160c640 1 --2- 172.21.15.45:0/4271563616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f3346e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.950+0000 7f2d2160c640 1 -- 172.21.15.45:0/4271563616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 msgr2=0x559f33477210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.950+0000 7f2d2160c640 1 --2- 172.21.15.45:0/4271563616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 0x559f33477210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.950+0000 7f2d2160c640 1 -- 172.21.15.45:0/4271563616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f333cde40 con 0x559f33477750 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.950+0000 7f2d21e0d640 1 --2- 172.21.15.45:0/4271563616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 0x559f33477210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.950+0000 7f2d2260e640 1 --2- 172.21.15.45:0/4271563616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f3346e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.950+0000 7f2d2160c640 1 --2- 172.21.15.45:0/4271563616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f33477750 0x559f33479b40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0ec7b0 tx=0x7f2d0c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbf1d67b94b7cc0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.950+0000 7f2d07fff640 1 -- 172.21.15.45:0/4271563616 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c005630 con 0x559f33477750 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.950+0000 7f2d2260e640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2d080a14f0 0x7f2d080a39b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009fd0 tx=0x7f2d14004ab0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.950+0000 7f2d07fff640 1 -- 172.21.15.45:0/4271563616 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d0c0bc560 con 0x559f33477750 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.950+0000 7f2d07fff640 1 -- 172.21.15.45:0/4271563616 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0b8570 con 0x559f33477750 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.951+0000 7f2d26e35740 1 -- 172.21.15.45:0/4271563616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f33477750 msgr2=0x559f33479b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.951+0000 7f2d26e35740 1 --2- 172.21.15.45:0/4271563616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f33477750 0x559f33479b40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0ec7b0 tx=0x7f2d0c02bcd0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.951+0000 7f2d26e35740 1 -- 172.21.15.45:0/4271563616 shutdown_connections 2024-01-31T05:36:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.951+0000 7f2d26e35740 1 --2- 172.21.15.45:0/4271563616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f33477750 0x559f33479b40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.951+0000 7f2d26e35740 1 --2- 172.21.15.45:0/4271563616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 0x559f33477210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.951+0000 7f2d26e35740 1 --2- 172.21.15.45:0/4271563616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f3346e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.951+0000 7f2d26e35740 1 -- 172.21.15.45:0/4271563616 >> 172.21.15.45:0/4271563616 conn(0x559f33468980 msgr2=0x559f33468d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.951+0000 7f2d26e35740 1 -- 172.21.15.45:0/4271563616 shutdown_connections 2024-01-31T05:36:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.951+0000 7f2d26e35740 1 -- 172.21.15.45:0/4271563616 wait complete. 2024-01-31T05:36:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.951+0000 7f2d26e35740 1 -- start start 2024-01-31T05:36:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.951+0000 7f2d26e35740 1 Processor -- start 2024-01-31T05:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d26e35740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f33474710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d26e35740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 0x559f33474c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d26e35740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f33477750 0x559f33475190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d26e35740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3338f1b0 con 0x559f3346e980 2024-01-31T05:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d26e35740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3338ee50 con 0x559f33477750 2024-01-31T05:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d26e35740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f33354090 con 0x559f3346e060 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d26e35740 10 client.0 _ll_register_callbacks cb 0x559f33379a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d2160c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 0x559f33474c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d2160c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 0x559f33474c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57024/0 (socket says 172.21.15.45:57024) 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d2160c640 1 -- 172.21.15.45:0/3077326487 learned_addr learned my addr 172.21.15.45:0/3077326487 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d21e0d640 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f33474710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d2260e640 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f33477750 0x559f33475190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d2160c640 1 -- 172.21.15.45:0/3077326487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 msgr2=0x559f33474710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d2160c640 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f33474710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d2160c640 1 -- 172.21.15.45:0/3077326487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f33477750 msgr2=0x559f33475190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d2160c640 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f33477750 0x559f33475190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d2160c640 1 -- 172.21.15.45:0/3077326487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c0ffe40 con 0x559f3346e980 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d2160c640 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 0x559f33474c50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0ec860 tx=0x7f2d0c0b87a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77db57d1e3599be4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d057fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d057fa640 1 -- 172.21.15.45:0/3077326487 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c005630 con 0x559f3346e980 2024-01-31T05:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d057fa640 1 -- 172.21.15.45:0/3077326487 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2d0c0be040 con 0x559f3346e980 2024-01-31T05:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.952+0000 7f2d26e35740 1 -- 172.21.15.45:0/3077326487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f333cf010 con 0x559f3346e980 2024-01-31T05:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.953+0000 7f2d057fa640 1 -- 172.21.15.45:0/3077326487 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d0c0bbe60 con 0x559f3346e980 2024-01-31T05:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.953+0000 7f2d26e35740 10 client.16551 fetch_fsmap learned FSMap version 31 2024-01-31T05:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.953+0000 7f2d26e35740 1 -- 172.21.15.45:0/3077326487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559f333d46a0 con 0x559f3346e980 2024-01-31T05:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.953+0000 7f2d057fa640 1 -- 172.21.15.45:0/3077326487 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f333cf010 con 0x559f3346e980 2024-01-31T05:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.953+0000 7f2d057fa640 1 -- 172.21.15.45:0/3077326487 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2d0c03c070 con 0x559f3346e980 2024-01-31T05:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.953+0000 7f2d26e35740 10 client.16551 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.953+0000 7f2d26e35740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.953+0000 7f2d26e35740 20 client.16551 populate_metadata read hostname 'smithi045' 2024-01-31T05:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.953+0000 7f2d26e35740 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559f33486ef0 0x559f334892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.953+0000 7f2d26e35740 4 client.16551 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.953+0000 7f2d26e35740 1 -- 172.21.15.45:0/3077326487 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f333ced30 con 0x559f33486ef0 2024-01-31T05:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.953+0000 7f2d21e0d640 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559f33486ef0 0x559f334892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d21e0d640 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559f33486ef0 0x559f334892e0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a417370354d18 server_cookie=538f0cfcdb009471 in_seq=0 out_seq=0 2024-01-31T05:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d057fa640 10 client.16551 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d057fa640 1 -- 172.21.15.45:0/3077326487 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559f333ced30 con 0x559f33486ef0 2024-01-31T05:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d057fa640 10 client.16551 handle_command_reply: tid=0 2024-01-31T05:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d26e35740 1 client.16551 shutdown 2024-01-31T05:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d26e35740 1 -- 172.21.15.45:0/3077326487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 msgr2=0x559f33474c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d26e35740 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 0x559f33474c50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0ec860 tx=0x7f2d0c0b87a0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d26e35740 1 -- 172.21.15.45:0/3077326487 shutdown_connections 2024-01-31T05:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d26e35740 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559f33486ef0 0x559f334892e0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d26e35740 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 0x559f33474c50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d26e35740 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f33477750 0x559f33475190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d057fa640 0 client.16551 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.954+0000 7f2d26e35740 1 --2- 172.21.15.45:0/3077326487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f33474710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.955+0000 7f2d26e35740 1 -- 172.21.15.45:0/3077326487 >> 172.21.15.45:0/3077326487 conn(0x559f33468980 msgr2=0x559f3346a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.955+0000 7f2d26e35740 1 -- 172.21.15.45:0/3077326487 shutdown_connections 2024-01-31T05:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.955+0000 7f2d26e35740 1 -- 172.21.15.45:0/3077326487 wait complete. 2024-01-31T05:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.955+0000 7f2d26e35740 20 client.16551 trim_cache size 0 max 16384 2024-01-31T05:36:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.976+0000 7f2d26e35740 1 Processor -- start 2024-01-31T05:36:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.976+0000 7f2d26e35740 1 -- start start 2024-01-31T05:36:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d26e35740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f33476e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d26e35740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 0x559f33474fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d26e35740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f33477750 0x559f334719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d26e35740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f333cf010 con 0x559f33477750 2024-01-31T05:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d26e35740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f33462f00 con 0x559f3346e980 2024-01-31T05:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d26e35740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f33469980 con 0x559f3346e060 2024-01-31T05:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d2160c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 0x559f33474fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d2160c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 0x559f33474fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:53358/0 (socket says 172.21.15.45:53358) 2024-01-31T05:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d2160c640 1 -- 172.21.15.45:0/3793773186 learned_addr learned my addr 172.21.15.45:0/3793773186 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d2260e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f33477750 0x559f334719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d2160c640 1 -- 172.21.15.45:0/3793773186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 msgr2=0x559f33476e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d2160c640 1 --2- 172.21.15.45:0/3793773186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f33476e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d2160c640 1 -- 172.21.15.45:0/3793773186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f33477750 msgr2=0x559f334719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d2160c640 1 --2- 172.21.15.45:0/3793773186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f33477750 0x559f334719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.977+0000 7f2d2160c640 1 -- 172.21.15.45:0/3793773186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f33371030 con 0x559f3346e980 2024-01-31T05:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.978+0000 7f2d2160c640 1 --2- 172.21.15.45:0/3793773186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 0x559f33474fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0bbdf0 tx=0x7f2d0c02bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d360f958b9a4080a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:55.979 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:55 smithi156 ceph-mon[35720]: pgmap v452: 97 pgs: 97 active+clean; 828 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.0 MiB/s wr, 399 op/s 2024-01-31T05:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.978+0000 7f2d077fe640 1 -- 172.21.15.45:0/3793773186 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c005630 con 0x559f3346e980 2024-01-31T05:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.978+0000 7f2d077fe640 1 -- 172.21.15.45:0/3793773186 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d0c0bbe60 con 0x559f3346e980 2024-01-31T05:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.978+0000 7f2d077fe640 1 -- 172.21.15.45:0/3793773186 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0bb350 con 0x559f3346e980 2024-01-31T05:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.979+0000 7f2d26e35740 1 -- 172.21.15.45:0/3793773186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 msgr2=0x559f33474fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.979+0000 7f2d26e35740 1 --2- 172.21.15.45:0/3793773186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 0x559f33474fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0bbdf0 tx=0x7f2d0c02bcd0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.980+0000 7f2d26e35740 1 -- 172.21.15.45:0/3793773186 shutdown_connections 2024-01-31T05:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.980+0000 7f2d26e35740 1 --2- 172.21.15.45:0/3793773186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f33477750 0x559f334719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.980+0000 7f2d26e35740 1 --2- 172.21.15.45:0/3793773186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e980 0x559f33474fb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.980+0000 7f2d26e35740 1 --2- 172.21.15.45:0/3793773186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f3346e060 0x559f33476e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.980+0000 7f2d26e35740 1 -- 172.21.15.45:0/3793773186 >> 172.21.15.45:0/3793773186 conn(0x559f33468980 msgr2=0x559f33473000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.980+0000 7f2d26e35740 1 -- 172.21.15.45:0/3793773186 shutdown_connections 2024-01-31T05:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.980+0000 7f2d26e35740 1 -- 172.21.15.45:0/3793773186 wait complete. 2024-01-31T05:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.980+0000 7f2d26e35740 1 -- start start 2024-01-31T05:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.980+0000 7f2d26e35740 1 Processor -- start 2024-01-31T05:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d26e35740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e060 0x559f3347e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d26e35740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 0x559f3347c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d26e35740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f33477750 0x559f3347cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d26e35740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3347d280 con 0x559f3346e980 2024-01-31T05:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d26e35740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3347d3f0 con 0x559f3346e060 2024-01-31T05:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d26e35740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f334dd3d0 con 0x559f33477750 2024-01-31T05:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d26e35740 10 client.0 _ll_register_callbacks cb 0x7f2d0c0bbc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d21e0d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e060 0x559f3347e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d21e0d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e060 0x559f3347e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:53370/0 (socket says 172.21.15.45:53370) 2024-01-31T05:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d21e0d640 1 -- 172.21.15.45:0/1116662358 learned_addr learned my addr 172.21.15.45:0/1116662358 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d2160c640 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 0x559f3347c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d2260e640 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f33477750 0x559f3347cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.981+0000 7f2d21e0d640 1 -- 172.21.15.45:0/1116662358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f33477750 msgr2=0x559f3347cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d21e0d640 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f33477750 0x559f3347cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d21e0d640 1 -- 172.21.15.45:0/1116662358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 msgr2=0x559f3347c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d21e0d640 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 0x559f3347c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d21e0d640 1 -- 172.21.15.45:0/1116662358 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f333d46a0 con 0x559f3346e060 2024-01-31T05:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d2260e640 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f33477750 0x559f3347cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d2160c640 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 0x559f3347c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:36:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d21e0d640 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e060 0x559f3347e1b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d18012010 tx=0x7f2d18022ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70a7732a4057c596 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:36:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d057fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:36:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d057fa640 1 -- 172.21.15.45:0/1116662358 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18015040 con 0x559f3346e060 2024-01-31T05:36:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d057fa640 1 -- 172.21.15.45:0/1116662358 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2d1801cac0 con 0x559f3346e060 2024-01-31T05:36:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d26e35740 1 -- 172.21.15.45:0/1116662358 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d0c0bc560 con 0x559f3346e060 2024-01-31T05:36:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d057fa640 1 -- 172.21.15.45:0/1116662358 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d1801cc80 con 0x559f3346e060 2024-01-31T05:36:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d26e35740 10 client.26048 fetch_fsmap learned FSMap version 31 2024-01-31T05:36:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.982+0000 7f2d26e35740 1 -- 172.21.15.45:0/1116662358 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559f33396a50 con 0x559f3346e060 2024-01-31T05:36:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.983+0000 7f2d057fa640 1 -- 172.21.15.45:0/1116662358 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d0c0bc560 con 0x559f3346e060 2024-01-31T05:36:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.983+0000 7f2d057fa640 1 -- 172.21.15.45:0/1116662358 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2d18005460 con 0x559f3346e060 2024-01-31T05:36:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.983+0000 7f2d26e35740 10 client.26048 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:36:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.983+0000 7f2d26e35740 10 client.26048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:36:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.983+0000 7f2d26e35740 20 client.26048 populate_metadata read hostname 'smithi045' 2024-01-31T05:36:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.983+0000 7f2d26e35740 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559f334deb10 0x559f334e0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:36:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.983+0000 7f2d26e35740 4 client.26048 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:36:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.983+0000 7f2d26e35740 1 -- 172.21.15.45:0/1116662358 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559f330d5600 con 0x559f334deb10 2024-01-31T05:36:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.983+0000 7f2d2160c640 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559f334deb10 0x559f334e0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:36:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.984+0000 7f2d057fa640 10 client.26048 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:55.984+0000 7f2d2160c640 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559f334deb10 0x559f334e0f30 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a895c391622428 server_cookie=da69893a23be0af0 in_seq=0 out_seq=0 2024-01-31T05:36:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d057fa640 1 -- 172.21.15.45:0/1116662358 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559f330d5600 con 0x559f334deb10 2024-01-31T05:36:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d057fa640 10 client.26048 handle_command_reply: tid=0 2024-01-31T05:36:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d26e35740 1 client.26048 shutdown 2024-01-31T05:36:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d26e35740 1 -- 172.21.15.45:0/1116662358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e060 msgr2=0x559f3347e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d26e35740 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e060 0x559f3347e1b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d18012010 tx=0x7f2d18022ed0 comp rx=0 tx=0).stop 2024-01-31T05:36:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d26e35740 1 -- 172.21.15.45:0/1116662358 shutdown_connections 2024-01-31T05:36:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d26e35740 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559f334deb10 0x559f334e0f30 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d26e35740 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3346e980 0x559f3347c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d057fa640 0 client.26048 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:36:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d26e35740 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f33477750 0x559f3347cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d26e35740 1 --2- 172.21.15.45:0/1116662358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f3346e060 0x559f3347e1b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d26e35740 1 -- 172.21.15.45:0/1116662358 >> 172.21.15.45:0/1116662358 conn(0x559f33468980 msgr2=0x559f33469cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d26e35740 1 -- 172.21.15.45:0/1116662358 shutdown_connections 2024-01-31T05:36:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.028+0000 7f2d26e35740 1 -- 172.21.15.45:0/1116662358 wait complete. 2024-01-31T05:36:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d26e35740 20 client.26048 trim_cache size 0 max 16384 2024-01-31T05:36:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:36:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:36:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:36:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:36:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d24071640 1 -- 172.21.15.45:0/3443808133 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2d080a14f0 msgr2=0x7f2d080a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d24071640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2d080a14f0 0x7f2d080a39b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009fd0 tx=0x7f2d14004ab0 comp rx=0 tx=0).stop 2024-01-31T05:36:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d24071640 1 -- 172.21.15.45:0/3443808133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d1c068d10 msgr2=0x7f2d1c197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:36:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d24071640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d1c068d10 0x7f2d1c197720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c036040 tx=0x7f2d0c0045e0 comp rx=0 tx=0).stop 2024-01-31T05:36:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d24071640 1 -- 172.21.15.45:0/3443808133 shutdown_connections 2024-01-31T05:36:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d24071640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f2d080a14f0 0x7f2d080a39b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d24071640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d1c106de0 0x7f2d1c19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d24071640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d1c068d10 0x7f2d1c197720 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d24071640 1 --2- 172.21.15.45:0/3443808133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d1c068370 0x7f2d1c1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:36:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d24071640 1 -- 172.21.15.45:0/3443808133 >> 172.21.15.45:0/3443808133 conn(0x7f2d1c076a40 msgr2=0x7f2d1c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:36:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d24071640 1 -- 172.21.15.45:0/3443808133 shutdown_connections 2024-01-31T05:36:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:36:56.029+0000 7f2d24071640 1 -- 172.21.15.45:0/3443808133 wait complete. 2024-01-31T05:36:56.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:55 smithi186 ceph-mon[35659]: pgmap v452: 97 pgs: 97 active+clean; 828 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.0 MiB/s wr, 399 op/s 2024-01-31T05:36:56.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:55 smithi045 ceph-mon[31585]: pgmap v452: 97 pgs: 97 active+clean; 828 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.0 MiB/s wr, 399 op/s 2024-01-31T05:36:56.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 25697 28.54 MB/sec warmup 55 sec latency 70.473 ms 2024-01-31T05:36:57.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 26063 28.39 MB/sec warmup 56 sec latency 184.124 ms 2024-01-31T05:36:57.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:57 smithi156 ceph-mon[35720]: pgmap v453: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 600 op/s 2024-01-31T05:36:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:57 smithi186 ceph-mon[35659]: pgmap v453: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 600 op/s 2024-01-31T05:36:58.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:57 smithi045 ceph-mon[31585]: pgmap v453: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 600 op/s 2024-01-31T05:36:58.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 26385 28.05 MB/sec warmup 57 sec latency 59.484 ms 2024-01-31T05:36:59.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 26705 27.81 MB/sec warmup 58 sec latency 250.616 ms 2024-01-31T05:36:59.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:36:59 smithi156 ceph-mon[35720]: pgmap v454: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.4 MiB/s wr, 411 op/s 2024-01-31T05:37:00.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:36:59 smithi186 ceph-mon[35659]: pgmap v454: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.4 MiB/s wr, 411 op/s 2024-01-31T05:37:00.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:36:59 smithi045 ceph-mon[31585]: pgmap v454: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.4 MiB/s wr, 411 op/s 2024-01-31T05:37:00.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 26986 27.54 MB/sec warmup 59 sec latency 299.359 ms 2024-01-31T05:37:01.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 26986 27.08 MB/sec warmup 60 sec latency 1299.478 ms 2024-01-31T05:37:01.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:01 smithi156 ceph-mon[35720]: pgmap v455: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.4 MiB/s wr, 411 op/s 2024-01-31T05:37:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:01 smithi186 ceph-mon[35659]: pgmap v455: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.4 MiB/s wr, 411 op/s 2024-01-31T05:37:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:01 smithi045 ceph-mon[31585]: pgmap v455: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.4 MiB/s wr, 411 op/s 2024-01-31T05:37:02.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 26986 26.64 MB/sec warmup 61 sec latency 2299.536 ms 2024-01-31T05:37:02.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:02 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:37:02.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:02 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:37:02.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:02 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:37:02.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:02 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:37:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:02 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:37:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:02 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:37:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:02 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:37:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:02 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:37:03.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:02 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:37:03.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:02 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:37:03.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:02 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:37:03.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:02 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:37:03.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 26986 26.21 MB/sec warmup 62 sec latency 3299.623 ms 2024-01-31T05:37:03.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:03 smithi156 ceph-mon[35720]: pgmap v456: 97 pgs: 97 active+clean; 848 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.6 MiB/s wr, 556 op/s 2024-01-31T05:37:04.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:03 smithi186 ceph-mon[35659]: pgmap v456: 97 pgs: 97 active+clean; 848 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.6 MiB/s wr, 556 op/s 2024-01-31T05:37:04.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:03 smithi045 ceph-mon[31585]: pgmap v456: 97 pgs: 97 active+clean; 848 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.6 MiB/s wr, 556 op/s 2024-01-31T05:37:04.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 26986 25.79 MB/sec warmup 63 sec latency 4299.721 ms 2024-01-31T05:37:05.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 26999 25.39 MB/sec warmup 64 sec latency 5299.824 ms 2024-01-31T05:37:05.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:05 smithi156 ceph-mon[35720]: pgmap v457: 97 pgs: 97 active+clean; 848 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 346 op/s 2024-01-31T05:37:06.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:05 smithi186 ceph-mon[35659]: pgmap v457: 97 pgs: 97 active+clean; 848 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 346 op/s 2024-01-31T05:37:06.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:05 smithi045 ceph-mon[31585]: pgmap v457: 97 pgs: 97 active+clean; 848 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 346 op/s 2024-01-31T05:37:06.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 27152 25.10 MB/sec warmup 65 sec latency 6022.920 ms 2024-01-31T05:37:07.644 INFO:tasks.workunit.client.0.smithi045.stdout: 10 27603 25.08 MB/sec warmup 66 sec latency 157.388 ms 2024-01-31T05:37:07.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:07 smithi156 ceph-mon[35720]: pgmap v458: 97 pgs: 97 active+clean; 851 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.3 MiB/s wr, 598 op/s 2024-01-31T05:37:08.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:07 smithi186 ceph-mon[35659]: pgmap v458: 97 pgs: 97 active+clean; 851 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.3 MiB/s wr, 598 op/s 2024-01-31T05:37:08.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:07 smithi045 ceph-mon[31585]: pgmap v458: 97 pgs: 97 active+clean; 851 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.3 MiB/s wr, 598 op/s 2024-01-31T05:37:08.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 28169 25.06 MB/sec warmup 67 sec latency 192.518 ms 2024-01-31T05:37:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.134+0000 7f1950699640 1 -- 172.21.15.45:0/2789763769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19480ffae0 msgr2=0x7f19480ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.134+0000 7f1950699640 1 --2- 172.21.15.45:0/2789763769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19480ffae0 0x7f19480ffec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1938013080 tx=0x7f1938033f10 comp rx=0 tx=0).stop 2024-01-31T05:37:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.134+0000 7f1950699640 1 -- 172.21.15.45:0/2789763769 shutdown_connections 2024-01-31T05:37:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.134+0000 7f1950699640 1 --2- 172.21.15.45:0/2789763769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1948100400 0x7f194810d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.134+0000 7f1950699640 1 --2- 172.21.15.45:0/2789763769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f194810d690 0x7f194810fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.134+0000 7f1950699640 1 --2- 172.21.15.45:0/2789763769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19480ffae0 0x7f19480ffec0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.134+0000 7f1950699640 1 -- 172.21.15.45:0/2789763769 >> 172.21.15.45:0/2789763769 conn(0x7f1948076d10 msgr2=0x7f1948077120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.135+0000 7f1950699640 1 -- 172.21.15.45:0/2789763769 shutdown_connections 2024-01-31T05:37:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.135+0000 7f1950699640 1 -- 172.21.15.45:0/2789763769 wait complete. 2024-01-31T05:37:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.135+0000 7f1950699640 1 Processor -- start 2024-01-31T05:37:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.135+0000 7f1950699640 1 -- start start 2024-01-31T05:37:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.136+0000 7f1950699640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19480ffae0 0x7f1948104f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.136+0000 7f1950699640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1948100400 0x7f1948105490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.136+0000 7f1950699640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f194810d690 0x7f19481092e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.136+0000 7f1950699640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1948109820 con 0x7f19480ffae0 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.136+0000 7f1950699640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1948109990 con 0x7f194810d690 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.136+0000 7f1950699640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1948109b50 con 0x7f1948100400 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.136+0000 7f194dc34640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1948100400 0x7f1948105490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.136+0000 7f194dc34640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1948100400 0x7f1948105490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:37170/0 (socket says 172.21.15.45:37170) 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.136+0000 7f194dc34640 1 -- 172.21.15.45:0/2601927062 learned_addr learned my addr 172.21.15.45:0/2601927062 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.137+0000 7f194ec36640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f194810d690 0x7f19481092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.137+0000 7f194e435640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19480ffae0 0x7f1948104f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.137+0000 7f194dc34640 1 -- 172.21.15.45:0/2601927062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f194810d690 msgr2=0x7f19481092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.137+0000 7f194dc34640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f194810d690 0x7f19481092e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.137+0000 7f194dc34640 1 -- 172.21.15.45:0/2601927062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19480ffae0 msgr2=0x7f1948104f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.137+0000 7f194dc34640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19480ffae0 0x7f1948104f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.137+0000 7f194dc34640 1 -- 172.21.15.45:0/2601927062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1938011ca0 con 0x7f1948100400 2024-01-31T05:37:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.137+0000 7f194ec36640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f194810d690 0x7f19481092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:37:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.137+0000 7f194e435640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19480ffae0 0x7f1948104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:37:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.137+0000 7f194dc34640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1948100400 0x7f1948105490 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f194400b450 tx=0x7f194400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.138+0000 7f193f7fe640 1 -- 172.21.15.45:0/2601927062 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944011890 con 0x7f1948100400 2024-01-31T05:37:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.138+0000 7f193f7fe640 1 -- 172.21.15.45:0/2601927062 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19440103e0 con 0x7f1948100400 2024-01-31T05:37:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.138+0000 7f193f7fe640 1 -- 172.21.15.45:0/2601927062 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194400f5a0 con 0x7f1948100400 2024-01-31T05:37:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.138+0000 7f1950699640 1 -- 172.21.15.45:0/2601927062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1948109e30 con 0x7f1948100400 2024-01-31T05:37:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.138+0000 7f1950699640 1 -- 172.21.15.45:0/2601927062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194810a160 con 0x7f1948100400 2024-01-31T05:37:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.141+0000 7f193f7fe640 1 -- 172.21.15.45:0/2601927062 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f1944010550 con 0x7f1948100400 2024-01-31T05:37:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.141+0000 7f1953457740 1 Processor -- start 2024-01-31T05:37:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.141+0000 7f1953457740 1 -- start start 2024-01-31T05:37:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f193f7fe640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f19340a1570 0x7f19340a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194ec36640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f19340a1570 0x7f19340a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f1953457740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde060 0x556666cde440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f193f7fe640 1 -- 172.21.15.45:0/2601927062 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f19440ccb00 con 0x7f1948100400 2024-01-31T05:37:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f1953457740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde980 0x556666ce7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f1953457740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666ce9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f1953457740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666c4d040 con 0x556666cde060 2024-01-31T05:37:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f1953457740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666c4a730 con 0x556666ce7750 2024-01-31T05:37:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f1953457740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666c46020 con 0x556666cde980 2024-01-31T05:37:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194e435640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde060 0x556666cde440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194e435640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde060 0x556666cde440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43010/0 (socket says 172.21.15.45:43010) 2024-01-31T05:37:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194e435640 1 -- 172.21.15.45:0/2529265619 learned_addr learned my addr 172.21.15.45:0/2529265619 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194dc34640 1 --2- 172.21.15.45:0/2529265619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde980 0x556666ce7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194ec36640 1 --2- 172.21.15.45:0/2529265619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666ce9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194ec36640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f19340a1570 0x7f19340a3a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1940005e30 tx=0x7f194000f7d0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194e435640 1 -- 172.21.15.45:0/2529265619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde980 msgr2=0x556666ce7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194e435640 1 --2- 172.21.15.45:0/2529265619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde980 0x556666ce7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194e435640 1 -- 172.21.15.45:0/2529265619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 msgr2=0x556666ce9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194e435640 1 --2- 172.21.15.45:0/2529265619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666ce9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194e435640 1 -- 172.21.15.45:0/2529265619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556666c3de40 con 0x556666cde060 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.142+0000 7f194e435640 1 --2- 172.21.15.45:0/2529265619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde060 0x556666cde440 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1938013050 tx=0x7f1938011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c365d6b618528b87 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.143+0000 7f193cff9640 1 -- 172.21.15.45:0/2529265619 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938010d70 con 0x556666cde060 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.143+0000 7f193cff9640 1 -- 172.21.15.45:0/2529265619 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1938010ed0 con 0x556666cde060 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.143+0000 7f193cff9640 1 -- 172.21.15.45:0/2529265619 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19380042a0 con 0x556666cde060 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.143+0000 7f1953457740 1 -- 172.21.15.45:0/2529265619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde060 msgr2=0x556666cde440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.143+0000 7f1953457740 1 --2- 172.21.15.45:0/2529265619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde060 0x556666cde440 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1938013050 tx=0x7f1938011960 comp rx=0 tx=0).stop 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.143+0000 7f1953457740 1 -- 172.21.15.45:0/2529265619 shutdown_connections 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.143+0000 7f1953457740 1 --2- 172.21.15.45:0/2529265619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666ce9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.143+0000 7f1953457740 1 --2- 172.21.15.45:0/2529265619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde980 0x556666ce7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.143+0000 7f1953457740 1 --2- 172.21.15.45:0/2529265619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde060 0x556666cde440 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.143+0000 7f1953457740 1 -- 172.21.15.45:0/2529265619 >> 172.21.15.45:0/2529265619 conn(0x556666cd8980 msgr2=0x556666cd8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.144+0000 7f1953457740 1 -- 172.21.15.45:0/2529265619 shutdown_connections 2024-01-31T05:37:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.144+0000 7f1953457740 1 -- 172.21.15.45:0/2529265619 wait complete. 2024-01-31T05:37:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.144+0000 7f1953457740 1 -- start start 2024-01-31T05:37:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.144+0000 7f1953457740 1 Processor -- start 2024-01-31T05:37:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.144+0000 7f1953457740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666ce46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.144+0000 7f1953457740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde980 0x556666ce4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.144+0000 7f1953457740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666ce5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.144+0000 7f1953457740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666bff1b0 con 0x556666cde980 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.144+0000 7f1953457740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666bfee50 con 0x556666ce7750 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.144+0000 7f1953457740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666bc4090 con 0x556666cde060 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.144+0000 7f1953457740 10 client.0 _ll_register_callbacks cb 0x556666be9a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194e435640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666ce46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194ec36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666ce5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194e435640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666ce46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:37190/0 (socket says 172.21.15.45:37190) 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194ec36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666ce5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:44320/0 (socket says 172.21.15.45:44320) 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194e435640 1 -- 172.21.15.45:0/4067444171 learned_addr learned my addr 172.21.15.45:0/4067444171 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194dc34640 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde980 0x556666ce4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194e435640 1 -- 172.21.15.45:0/4067444171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 msgr2=0x556666ce5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194e435640 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666ce5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194e435640 1 -- 172.21.15.45:0/4067444171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde980 msgr2=0x556666ce4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194e435640 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde980 0x556666ce4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194e435640 1 -- 172.21.15.45:0/4067444171 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1948110610 con 0x556666cde060 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194ec36640 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666ce5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f194e435640 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666ce46a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1938034520 tx=0x7f1938034690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=106b90ed4a41bfa6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f1931ffb640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f1931ffb640 1 -- 172.21.15.45:0/4067444171 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19380110d0 con 0x556666cde060 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f1931ffb640 1 -- 172.21.15.45:0/4067444171 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1938034ac0 con 0x556666cde060 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.145+0000 7f1953457740 1 -- 172.21.15.45:0/4067444171 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556666c3f010 con 0x556666cde060 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.146+0000 7f1931ffb640 1 -- 172.21.15.45:0/4067444171 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1938004040 con 0x556666cde060 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.146+0000 7f1953457740 10 client.26128 fetch_fsmap learned FSMap version 31 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.146+0000 7f1953457740 1 -- 172.21.15.45:0/4067444171 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556666c446a0 con 0x556666cde060 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.146+0000 7f1931ffb640 1 -- 172.21.15.45:0/4067444171 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556666c3f010 con 0x556666cde060 2024-01-31T05:37:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.146+0000 7f1931ffb640 1 -- 172.21.15.45:0/4067444171 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f193800b070 con 0x556666cde060 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.146+0000 7f1953457740 10 client.26128 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.146+0000 7f1953457740 10 client.26128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.146+0000 7f1953457740 20 client.26128 populate_metadata read hostname 'smithi045' 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.146+0000 7f1953457740 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556666cf7230 0x556666cf9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.146+0000 7f1953457740 4 client.26128 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.146+0000 7f1953457740 1 -- 172.21.15.45:0/4067444171 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556666c3ed30 con 0x556666cf7230 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.146+0000 7f194dc34640 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556666cf7230 0x556666cf9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1931ffb640 10 client.26128 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f194dc34640 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556666cf7230 0x556666cf9620 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de8b95dbc210f81 server_cookie=5d191193bd3f36e0 in_seq=0 out_seq=0 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1931ffb640 1 -- 172.21.15.45:0/4067444171 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556666c3ed30 con 0x556666cf7230 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1931ffb640 10 client.26128 handle_command_reply: tid=0 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1953457740 1 client.26128 shutdown 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1953457740 1 -- 172.21.15.45:0/4067444171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 msgr2=0x556666ce46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1953457740 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666ce46a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1938034520 tx=0x7f1938034690 comp rx=0 tx=0).stop 2024-01-31T05:37:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1953457740 1 -- 172.21.15.45:0/4067444171 shutdown_connections 2024-01-31T05:37:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1953457740 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556666cf7230 0x556666cf9620 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1953457740 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde980 0x556666ce4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1953457740 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666ce5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1953457740 1 --2- 172.21.15.45:0/4067444171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666ce46a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1931ffb640 0 client.26128 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.147+0000 7f1953457740 1 -- 172.21.15.45:0/4067444171 >> 172.21.15.45:0/4067444171 conn(0x556666cd8980 msgr2=0x556666cda6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.148+0000 7f1953457740 1 -- 172.21.15.45:0/4067444171 shutdown_connections 2024-01-31T05:37:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.148+0000 7f1953457740 1 -- 172.21.15.45:0/4067444171 wait complete. 2024-01-31T05:37:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.148+0000 7f1953457740 20 client.26128 trim_cache size 0 max 16384 2024-01-31T05:37:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f1953457740 1 Processor -- start 2024-01-31T05:37:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f1953457740 1 -- start start 2024-01-31T05:37:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f1953457740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666ce6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f1953457740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde980 0x556666cee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f1953457740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666cee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f1953457740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666c3f010 con 0x556666cde980 2024-01-31T05:37:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f1953457740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666cd2f20 con 0x556666ce7750 2024-01-31T05:37:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f1953457740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666cd9980 con 0x556666cde060 2024-01-31T05:37:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f194dc34640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde980 0x556666cee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f194dc34640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde980 0x556666cee3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43040/0 (socket says 172.21.15.45:43040) 2024-01-31T05:37:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f194e435640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666ce6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f194dc34640 1 -- 172.21.15.45:0/679342580 learned_addr learned my addr 172.21.15.45:0/679342580 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f194ec36640 1 --2- 172.21.15.45:0/679342580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666cee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f194e435640 1 -- 172.21.15.45:0/679342580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 msgr2=0x556666cee920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f194e435640 1 --2- 172.21.15.45:0/679342580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666cee920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f194e435640 1 -- 172.21.15.45:0/679342580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde980 msgr2=0x556666cee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f194e435640 1 --2- 172.21.15.45:0/679342580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde980 0x556666cee3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.163+0000 7f194e435640 1 -- 172.21.15.45:0/679342580 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556666be1030 con 0x556666cde060 2024-01-31T05:37:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.164+0000 7f194e435640 1 --2- 172.21.15.45:0/679342580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666ce6dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19380116c0 tx=0x7f1938014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6ddd135de30eaf8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.164+0000 7f1933fff640 1 -- 172.21.15.45:0/679342580 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193803c700 con 0x556666cde060 2024-01-31T05:37:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.164+0000 7f1933fff640 1 -- 172.21.15.45:0/679342580 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19380110d0 con 0x556666cde060 2024-01-31T05:37:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.164+0000 7f1933fff640 1 -- 172.21.15.45:0/679342580 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19380044a0 con 0x556666cde060 2024-01-31T05:37:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.164+0000 7f1953457740 1 -- 172.21.15.45:0/679342580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 msgr2=0x556666ce6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 --2- 172.21.15.45:0/679342580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666ce6dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19380116c0 tx=0x7f1938014000 comp rx=0 tx=0).stop 2024-01-31T05:37:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 -- 172.21.15.45:0/679342580 shutdown_connections 2024-01-31T05:37:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 --2- 172.21.15.45:0/679342580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666cde980 0x556666cee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 --2- 172.21.15.45:0/679342580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666ce7750 0x556666cee920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 --2- 172.21.15.45:0/679342580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666ce6dd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 -- 172.21.15.45:0/679342580 >> 172.21.15.45:0/679342580 conn(0x556666cd8980 msgr2=0x556666cfdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 -- 172.21.15.45:0/679342580 shutdown_connections 2024-01-31T05:37:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 -- 172.21.15.45:0/679342580 wait complete. 2024-01-31T05:37:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 -- start start 2024-01-31T05:37:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 Processor -- start 2024-01-31T05:37:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666d46120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666cde980 0x556666d46660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666ce7750 0x556666cedd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666cee2f0 con 0x556666ce7750 2024-01-31T05:37:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666cee460 con 0x556666cde980 2024-01-31T05:37:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666cee620 con 0x556666cde060 2024-01-31T05:37:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f1953457740 10 client.0 _ll_register_callbacks cb 0x7f1938011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:37:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f194ec36640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666ce7750 0x556666cedd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f194e435640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666d46120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f194ec36640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666ce7750 0x556666cedd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43056/0 (socket says 172.21.15.45:43056) 2024-01-31T05:37:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f194e435640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666d46120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:37216/0 (socket says 172.21.15.45:37216) 2024-01-31T05:37:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f194ec36640 1 -- 172.21.15.45:0/518095971 learned_addr learned my addr 172.21.15.45:0/518095971 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.165+0000 7f194dc34640 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666cde980 0x556666d46660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f194e435640 1 -- 172.21.15.45:0/518095971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666cde980 msgr2=0x556666d46660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f194e435640 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666cde980 0x556666d46660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f194e435640 1 -- 172.21.15.45:0/518095971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666ce7750 msgr2=0x556666cedd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f194e435640 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666ce7750 0x556666cedd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f194e435640 1 -- 172.21.15.45:0/518095971 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556666c446a0 con 0x556666cde060 2024-01-31T05:37:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f194ec36640 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666ce7750 0x556666cedd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:37:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f194dc34640 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666cde980 0x556666d46660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:37:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f194e435640 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666d46120 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1938011040 tx=0x7f1938011240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20eadf99b450d3ef server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f1931ffb640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:37:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f1931ffb640 1 -- 172.21.15.45:0/518095971 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193805f070 con 0x556666cde060 2024-01-31T05:37:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f1953457740 1 -- 172.21.15.45:0/518095971 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556666cd4080 con 0x556666cde060 2024-01-31T05:37:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f1931ffb640 1 -- 172.21.15.45:0/518095971 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1938000e40 con 0x556666cde060 2024-01-31T05:37:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f1931ffb640 1 -- 172.21.15.45:0/518095971 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f193803c700 con 0x556666cde060 2024-01-31T05:37:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f1953457740 10 client.26134 fetch_fsmap learned FSMap version 31 2024-01-31T05:37:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f1953457740 1 -- 172.21.15.45:0/518095971 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556666c06a50 con 0x556666cde060 2024-01-31T05:37:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.166+0000 7f1931ffb640 1 -- 172.21.15.45:0/518095971 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556666cd4080 con 0x556666cde060 2024-01-31T05:37:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.167+0000 7f1931ffb640 1 -- 172.21.15.45:0/518095971 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1938062020 con 0x556666cde060 2024-01-31T05:37:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.167+0000 7f1953457740 10 client.26134 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:37:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.167+0000 7f1953457740 10 client.26134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:37:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.167+0000 7f1953457740 20 client.26134 populate_metadata read hostname 'smithi045' 2024-01-31T05:37:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.167+0000 7f1953457740 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556666d4dba0 0x556666d4ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.167+0000 7f1953457740 4 client.26134 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:37:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.167+0000 7f1953457740 1 -- 172.21.15.45:0/518095971 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f1938010ed0 con 0x556666d4dba0 2024-01-31T05:37:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.167+0000 7f194dc34640 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556666d4dba0 0x556666d4ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.167+0000 7f1931ffb640 10 client.26134 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.167+0000 7f194dc34640 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556666d4dba0 0x556666d4ffc0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06ecd625323da34 server_cookie=daa314dea6d59638 in_seq=0 out_seq=0 2024-01-31T05:37:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1931ffb640 1 -- 172.21.15.45:0/518095971 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f1938010ed0 con 0x556666d4dba0 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1931ffb640 10 client.26134 handle_command_reply: tid=0 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1953457740 1 client.26134 shutdown 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1953457740 1 -- 172.21.15.45:0/518095971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 msgr2=0x556666d46120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1953457740 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666d46120 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1938011040 tx=0x7f1938011240 comp rx=0 tx=0).stop 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1953457740 1 -- 172.21.15.45:0/518095971 shutdown_connections 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1953457740 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x556666d4dba0 0x556666d4ffc0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1953457740 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556666ce7750 0x556666cedd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1931ffb640 0 client.26134 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1953457740 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556666cde980 0x556666d46660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1953457740 1 --2- 172.21.15.45:0/518095971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556666cde060 0x556666d46120 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1953457740 1 -- 172.21.15.45:0/518095971 >> 172.21.15.45:0/518095971 conn(0x556666cd8980 msgr2=0x556666cdf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1953457740 1 -- 172.21.15.45:0/518095971 shutdown_connections 2024-01-31T05:37:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1953457740 1 -- 172.21.15.45:0/518095971 wait complete. 2024-01-31T05:37:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.220+0000 7f1953457740 20 client.26134 trim_cache size 0 max 16384 2024-01-31T05:37:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:37:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:37:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:37:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:37:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.221+0000 7f1950699640 1 -- 172.21.15.45:0/2601927062 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f19340a1570 msgr2=0x7f19340a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.221+0000 7f1950699640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f19340a1570 0x7f19340a3a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1940005e30 tx=0x7f194000f7d0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.221+0000 7f1950699640 1 -- 172.21.15.45:0/2601927062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1948100400 msgr2=0x7f1948105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.221+0000 7f1950699640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1948100400 0x7f1948105490 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f194400b450 tx=0x7f194400b910 comp rx=0 tx=0).stop 2024-01-31T05:37:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.221+0000 7f1950699640 1 -- 172.21.15.45:0/2601927062 shutdown_connections 2024-01-31T05:37:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.221+0000 7f1950699640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f19340a1570 0x7f19340a3a30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.221+0000 7f1950699640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f194810d690 0x7f19481092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.221+0000 7f1950699640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1948100400 0x7f1948105490 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.221+0000 7f1950699640 1 --2- 172.21.15.45:0/2601927062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19480ffae0 0x7f1948104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.221+0000 7f1950699640 1 -- 172.21.15.45:0/2601927062 >> 172.21.15.45:0/2601927062 conn(0x7f1948076d10 msgr2=0x7f194810eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.221+0000 7f1950699640 1 -- 172.21.15.45:0/2601927062 shutdown_connections 2024-01-31T05:37:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:09.221+0000 7f1950699640 1 -- 172.21.15.45:0/2601927062 wait complete. 2024-01-31T05:37:09.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 28711 25.10 MB/sec warmup 68 sec latency 321.723 ms 2024-01-31T05:37:09.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:09 smithi156 ceph-mon[35720]: pgmap v459: 97 pgs: 97 active+clean; 851 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.8 MiB/s wr, 397 op/s 2024-01-31T05:37:10.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:09 smithi186 ceph-mon[35659]: pgmap v459: 97 pgs: 97 active+clean; 851 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.8 MiB/s wr, 397 op/s 2024-01-31T05:37:10.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:09 smithi045 ceph-mon[31585]: pgmap v459: 97 pgs: 97 active+clean; 851 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.8 MiB/s wr, 397 op/s 2024-01-31T05:37:10.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 29257 25.25 MB/sec warmup 69 sec latency 178.995 ms 2024-01-31T05:37:11.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 29775 25.45 MB/sec warmup 70 sec latency 67.654 ms 2024-01-31T05:37:11.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:11 smithi156 ceph-mon[35720]: osdmap e112: 12 total, 12 up, 12 in 2024-01-31T05:37:11.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:11 smithi156 ceph-mon[35720]: pgmap v461: 97 pgs: 97 active+clean; 851 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 476 op/s 2024-01-31T05:37:12.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:11 smithi186 ceph-mon[35659]: osdmap e112: 12 total, 12 up, 12 in 2024-01-31T05:37:12.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:11 smithi186 ceph-mon[35659]: pgmap v461: 97 pgs: 97 active+clean; 851 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 476 op/s 2024-01-31T05:37:12.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:11 smithi045 ceph-mon[31585]: osdmap e112: 12 total, 12 up, 12 in 2024-01-31T05:37:12.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:11 smithi045 ceph-mon[31585]: pgmap v461: 97 pgs: 97 active+clean; 851 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 476 op/s 2024-01-31T05:37:12.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 30211 25.51 MB/sec warmup 71 sec latency 53.220 ms 2024-01-31T05:37:13.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 30569 25.49 MB/sec warmup 72 sec latency 170.603 ms 2024-01-31T05:37:13.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:13 smithi156 ceph-mon[35720]: pgmap v462: 97 pgs: 97 active+clean; 850 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 614 op/s 2024-01-31T05:37:14.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:13 smithi186 ceph-mon[35659]: pgmap v462: 97 pgs: 97 active+clean; 850 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 614 op/s 2024-01-31T05:37:14.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:13 smithi045 ceph-mon[31585]: pgmap v462: 97 pgs: 97 active+clean; 850 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 614 op/s 2024-01-31T05:37:14.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 30814 25.22 MB/sec warmup 73 sec latency 265.066 ms 2024-01-31T05:37:15.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 31057 25.01 MB/sec warmup 74 sec latency 485.315 ms 2024-01-31T05:37:15.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:15 smithi156 ceph-mon[35720]: pgmap v463: 97 pgs: 97 active+clean; 850 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 614 op/s 2024-01-31T05:37:16.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:15 smithi186 ceph-mon[35659]: pgmap v463: 97 pgs: 97 active+clean; 850 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 614 op/s 2024-01-31T05:37:16.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:15 smithi045 ceph-mon[31585]: pgmap v463: 97 pgs: 97 active+clean; 850 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 614 op/s 2024-01-31T05:37:16.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 31468 24.90 MB/sec warmup 75 sec latency 203.051 ms 2024-01-31T05:37:17.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 31916 24.91 MB/sec warmup 76 sec latency 183.716 ms 2024-01-31T05:37:17.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:17 smithi156 ceph-mon[35720]: pgmap v464: 97 pgs: 97 active+clean; 871 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.0 MiB/s wr, 556 op/s 2024-01-31T05:37:17.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:17 smithi156 ceph-mon[35720]: osdmap e113: 12 total, 12 up, 12 in 2024-01-31T05:37:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:17 smithi186 ceph-mon[35659]: pgmap v464: 97 pgs: 97 active+clean; 871 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.0 MiB/s wr, 556 op/s 2024-01-31T05:37:18.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:17 smithi186 ceph-mon[35659]: osdmap e113: 12 total, 12 up, 12 in 2024-01-31T05:37:18.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:17 smithi045 ceph-mon[31585]: pgmap v464: 97 pgs: 97 active+clean; 871 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.0 MiB/s wr, 556 op/s 2024-01-31T05:37:18.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:17 smithi045 ceph-mon[31585]: osdmap e113: 12 total, 12 up, 12 in 2024-01-31T05:37:18.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 32587 24.91 MB/sec warmup 77 sec latency 52.390 ms 2024-01-31T05:37:19.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 33143 25.06 MB/sec warmup 78 sec latency 152.720 ms 2024-01-31T05:37:19.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:19 smithi156 ceph-mon[35720]: pgmap v466: 97 pgs: 97 active+clean; 871 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 622 op/s 2024-01-31T05:37:20.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:19 smithi186 ceph-mon[35659]: pgmap v466: 97 pgs: 97 active+clean; 871 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 622 op/s 2024-01-31T05:37:20.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:19 smithi045 ceph-mon[31585]: pgmap v466: 97 pgs: 97 active+clean; 871 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 622 op/s 2024-01-31T05:37:20.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 33641 25.06 MB/sec warmup 79 sec latency 327.603 ms 2024-01-31T05:37:21.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 34103 25.19 MB/sec warmup 80 sec latency 146.864 ms 2024-01-31T05:37:21.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:21 smithi156 ceph-mon[35720]: pgmap v467: 97 pgs: 97 active+clean; 871 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.0 MiB/s wr, 556 op/s 2024-01-31T05:37:22.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:21 smithi186 ceph-mon[35659]: pgmap v467: 97 pgs: 97 active+clean; 871 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.0 MiB/s wr, 556 op/s 2024-01-31T05:37:22.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:21 smithi045 ceph-mon[31585]: pgmap v467: 97 pgs: 97 active+clean; 871 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.0 MiB/s wr, 556 op/s 2024-01-31T05:37:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.317+0000 7f3faa905640 1 -- 172.21.15.45:0/2125354291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa4104b90 msgr2=0x7f3fa4104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.317+0000 7f3faa905640 1 --2- 172.21.15.45:0/2125354291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa4104b90 0x7f3fa4104f70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009900 tx=0x7f3f9402bc30 comp rx=0 tx=0).stop 2024-01-31T05:37:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.317+0000 7f3faa905640 1 -- 172.21.15.45:0/2125354291 shutdown_connections 2024-01-31T05:37:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.317+0000 7f3faa905640 1 --2- 172.21.15.45:0/2125354291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fa4100800 0x7f3fa410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.317+0000 7f3faa905640 1 --2- 172.21.15.45:0/2125354291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa40ffe40 0x7f3fa41002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.317+0000 7f3faa905640 1 --2- 172.21.15.45:0/2125354291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa4104b90 0x7f3fa4104f70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.317+0000 7f3faa905640 1 -- 172.21.15.45:0/2125354291 >> 172.21.15.45:0/2125354291 conn(0x7f3fa4076a40 msgr2=0x7f3fa4076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.317+0000 7f3faa905640 1 -- 172.21.15.45:0/2125354291 shutdown_connections 2024-01-31T05:37:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.317+0000 7f3faa905640 1 -- 172.21.15.45:0/2125354291 wait complete. 2024-01-31T05:37:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3faa905640 1 Processor -- start 2024-01-31T05:37:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3faa905640 1 -- start start 2024-01-31T05:37:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3faa905640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa40ffe40 0x7f3fa41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3faa905640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fa4100800 0x7f3fa4198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3faa905640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4104b90 0x7f3fa41a3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3faa905640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa4198e70 con 0x7f3fa40ffe40 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3faa905640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa41a3690 con 0x7f3fa4104b90 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3faa905640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa41a37d0 con 0x7f3fa4100800 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa40ffe40 0x7f3fa41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa40ffe40 0x7f3fa41980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46686/0 (socket says 172.21.15.45:46686) 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3fa3fff640 1 -- 172.21.15.45:0/2686881933 learned_addr learned my addr 172.21.15.45:0/2686881933 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fa4100800 0x7f3fa4198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.318+0000 7f3fa8ea2640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4104b90 0x7f3fa41a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.319+0000 7f3fa3fff640 1 -- 172.21.15.45:0/2686881933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fa4100800 msgr2=0x7f3fa4198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.319+0000 7f3fa3fff640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fa4100800 0x7f3fa4198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.319+0000 7f3fa3fff640 1 -- 172.21.15.45:0/2686881933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4104b90 msgr2=0x7f3fa41a3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.319+0000 7f3fa3fff640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4104b90 0x7f3fa41a3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.319+0000 7f3fa3fff640 1 -- 172.21.15.45:0/2686881933 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f94009560 con 0x7f3fa40ffe40 2024-01-31T05:37:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.319+0000 7f3fa8ea2640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4104b90 0x7f3fa41a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:37:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.319+0000 7f3fa3fff640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa40ffe40 0x7f3fa41980d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3f94005940 tx=0x7f3f9402c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.319+0000 7f3fa17fa640 1 -- 172.21.15.45:0/2686881933 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94004250 con 0x7f3fa40ffe40 2024-01-31T05:37:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.319+0000 7f3fa17fa640 1 -- 172.21.15.45:0/2686881933 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f940043b0 con 0x7f3fa40ffe40 2024-01-31T05:37:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.319+0000 7f3faa905640 1 -- 172.21.15.45:0/2686881933 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa41a39d0 con 0x7f3fa40ffe40 2024-01-31T05:37:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.319+0000 7f3fa17fa640 1 -- 172.21.15.45:0/2686881933 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94003b90 con 0x7f3fa40ffe40 2024-01-31T05:37:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.319+0000 7f3faa905640 1 -- 172.21.15.45:0/2686881933 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa41a3ee0 con 0x7f3fa40ffe40 2024-01-31T05:37:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.322+0000 7f3fa17fa640 1 -- 172.21.15.45:0/2686881933 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f3f94003cf0 con 0x7f3fa40ffe40 2024-01-31T05:37:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.322+0000 7f3fad79d740 1 Processor -- start 2024-01-31T05:37:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.322+0000 7f3fad79d740 1 -- start start 2024-01-31T05:37:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.322+0000 7f3fad79d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9c1060 0x558a4c9c1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.322+0000 7f3fad79d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.322+0000 7f3fa17fa640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f3f900a14f0 0x7f3f900a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.322+0000 7f3fad79d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9ca750 0x558a4c9ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.322+0000 7f3fad79d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4c930040 con 0x558a4c9ca750 2024-01-31T05:37:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.322+0000 7f3fad79d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4c92d730 con 0x558a4c9c1060 2024-01-31T05:37:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.322+0000 7f3fad79d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4c929020 con 0x558a4c9c1980 2024-01-31T05:37:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa8ea2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9c1060 0x558a4c9c1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa37fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa8ea2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9ca750 0x558a4c9ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa37fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:46600/0 (socket says 172.21.15.45:46600) 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa37fe640 1 -- 172.21.15.45:0/3623793308 learned_addr learned my addr 172.21.15.45:0/3623793308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa3fff640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f3f900a14f0 0x7f3f900a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa17fa640 1 -- 172.21.15.45:0/2686881933 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3f9402de40 con 0x7f3fa40ffe40 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa37fe640 1 -- 172.21.15.45:0/3623793308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9c1060 msgr2=0x558a4c9c1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/3623793308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9c1060 0x558a4c9c1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa37fe640 1 -- 172.21.15.45:0/3623793308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9ca750 msgr2=0x558a4c9ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/3623793308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9ca750 0x558a4c9ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa37fe640 1 -- 172.21.15.45:0/3623793308 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a4c920e40 con 0x558a4c9c1980 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa3fff640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f3f900a14f0 0x7f3f900a39b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f940bc2f0 tx=0x7f3f940b1d60 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/3623793308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9ca210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009c90 tx=0x7f3f8c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8a98d86a4ae0d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3f9a7fc640 1 -- 172.21.15.45:0/3623793308 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c019630 con 0x558a4c9c1980 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3f9a7fc640 1 -- 172.21.15.45:0/3623793308 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f8c019c50 con 0x558a4c9c1980 2024-01-31T05:37:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.323+0000 7f3f9a7fc640 1 -- 172.21.15.45:0/3623793308 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00e9c0 con 0x558a4c9c1980 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.324+0000 7f3fad79d740 1 -- 172.21.15.45:0/3623793308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 msgr2=0x558a4c9ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.324+0000 7f3fad79d740 1 --2- 172.21.15.45:0/3623793308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9ca210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009c90 tx=0x7f3f8c011810 comp rx=0 tx=0).stop 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.324+0000 7f3fad79d740 1 -- 172.21.15.45:0/3623793308 shutdown_connections 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.324+0000 7f3fad79d740 1 --2- 172.21.15.45:0/3623793308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9ca750 0x558a4c9ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.324+0000 7f3fad79d740 1 --2- 172.21.15.45:0/3623793308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9ca210 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.324+0000 7f3fad79d740 1 --2- 172.21.15.45:0/3623793308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9c1060 0x558a4c9c1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.324+0000 7f3fad79d740 1 -- 172.21.15.45:0/3623793308 >> 172.21.15.45:0/3623793308 conn(0x558a4c9bb980 msgr2=0x558a4c9bbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.324+0000 7f3fad79d740 1 -- 172.21.15.45:0/3623793308 shutdown_connections 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.324+0000 7f3fad79d740 1 -- 172.21.15.45:0/3623793308 wait complete. 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.324+0000 7f3fad79d740 1 -- start start 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fad79d740 1 Processor -- start 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fad79d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9c1060 0x558a4c9c7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fad79d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fad79d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9ca750 0x558a4c9c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fad79d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4c8e21b0 con 0x558a4c9ca750 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fad79d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4c8e1e50 con 0x558a4c9c1060 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fad79d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4c8a7090 con 0x558a4c9c1980 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fa37fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fa8ea2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9c1060 0x558a4c9c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fad79d740 10 client.0 _ll_register_callbacks cb 0x558a4c8cca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fa8ea2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9c1060 0x558a4c9c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:40534/0 (socket says 172.21.15.45:40534) 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fa37fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:46614/0 (socket says 172.21.15.45:46614) 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fa8ea2640 1 -- 172.21.15.45:0/2227206816 learned_addr learned my addr 172.21.15.45:0/2227206816 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.325+0000 7f3fa8ea2640 1 --2- 172.21.15.45:0/2227206816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9ca750 0x558a4c9c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3fa8ea2640 1 -- 172.21.15.45:0/2227206816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 msgr2=0x558a4c9c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3fa8ea2640 1 --2- 172.21.15.45:0/2227206816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3fa8ea2640 1 -- 172.21.15.45:0/2227206816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9ca750 msgr2=0x558a4c9c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3fa8ea2640 1 --2- 172.21.15.45:0/2227206816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9ca750 0x558a4c9c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3fa8ea2640 1 -- 172.21.15.45:0/2227206816 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8c0098f0 con 0x558a4c9c1060 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3fa8ea2640 1 --2- 172.21.15.45:0/2227206816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9c1060 0x558a4c9c7690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c010ea0 tx=0x7f3f9c0105e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7435982a045d3996 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3f837fe640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3f837fe640 1 -- 172.21.15.45:0/2227206816 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c01d2d0 con 0x558a4c9c1060 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3fad79d740 1 -- 172.21.15.45:0/2227206816 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a4c922010 con 0x558a4c9c1060 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3f837fe640 1 -- 172.21.15.45:0/2227206816 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3f9c00fe60 con 0x558a4c9c1060 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3f837fe640 1 -- 172.21.15.45:0/2227206816 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f9c01d430 con 0x558a4c9c1060 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3fad79d740 10 client.26075 fetch_fsmap learned FSMap version 31 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.326+0000 7f3fad79d740 1 -- 172.21.15.45:0/2227206816 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558a4c9276a0 con 0x558a4c9c1060 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.327+0000 7f3f837fe640 1 -- 172.21.15.45:0/2227206816 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a4c922010 con 0x558a4c9c1060 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.327+0000 7f3f837fe640 1 -- 172.21.15.45:0/2227206816 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3f9c004050 con 0x558a4c9c1060 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.327+0000 7f3fad79d740 10 client.26075 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.327+0000 7f3fad79d740 10 client.26075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.327+0000 7f3fad79d740 20 client.26075 populate_metadata read hostname 'smithi045' 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.327+0000 7f3fad79d740 1 --2- 172.21.15.45:0/2227206816 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558a4c9da5a0 0x558a4c9dc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.327+0000 7f3fad79d740 4 client.26075 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:37:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.327+0000 7f3fad79d740 1 -- 172.21.15.45:0/2227206816 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a4c921d30 con 0x558a4c9da5a0 2024-01-31T05:37:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.327+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/2227206816 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558a4c9da5a0 0x558a4c9dc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.328+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/2227206816 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558a4c9da5a0 0x558a4c9dc9c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c431d551b693b7cb server_cookie=ba75f1504096bf39 in_seq=0 out_seq=0 2024-01-31T05:37:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.328+0000 7f3f837fe640 10 client.26075 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.328+0000 7f3f837fe640 1 -- 172.21.15.45:0/2227206816 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558a4c921d30 con 0x558a4c9da5a0 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3f837fe640 10 client.26075 handle_command_reply: tid=0 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3fad79d740 1 client.26075 shutdown 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3fad79d740 1 -- 172.21.15.45:0/2227206816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9c1060 msgr2=0x558a4c9c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3fad79d740 1 --2- 172.21.15.45:0/2227206816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9c1060 0x558a4c9c7690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c010ea0 tx=0x7f3f9c0105e0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3fad79d740 1 -- 172.21.15.45:0/2227206816 shutdown_connections 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3fad79d740 1 --2- 172.21.15.45:0/2227206816 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558a4c9da5a0 0x558a4c9dc9c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3f837fe640 0 client.26075 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3fad79d740 1 --2- 172.21.15.45:0/2227206816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9ca750 0x558a4c9c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3fad79d740 1 --2- 172.21.15.45:0/2227206816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3fad79d740 1 --2- 172.21.15.45:0/2227206816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9c1060 0x558a4c9c7690 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3fad79d740 1 -- 172.21.15.45:0/2227206816 >> 172.21.15.45:0/2227206816 conn(0x558a4c9bb980 msgr2=0x558a4c9bd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3fad79d740 1 -- 172.21.15.45:0/2227206816 shutdown_connections 2024-01-31T05:37:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3fad79d740 1 -- 172.21.15.45:0/2227206816 wait complete. 2024-01-31T05:37:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.329+0000 7f3fad79d740 20 client.26075 trim_cache size 0 max 16384 2024-01-31T05:37:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.342+0000 7f3fad79d740 1 Processor -- start 2024-01-31T05:37:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fad79d740 1 -- start start 2024-01-31T05:37:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fad79d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1060 0x558a4c9c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fad79d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9ba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fad79d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9ca750 0x558a4c9c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fad79d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4c9ccf20 con 0x558a4c9c1060 2024-01-31T05:37:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fad79d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4c921d30 con 0x558a4c9ca750 2024-01-31T05:37:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fad79d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4c9c8660 con 0x558a4c9c1980 2024-01-31T05:37:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fa8ea2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1060 0x558a4c9c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fa8ea2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1060 0x558a4c9c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46726/0 (socket says 172.21.15.45:46726) 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fa8ea2640 1 -- 172.21.15.45:0/2716614390 learned_addr learned my addr 172.21.15.45:0/2716614390 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/2716614390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9ba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fa8ea2640 1 --2- 172.21.15.45:0/2716614390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9ca750 0x558a4c9c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fa37fe640 1 -- 172.21.15.45:0/2716614390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9ca750 msgr2=0x558a4c9c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/2716614390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9ca750 0x558a4c9c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fa37fe640 1 -- 172.21.15.45:0/2716614390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1060 msgr2=0x558a4c9c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/2716614390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1060 0x558a4c9c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fa37fe640 1 -- 172.21.15.45:0/2716614390 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a4c8c4030 con 0x558a4c9c1980 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fa8ea2640 1 --2- 172.21.15.45:0/2716614390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1060 0x558a4c9c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.343+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/2716614390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9ba8a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c000c00 tx=0x7f3f8c00e600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d482c21bd723da91 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.344+0000 7f3f99ffb640 1 -- 172.21.15.45:0/2716614390 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c029070 con 0x558a4c9c1980 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.344+0000 7f3f99ffb640 1 -- 172.21.15.45:0/2716614390 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f8c005680 con 0x558a4c9c1980 2024-01-31T05:37:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.344+0000 7f3f99ffb640 1 -- 172.21.15.45:0/2716614390 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c019690 con 0x558a4c9c1980 2024-01-31T05:37:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.345+0000 7f3fad79d740 1 -- 172.21.15.45:0/2716614390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 msgr2=0x558a4c9ba8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.345+0000 7f3fad79d740 1 --2- 172.21.15.45:0/2716614390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9ba8a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c000c00 tx=0x7f3f8c00e600 comp rx=0 tx=0).stop 2024-01-31T05:37:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.345+0000 7f3fad79d740 1 -- 172.21.15.45:0/2716614390 shutdown_connections 2024-01-31T05:37:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.345+0000 7f3fad79d740 1 --2- 172.21.15.45:0/2716614390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9ca750 0x558a4c9c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.345+0000 7f3fad79d740 1 --2- 172.21.15.45:0/2716614390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1980 0x558a4c9ba8a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.345+0000 7f3fad79d740 1 --2- 172.21.15.45:0/2716614390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1060 0x558a4c9c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.345+0000 7f3fad79d740 1 -- 172.21.15.45:0/2716614390 >> 172.21.15.45:0/2716614390 conn(0x558a4c9bb980 msgr2=0x558a4c9ba190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.345+0000 7f3fad79d740 1 -- 172.21.15.45:0/2716614390 shutdown_connections 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.345+0000 7f3fad79d740 1 -- 172.21.15.45:0/2716614390 wait complete. 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.345+0000 7f3fad79d740 1 -- start start 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fad79d740 1 Processor -- start 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fad79d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1060 0x558a4c9d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fad79d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1980 0x558a4c9cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fad79d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9ca750 0x558a4c9d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fad79d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4c9d08c0 con 0x558a4c9c1980 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fad79d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4c9d0a30 con 0x558a4c9ca750 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fad79d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4ca30ab0 con 0x558a4c9c1060 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fad79d740 10 client.0 _ll_register_callbacks cb 0x7f3f8c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fa37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1980 0x558a4c9cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fa37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1980 0x558a4c9cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46742/0 (socket says 172.21.15.45:46742) 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fa8ea2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1060 0x558a4c9d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fa37fe640 1 -- 172.21.15.45:0/4084474640 learned_addr learned my addr 172.21.15.45:0/4084474640 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fa8ea2640 1 --2- 172.21.15.45:0/4084474640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9ca750 0x558a4c9d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fa37fe640 1 -- 172.21.15.45:0/4084474640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1060 msgr2=0x558a4c9d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/4084474640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1060 0x558a4c9d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fa37fe640 1 -- 172.21.15.45:0/4084474640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9ca750 msgr2=0x558a4c9d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/4084474640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9ca750 0x558a4c9d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.346+0000 7f3fa37fe640 1 -- 172.21.15.45:0/4084474640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a4c9276a0 con 0x558a4c9c1980 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.347+0000 7f3fa37fe640 1 --2- 172.21.15.45:0/4084474640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1980 0x558a4c9cfc80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c01c040 tx=0x7f3f8c009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c5bf9a0635ebff server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.347+0000 7f3f837fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.347+0000 7f3f837fe640 1 -- 172.21.15.45:0/4084474640 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c029040 con 0x558a4c9c1980 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.347+0000 7f3f837fe640 1 -- 172.21.15.45:0/4084474640 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3f8c02a210 con 0x558a4c9c1980 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.347+0000 7f3fad79d740 1 -- 172.21.15.45:0/4084474640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f8c019c50 con 0x558a4c9c1980 2024-01-31T05:37:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.347+0000 7f3f837fe640 1 -- 172.21.15.45:0/4084474640 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f8c02a430 con 0x558a4c9c1980 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.347+0000 7f3fad79d740 10 client.16587 fetch_fsmap learned FSMap version 31 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.347+0000 7f3fad79d740 1 -- 172.21.15.45:0/4084474640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558a4c920e40 con 0x558a4c9c1980 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.347+0000 7f3f837fe640 1 -- 172.21.15.45:0/4084474640 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f8c03f020 con 0x558a4c9c1980 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.347+0000 7f3f837fe640 1 -- 172.21.15.45:0/4084474640 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3f8c0197a0 con 0x558a4c9c1980 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.347+0000 7f3fad79d740 10 client.16587 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.347+0000 7f3fad79d740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.348+0000 7f3fad79d740 20 client.16587 populate_metadata read hostname 'smithi045' 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.348+0000 7f3fad79d740 1 --2- 172.21.15.45:0/4084474640 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558a4c9da5a0 0x558a4ca33e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.348+0000 7f3fad79d740 4 client.16587 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.348+0000 7f3fad79d740 1 -- 172.21.15.45:0/4084474640 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558a4c628600 con 0x558a4c9da5a0 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.348+0000 7f3fa8ea2640 1 --2- 172.21.15.45:0/4084474640 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558a4c9da5a0 0x558a4ca33e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.348+0000 7f3f837fe640 10 client.16587 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.348+0000 7f3fa8ea2640 1 --2- 172.21.15.45:0/4084474640 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558a4c9da5a0 0x558a4ca33e80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506061ed0df9ee8d server_cookie=8c69dc064233074e in_seq=0 out_seq=0 2024-01-31T05:37:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.393+0000 7f3f837fe640 1 -- 172.21.15.45:0/4084474640 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558a4c628600 con 0x558a4c9da5a0 2024-01-31T05:37:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.393+0000 7f3f837fe640 10 client.16587 handle_command_reply: tid=0 2024-01-31T05:37:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.393+0000 7f3fad79d740 1 client.16587 shutdown 2024-01-31T05:37:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3fad79d740 1 -- 172.21.15.45:0/4084474640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1980 msgr2=0x558a4c9cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3fad79d740 1 --2- 172.21.15.45:0/4084474640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1980 0x558a4c9cfc80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c01c040 tx=0x7f3f8c009640 comp rx=0 tx=0).stop 2024-01-31T05:37:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3fad79d740 1 -- 172.21.15.45:0/4084474640 shutdown_connections 2024-01-31T05:37:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3fad79d740 1 --2- 172.21.15.45:0/4084474640 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x558a4c9da5a0 0x558a4ca33e80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3fad79d740 1 --2- 172.21.15.45:0/4084474640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558a4c9c1980 0x558a4c9cfc80 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3fad79d740 1 --2- 172.21.15.45:0/4084474640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a4c9ca750 0x558a4c9d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3fad79d740 1 --2- 172.21.15.45:0/4084474640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a4c9c1060 0x558a4c9d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3f837fe640 0 client.16587 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3fad79d740 1 -- 172.21.15.45:0/4084474640 >> 172.21.15.45:0/4084474640 conn(0x558a4c9bb980 msgr2=0x558a4c9c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3fad79d740 1 -- 172.21.15.45:0/4084474640 shutdown_connections 2024-01-31T05:37:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3fad79d740 1 -- 172.21.15.45:0/4084474640 wait complete. 2024-01-31T05:37:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3fad79d740 20 client.16587 trim_cache size 0 max 16384 2024-01-31T05:37:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:37:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:37:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:37:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:37:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3faa905640 1 -- 172.21.15.45:0/2686881933 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f3f900a14f0 msgr2=0x7f3f900a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3faa905640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f3f900a14f0 0x7f3f900a39b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f940bc2f0 tx=0x7f3f940b1d60 comp rx=0 tx=0).stop 2024-01-31T05:37:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3faa905640 1 -- 172.21.15.45:0/2686881933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa40ffe40 msgr2=0x7f3fa41980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.394+0000 7f3faa905640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa40ffe40 0x7f3fa41980d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3f94005940 tx=0x7f3f9402c670 comp rx=0 tx=0).stop 2024-01-31T05:37:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.395+0000 7f3faa905640 1 -- 172.21.15.45:0/2686881933 shutdown_connections 2024-01-31T05:37:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.395+0000 7f3faa905640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f3f900a14f0 0x7f3f900a39b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.395+0000 7f3faa905640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fa4104b90 0x7f3fa41a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.395+0000 7f3faa905640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fa4100800 0x7f3fa4198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.395+0000 7f3faa905640 1 --2- 172.21.15.45:0/2686881933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa40ffe40 0x7f3fa41980d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.395+0000 7f3faa905640 1 -- 172.21.15.45:0/2686881933 >> 172.21.15.45:0/2686881933 conn(0x7f3fa4076a40 msgr2=0x7f3fa40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.395+0000 7f3faa905640 1 -- 172.21.15.45:0/2686881933 shutdown_connections 2024-01-31T05:37:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:22.395+0000 7f3faa905640 1 -- 172.21.15.45:0/2686881933 wait complete. 2024-01-31T05:37:22.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 34551 25.24 MB/sec warmup 81 sec latency 184.429 ms 2024-01-31T05:37:23.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 34899 25.20 MB/sec warmup 82 sec latency 195.523 ms 2024-01-31T05:37:23.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:23 smithi156 ceph-mon[35720]: pgmap v468: 97 pgs: 97 active+clean; 889 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.9 MiB/s wr, 482 op/s 2024-01-31T05:37:24.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:23 smithi186 ceph-mon[35659]: pgmap v468: 97 pgs: 97 active+clean; 889 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.9 MiB/s wr, 482 op/s 2024-01-31T05:37:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:23 smithi045 ceph-mon[31585]: pgmap v468: 97 pgs: 97 active+clean; 889 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.9 MiB/s wr, 482 op/s 2024-01-31T05:37:24.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 35229 25.01 MB/sec warmup 83 sec latency 207.306 ms 2024-01-31T05:37:25.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 35446 24.82 MB/sec warmup 84 sec latency 485.310 ms 2024-01-31T05:37:25.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:25 smithi156 ceph-mon[35720]: pgmap v469: 97 pgs: 97 active+clean; 889 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.9 MiB/s wr, 482 op/s 2024-01-31T05:37:26.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:25 smithi186 ceph-mon[35659]: pgmap v469: 97 pgs: 97 active+clean; 889 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.9 MiB/s wr, 482 op/s 2024-01-31T05:37:26.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:25 smithi045 ceph-mon[31585]: pgmap v469: 97 pgs: 97 active+clean; 889 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.9 MiB/s wr, 482 op/s 2024-01-31T05:37:26.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 35820 24.74 MB/sec warmup 85 sec latency 346.941 ms 2024-01-31T05:37:27.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 36328 24.76 MB/sec warmup 86 sec latency 174.799 ms 2024-01-31T05:37:27.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:27 smithi156 ceph-mon[35720]: pgmap v470: 97 pgs: 97 active+clean; 902 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.3 MiB/s wr, 474 op/s 2024-01-31T05:37:28.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:27 smithi186 ceph-mon[35659]: pgmap v470: 97 pgs: 97 active+clean; 902 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.3 MiB/s wr, 474 op/s 2024-01-31T05:37:28.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:27 smithi045 ceph-mon[31585]: pgmap v470: 97 pgs: 97 active+clean; 902 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.3 MiB/s wr, 474 op/s 2024-01-31T05:37:28.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 36925 24.74 MB/sec warmup 87 sec latency 152.457 ms 2024-01-31T05:37:29.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 37577 24.94 MB/sec warmup 88 sec latency 49.394 ms 2024-01-31T05:37:29.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:29 smithi156 ceph-mon[35720]: pgmap v471: 97 pgs: 97 active+clean; 902 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.8 MiB/s wr, 395 op/s 2024-01-31T05:37:30.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:29 smithi186 ceph-mon[35659]: pgmap v471: 97 pgs: 97 active+clean; 902 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.8 MiB/s wr, 395 op/s 2024-01-31T05:37:30.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:29 smithi045 ceph-mon[31585]: pgmap v471: 97 pgs: 97 active+clean; 902 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.8 MiB/s wr, 395 op/s 2024-01-31T05:37:30.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 38141 25.00 MB/sec warmup 89 sec latency 146.947 ms 2024-01-31T05:37:31.645 INFO:tasks.workunit.client.0.smithi045.stdout: 10 38597 25.12 MB/sec warmup 90 sec latency 195.207 ms 2024-01-31T05:37:31.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:31 smithi156 ceph-mon[35720]: pgmap v472: 97 pgs: 97 active+clean; 902 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.8 MiB/s wr, 395 op/s 2024-01-31T05:37:32.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:31 smithi186 ceph-mon[35659]: pgmap v472: 97 pgs: 97 active+clean; 902 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.8 MiB/s wr, 395 op/s 2024-01-31T05:37:32.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:31 smithi045 ceph-mon[31585]: pgmap v472: 97 pgs: 97 active+clean; 902 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.8 MiB/s wr, 395 op/s 2024-01-31T05:37:32.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 39021 25.11 MB/sec warmup 91 sec latency 202.146 ms 2024-01-31T05:37:33.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 39471 25.10 MB/sec warmup 92 sec latency 173.301 ms 2024-01-31T05:37:33.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:33 smithi156 ceph-mon[35720]: pgmap v473: 97 pgs: 97 active+clean; 827 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 580 op/s 2024-01-31T05:37:34.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:33 smithi186 ceph-mon[35659]: pgmap v473: 97 pgs: 97 active+clean; 827 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 580 op/s 2024-01-31T05:37:34.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:33 smithi045 ceph-mon[31585]: pgmap v473: 97 pgs: 97 active+clean; 827 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 580 op/s 2024-01-31T05:37:34.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 39838 25.03 MB/sec warmup 93 sec latency 192.065 ms 2024-01-31T05:37:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.491+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/970720304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfcc100800 msgr2=0x7fbfcc10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.491+0000 7fbfd3fe1640 1 --2- 172.21.15.45:0/970720304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfcc100800 0x7fbfcc10ca20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009ca0 tx=0x7fbfc402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.491+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/970720304 shutdown_connections 2024-01-31T05:37:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.491+0000 7fbfd3fe1640 1 --2- 172.21.15.45:0/970720304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfcc100800 0x7fbfcc10ca20 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.491+0000 7fbfd3fe1640 1 --2- 172.21.15.45:0/970720304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfcc0ffe40 0x7fbfcc1002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.491+0000 7fbfd3fe1640 1 --2- 172.21.15.45:0/970720304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc104b90 0x7fbfcc104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.491+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/970720304 >> 172.21.15.45:0/970720304 conn(0x7fbfcc076a40 msgr2=0x7fbfcc076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.491+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/970720304 shutdown_connections 2024-01-31T05:37:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.492+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/970720304 wait complete. 2024-01-31T05:37:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.492+0000 7fbfd3fe1640 1 Processor -- start 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.492+0000 7fbfd3fe1640 1 -- start start 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.492+0000 7fbfd3fe1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc0ffe40 0x7fbfcc196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.492+0000 7fbfd3fe1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfcc100800 0x7fbfcc1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.492+0000 7fbfd3fe1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfcc104b90 0x7fbfcc19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd3fe1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc19af50 con 0x7fbfcc104b90 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd3fe1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc19b0c0 con 0x7fbfcc100800 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd3fe1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc19b280 con 0x7fbfcc0ffe40 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd257e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfcc104b90 0x7fbfcc19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd257e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfcc104b90 0x7fbfcc19a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41890/0 (socket says 172.21.15.45:41890) 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd257e640 1 -- 172.21.15.45:0/3861958653 learned_addr learned my addr 172.21.15.45:0/3861958653 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd157c640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfcc100800 0x7fbfcc1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd1d7d640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc0ffe40 0x7fbfcc196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd257e640 1 -- 172.21.15.45:0/3861958653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc0ffe40 msgr2=0x7fbfcc196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd257e640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc0ffe40 0x7fbfcc196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd257e640 1 -- 172.21.15.45:0/3861958653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfcc100800 msgr2=0x7fbfcc1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd257e640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfcc100800 0x7fbfcc1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd257e640 1 -- 172.21.15.45:0/3861958653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc009490 con 0x7fbfcc104b90 2024-01-31T05:37:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd257e640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfcc104b90 0x7fbfcc19a950 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbfc402c700 tx=0x7fbfc4032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfc2ffd640 1 -- 172.21.15.45:0/3861958653 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4032800 con 0x7fbfcc104b90 2024-01-31T05:37:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfc2ffd640 1 -- 172.21.15.45:0/3861958653 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfc4003e30 con 0x7fbfcc104b90 2024-01-31T05:37:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/3861958653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc4009900 con 0x7fbfcc104b90 2024-01-31T05:37:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfc2ffd640 1 -- 172.21.15.45:0/3861958653 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40047f0 con 0x7fbfcc104b90 2024-01-31T05:37:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.493+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/3861958653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfcc1a2c00 con 0x7fbfcc104b90 2024-01-31T05:37:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.496+0000 7fbfc2ffd640 1 -- 172.21.15.45:0/3861958653 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fbfc403b040 con 0x7fbfcc104b90 2024-01-31T05:37:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.496+0000 7fbfd6da9740 1 Processor -- start 2024-01-31T05:37:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd6da9740 1 -- start start 2024-01-31T05:37:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd6da9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cbf0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd6da9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 0x55607cbf9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd6da9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 0x55607cbfbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd6da9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607cb5f040 con 0x55607cbf0980 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd6da9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607cb5c730 con 0x55607cbf0060 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd6da9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607cb58020 con 0x55607cbf9750 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfc2ffd640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fbfb80a14f0 0x7fbfb80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd157c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cbf0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd1d7d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 0x55607cbf9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd157c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cbf0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:49858/0 (socket says 172.21.15.45:49858) 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd157c640 1 -- 172.21.15.45:0/956373302 learned_addr learned my addr 172.21.15.45:0/956373302 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd157c640 1 --2- 172.21.15.45:0/956373302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 0x55607cbfbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd257e640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fbfb80a14f0 0x7fbfb80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfc2ffd640 1 -- 172.21.15.45:0/3861958653 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbfc40ef660 con 0x7fbfcc104b90 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd157c640 1 -- 172.21.15.45:0/956373302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 msgr2=0x55607cbfbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd157c640 1 --2- 172.21.15.45:0/956373302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 0x55607cbfbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd157c640 1 -- 172.21.15.45:0/956373302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 msgr2=0x55607cbf9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd157c640 1 --2- 172.21.15.45:0/956373302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 0x55607cbf9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd157c640 1 -- 172.21.15.45:0/956373302 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55607cb4fe40 con 0x55607cbf0060 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.497+0000 7fbfd1d7d640 1 --2- 172.21.15.45:0/956373302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 0x55607cbf9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.498+0000 7fbfd257e640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fbfb80a14f0 0x7fbfb80a39b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40b9090 tx=0x7fbfc402c0f0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.498+0000 7fbfd157c640 1 --2- 172.21.15.45:0/956373302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cbf0440 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0093e0 tx=0x7fbfbc009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79c7b42ae4edda97 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.498+0000 7fbfb7fff640 1 -- 172.21.15.45:0/956373302 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc011020 con 0x55607cbf0060 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.498+0000 7fbfb7fff640 1 -- 172.21.15.45:0/956373302 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfbc011180 con 0x55607cbf0060 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.498+0000 7fbfb7fff640 1 -- 172.21.15.45:0/956373302 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00fee0 con 0x55607cbf0060 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.499+0000 7fbfd6da9740 1 -- 172.21.15.45:0/956373302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 msgr2=0x55607cbf0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.499+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/956373302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cbf0440 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0093e0 tx=0x7fbfbc009370 comp rx=0 tx=0).stop 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.499+0000 7fbfd6da9740 1 -- 172.21.15.45:0/956373302 shutdown_connections 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.499+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/956373302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 0x55607cbf9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.499+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/956373302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 0x55607cbfbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.499+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/956373302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cbf0440 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.499+0000 7fbfd6da9740 1 -- 172.21.15.45:0/956373302 >> 172.21.15.45:0/956373302 conn(0x55607cbea980 msgr2=0x55607cbead60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.499+0000 7fbfd6da9740 1 -- 172.21.15.45:0/956373302 shutdown_connections 2024-01-31T05:37:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.499+0000 7fbfd6da9740 1 -- 172.21.15.45:0/956373302 wait complete. 2024-01-31T05:37:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.499+0000 7fbfd6da9740 1 -- start start 2024-01-31T05:37:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.499+0000 7fbfd6da9740 1 Processor -- start 2024-01-31T05:37:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd6da9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0060 0x55607cbfc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd6da9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0980 0x55607cbfc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd6da9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 0x55607cbfced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd6da9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607cb111b0 con 0x55607cbf0060 2024-01-31T05:37:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd6da9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607cb10e50 con 0x55607cbf0980 2024-01-31T05:37:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd6da9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607cad6090 con 0x55607cbf9750 2024-01-31T05:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd6da9740 10 client.0 _ll_register_callbacks cb 0x55607cafba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd157c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0060 0x55607cbfc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd157c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0060 0x55607cbfc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41900/0 (socket says 172.21.15.45:41900) 2024-01-31T05:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd157c640 1 -- 172.21.15.45:0/3141351726 learned_addr learned my addr 172.21.15.45:0/3141351726 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd1d7d640 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0980 0x55607cbfc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd157c640 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 0x55607cbfced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd157c640 1 -- 172.21.15.45:0/3141351726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 msgr2=0x55607cbfced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd157c640 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 0x55607cbfced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd157c640 1 -- 172.21.15.45:0/3141351726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0980 msgr2=0x55607cbfc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd157c640 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0980 0x55607cbfc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd157c640 1 -- 172.21.15.45:0/3141351726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc019040 con 0x55607cbf0060 2024-01-31T05:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd1d7d640 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0980 0x55607cbfc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfd157c640 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0060 0x55607cbfc450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55607cbea270 tx=0x7fbfbc011760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac428d8959ad0422 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfb57fa640 1 -- 172.21.15.45:0/3141351726 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0041b0 con 0x55607cbf0060 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.500+0000 7fbfb57fa640 1 -- 172.21.15.45:0/3141351726 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbfbc004310 con 0x55607cbf0060 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3141351726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55607cb51010 con 0x55607cbf0060 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfb57fa640 1 -- 172.21.15.45:0/3141351726 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfbc0044d0 con 0x55607cbf0060 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfd6da9740 10 client.16599 fetch_fsmap learned FSMap version 31 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3141351726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55607cb566a0 con 0x55607cbf0060 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfb57fa640 1 -- 172.21.15.45:0/3141351726 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55607cb51010 con 0x55607cbf0060 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfb57fa640 1 -- 172.21.15.45:0/3141351726 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbfbc01d070 con 0x55607cbf0060 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfd6da9740 10 client.16599 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfd6da9740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfd6da9740 20 client.16599 populate_metadata read hostname 'smithi045' 2024-01-31T05:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55607cc0b5b0 0x55607cc0d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfd6da9740 4 client.16599 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3141351726 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55607cb50d30 con 0x55607cc0b5b0 2024-01-31T05:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.501+0000 7fbfd1d7d640 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55607cc0b5b0 0x55607cc0d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.502+0000 7fbfb57fa640 10 client.16599 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.502+0000 7fbfd1d7d640 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55607cc0b5b0 0x55607cc0d9d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515070b3ab13ff server_cookie=2a75480949c41c73 in_seq=0 out_seq=0 2024-01-31T05:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.502+0000 7fbfb57fa640 1 -- 172.21.15.45:0/3141351726 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55607cb50d30 con 0x55607cc0b5b0 2024-01-31T05:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.502+0000 7fbfb57fa640 10 client.16599 handle_command_reply: tid=0 2024-01-31T05:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.502+0000 7fbfd6da9740 1 client.16599 shutdown 2024-01-31T05:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.503+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3141351726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0060 msgr2=0x55607cbfc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.503+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0060 0x55607cbfc450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55607cbea270 tx=0x7fbfbc011760 comp rx=0 tx=0).stop 2024-01-31T05:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.503+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3141351726 shutdown_connections 2024-01-31T05:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.503+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55607cc0b5b0 0x55607cc0d9d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.503+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 0x55607cbfced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.503+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0980 0x55607cbfc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.503+0000 7fbfb57fa640 0 client.16599 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.503+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/3141351726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0060 0x55607cbfc450 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.503+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3141351726 >> 172.21.15.45:0/3141351726 conn(0x55607cbea980 msgr2=0x55607cbec6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.503+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3141351726 shutdown_connections 2024-01-31T05:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.503+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3141351726 wait complete. 2024-01-31T05:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.503+0000 7fbfd6da9740 20 client.16599 trim_cache size 0 max 16384 2024-01-31T05:37:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.520+0000 7fbfd6da9740 1 Processor -- start 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.520+0000 7fbfd6da9740 1 -- start start 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd6da9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cc00440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd6da9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 0x55607cc00980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd6da9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 0x55607cbf4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd6da9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607cbeb780 con 0x55607cbf0980 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd6da9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607cc07360 con 0x55607cbf0060 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd6da9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607cbf8740 con 0x55607cbf9750 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd1d7d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 0x55607cc00980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd1d7d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 0x55607cc00980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41916/0 (socket says 172.21.15.45:41916) 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd1d7d640 1 -- 172.21.15.45:0/3689535616 learned_addr learned my addr 172.21.15.45:0/3689535616 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd157c640 1 --2- 172.21.15.45:0/3689535616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cc00440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd157c640 1 --2- 172.21.15.45:0/3689535616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 0x55607cbf4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd1d7d640 1 -- 172.21.15.45:0/3689535616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 msgr2=0x55607cbf4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd1d7d640 1 --2- 172.21.15.45:0/3689535616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 0x55607cbf4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd1d7d640 1 -- 172.21.15.45:0/3689535616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 msgr2=0x55607cc00440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd1d7d640 1 --2- 172.21.15.45:0/3689535616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cc00440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd1d7d640 1 -- 172.21.15.45:0/3689535616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55607caf3030 con 0x55607cbf0980 2024-01-31T05:37:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd157c640 1 --2- 172.21.15.45:0/3689535616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cc00440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:37:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfd1d7d640 1 --2- 172.21.15.45:0/3689535616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 0x55607cc00980 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8012600 tx=0x7fbfc8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cdfca749d61f95 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.521+0000 7fbfb77fe640 1 -- 172.21.15.45:0/3689535616 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8017070 con 0x55607cbf0980 2024-01-31T05:37:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.522+0000 7fbfb77fe640 1 -- 172.21.15.45:0/3689535616 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfc8018040 con 0x55607cbf0980 2024-01-31T05:37:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.522+0000 7fbfb77fe640 1 -- 172.21.15.45:0/3689535616 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8010aa0 con 0x55607cbf0980 2024-01-31T05:37:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.522+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3689535616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 msgr2=0x55607cc00980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.522+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/3689535616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 0x55607cc00980 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8012600 tx=0x7fbfc8005f90 comp rx=0 tx=0).stop 2024-01-31T05:37:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.522+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3689535616 shutdown_connections 2024-01-31T05:37:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.522+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/3689535616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf0980 0x55607cc00980 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.522+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/3689535616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf9750 0x55607cbf4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.522+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/3689535616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cc00440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.522+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3689535616 >> 172.21.15.45:0/3689535616 conn(0x55607cbea980 msgr2=0x55607cbf19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.522+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3689535616 shutdown_connections 2024-01-31T05:37:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.522+0000 7fbfd6da9740 1 -- 172.21.15.45:0/3689535616 wait complete. 2024-01-31T05:37:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd6da9740 1 -- start start 2024-01-31T05:37:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd6da9740 1 Processor -- start 2024-01-31T05:37:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd6da9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cc5b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd6da9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf0980 0x55607cc5bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd6da9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf9750 0x55607cc55d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd6da9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607cc562f0 con 0x55607cbf9750 2024-01-31T05:37:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd6da9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607cc56460 con 0x55607cbf0060 2024-01-31T05:37:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd6da9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607cc56620 con 0x55607cbf0980 2024-01-31T05:37:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd6da9740 10 client.0 _ll_register_callbacks cb 0x7fbfbc009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:37:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd1d7d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf0980 0x55607cc5bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd157c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf9750 0x55607cc55d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd1d7d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf0980 0x55607cc5bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:45728/0 (socket says 172.21.15.45:45728) 2024-01-31T05:37:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd1d7d640 1 -- 172.21.15.45:0/1872228428 learned_addr learned my addr 172.21.15.45:0/1872228428 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.523+0000 7fbfd157c640 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cc5b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfd1d7d640 1 -- 172.21.15.45:0/1872228428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 msgr2=0x55607cc5b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfd1d7d640 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cc5b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfd1d7d640 1 -- 172.21.15.45:0/1872228428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf9750 msgr2=0x55607cc55d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfd1d7d640 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf9750 0x55607cc55d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfd1d7d640 1 -- 172.21.15.45:0/1872228428 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55607cb566a0 con 0x55607cbf0980 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfd157c640 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf9750 0x55607cc55d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfd1d7d640 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf0980 0x55607cc5bd20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8024000 tx=0x7fbfc8004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb487e665ba65e23 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfb57fa640 1 -- 172.21.15.45:0/1872228428 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8017040 con 0x55607cbf0980 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfb57fa640 1 -- 172.21.15.45:0/1872228428 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbfc8010920 con 0x55607cbf0980 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfd6da9740 1 -- 172.21.15.45:0/1872228428 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfbc011180 con 0x55607cbf0980 2024-01-31T05:37:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfb57fa640 1 -- 172.21.15.45:0/1872228428 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfc8010ae0 con 0x55607cbf0980 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfd6da9740 10 client.26170 fetch_fsmap learned FSMap version 31 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfd6da9740 1 -- 172.21.15.45:0/1872228428 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fbfc8012040 con 0x55607cbf0980 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.524+0000 7fbfb57fa640 1 -- 172.21.15.45:0/1872228428 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfbc011180 con 0x55607cbf0980 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.525+0000 7fbfb57fa640 1 -- 172.21.15.45:0/1872228428 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbfc80035f0 con 0x55607cbf0980 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.525+0000 7fbfd6da9740 10 client.26170 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.525+0000 7fbfd6da9740 10 client.26170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.525+0000 7fbfd6da9740 20 client.26170 populate_metadata read hostname 'smithi045' 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.525+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55607cc63720 0x55607cc65b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.525+0000 7fbfd6da9740 4 client.26170 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.525+0000 7fbfd6da9740 1 -- 172.21.15.45:0/1872228428 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55607c857600 con 0x55607cc63720 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.525+0000 7fbfd157c640 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55607cc63720 0x55607cc65b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.525+0000 7fbfd157c640 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55607cc63720 0x55607cc65b40 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d7f0aa5293da4d server_cookie=e90bab7d56244a50 in_seq=0 out_seq=0 2024-01-31T05:37:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.525+0000 7fbfb57fa640 10 client.26170 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.578+0000 7fbfb57fa640 1 -- 172.21.15.45:0/1872228428 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55607c857600 con 0x55607cc63720 2024-01-31T05:37:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.578+0000 7fbfb57fa640 10 client.26170 handle_command_reply: tid=0 2024-01-31T05:37:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.578+0000 7fbfd6da9740 1 client.26170 shutdown 2024-01-31T05:37:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfd6da9740 1 -- 172.21.15.45:0/1872228428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf0980 msgr2=0x55607cc5bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf0980 0x55607cc5bd20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8024000 tx=0x7fbfc8004510 comp rx=0 tx=0).stop 2024-01-31T05:37:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfd6da9740 1 -- 172.21.15.45:0/1872228428 shutdown_connections 2024-01-31T05:37:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55607cc63720 0x55607cc65b40 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55607cbf9750 0x55607cc55d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55607cbf0980 0x55607cc5bd20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfd6da9740 1 --2- 172.21.15.45:0/1872228428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55607cbf0060 0x55607cc5b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfb57fa640 0 client.26170 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfd6da9740 1 -- 172.21.15.45:0/1872228428 >> 172.21.15.45:0/1872228428 conn(0x55607cbea980 msgr2=0x55607cbf4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfd6da9740 1 -- 172.21.15.45:0/1872228428 shutdown_connections 2024-01-31T05:37:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfd6da9740 1 -- 172.21.15.45:0/1872228428 wait complete. 2024-01-31T05:37:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfd6da9740 20 client.26170 trim_cache size 0 max 16384 2024-01-31T05:37:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:37:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:37:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:37:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:37:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.579+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/3861958653 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fbfb80a14f0 msgr2=0x7fbfb80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.580+0000 7fbfd3fe1640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fbfb80a14f0 0x7fbfb80a39b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40b9090 tx=0x7fbfc402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.580+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/3861958653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfcc104b90 msgr2=0x7fbfcc19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.580+0000 7fbfd3fe1640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfcc104b90 0x7fbfcc19a950 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbfc402c700 tx=0x7fbfc4032fc0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.580+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/3861958653 shutdown_connections 2024-01-31T05:37:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.580+0000 7fbfd3fe1640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fbfb80a14f0 0x7fbfb80a39b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.580+0000 7fbfd3fe1640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfcc104b90 0x7fbfcc19a950 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.580+0000 7fbfd3fe1640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfcc100800 0x7fbfcc1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.580+0000 7fbfd3fe1640 1 --2- 172.21.15.45:0/3861958653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc0ffe40 0x7fbfcc196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.580+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/3861958653 >> 172.21.15.45:0/3861958653 conn(0x7fbfcc076a40 msgr2=0x7fbfcc0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.580+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/3861958653 shutdown_connections 2024-01-31T05:37:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:35.580+0000 7fbfd3fe1640 1 -- 172.21.15.45:0/3861958653 wait complete. 2024-01-31T05:37:35.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 40180 24.94 MB/sec warmup 94 sec latency 53.520 ms 2024-01-31T05:37:35.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:35 smithi156 ceph-mon[35720]: pgmap v474: 97 pgs: 97 active+clean; 827 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 381 op/s 2024-01-31T05:37:36.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:35 smithi186 ceph-mon[35659]: pgmap v474: 97 pgs: 97 active+clean; 827 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 381 op/s 2024-01-31T05:37:36.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:35 smithi045 ceph-mon[31585]: pgmap v474: 97 pgs: 97 active+clean; 827 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 381 op/s 2024-01-31T05:37:36.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 40600 24.84 MB/sec warmup 95 sec latency 175.553 ms 2024-01-31T05:37:37.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 41160 24.79 MB/sec warmup 96 sec latency 199.716 ms 2024-01-31T05:37:37.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:37 smithi156 ceph-mon[35720]: pgmap v475: 97 pgs: 97 active+clean; 845 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.3 MiB/s wr, 536 op/s 2024-01-31T05:37:38.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:37 smithi186 ceph-mon[35659]: pgmap v475: 97 pgs: 97 active+clean; 845 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.3 MiB/s wr, 536 op/s 2024-01-31T05:37:38.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:37 smithi045 ceph-mon[31585]: pgmap v475: 97 pgs: 97 active+clean; 845 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.3 MiB/s wr, 536 op/s 2024-01-31T05:37:38.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 41721 24.90 MB/sec warmup 97 sec latency 52.971 ms 2024-01-31T05:37:39.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 42274 25.01 MB/sec warmup 98 sec latency 175.261 ms 2024-01-31T05:37:39.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:39 smithi156 ceph-mon[35720]: pgmap v476: 97 pgs: 97 active+clean; 845 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.5 MiB/s wr, 340 op/s 2024-01-31T05:37:40.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:39 smithi186 ceph-mon[35659]: pgmap v476: 97 pgs: 97 active+clean; 845 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.5 MiB/s wr, 340 op/s 2024-01-31T05:37:40.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:39 smithi045 ceph-mon[31585]: pgmap v476: 97 pgs: 97 active+clean; 845 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.5 MiB/s wr, 340 op/s 2024-01-31T05:37:40.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 42734 25.00 MB/sec warmup 99 sec latency 453.463 ms 2024-01-31T05:37:41.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 43093 24.92 MB/sec warmup 100 sec latency 427.002 ms 2024-01-31T05:37:41.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:41 smithi156 ceph-mon[35720]: pgmap v477: 97 pgs: 97 active+clean; 845 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.5 MiB/s wr, 340 op/s 2024-01-31T05:37:42.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:41 smithi186 ceph-mon[35659]: pgmap v477: 97 pgs: 97 active+clean; 845 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.5 MiB/s wr, 340 op/s 2024-01-31T05:37:42.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:41 smithi045 ceph-mon[31585]: pgmap v477: 97 pgs: 97 active+clean; 845 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.5 MiB/s wr, 340 op/s 2024-01-31T05:37:42.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 43561 25.06 MB/sec warmup 101 sec latency 190.067 ms 2024-01-31T05:37:43.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 43861 24.95 MB/sec warmup 102 sec latency 233.048 ms 2024-01-31T05:37:43.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:43 smithi156 ceph-mon[35720]: pgmap v478: 97 pgs: 97 active+clean; 859 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.2 MiB/s wr, 528 op/s 2024-01-31T05:37:44.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:43 smithi186 ceph-mon[35659]: pgmap v478: 97 pgs: 97 active+clean; 859 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.2 MiB/s wr, 528 op/s 2024-01-31T05:37:44.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:43 smithi045 ceph-mon[31585]: pgmap v478: 97 pgs: 97 active+clean; 859 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.2 MiB/s wr, 528 op/s 2024-01-31T05:37:44.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 44275 24.84 MB/sec warmup 103 sec latency 189.875 ms 2024-01-31T05:37:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 44623 24.74 MB/sec warmup 104 sec latency 154.566 ms 2024-01-31T05:37:45.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:45 smithi156 ceph-mon[35720]: pgmap v479: 97 pgs: 97 active+clean; 859 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.5 MiB/s wr, 342 op/s 2024-01-31T05:37:46.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:45 smithi186 ceph-mon[35659]: pgmap v479: 97 pgs: 97 active+clean; 859 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.5 MiB/s wr, 342 op/s 2024-01-31T05:37:46.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:45 smithi045 ceph-mon[31585]: pgmap v479: 97 pgs: 97 active+clean; 859 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.5 MiB/s wr, 342 op/s 2024-01-31T05:37:46.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 45127 24.86 MB/sec warmup 105 sec latency 61.938 ms 2024-01-31T05:37:47.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 45673 24.80 MB/sec warmup 106 sec latency 115.504 ms 2024-01-31T05:37:47.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:47 smithi156 ceph-mon[35720]: pgmap v480: 97 pgs: 97 active+clean; 876 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.1 MiB/s wr, 511 op/s 2024-01-31T05:37:48.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:47 smithi186 ceph-mon[35659]: pgmap v480: 97 pgs: 97 active+clean; 876 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.1 MiB/s wr, 511 op/s 2024-01-31T05:37:48.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:47 smithi045 ceph-mon[31585]: pgmap v480: 97 pgs: 97 active+clean; 876 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.1 MiB/s wr, 511 op/s 2024-01-31T05:37:48.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 46108 24.78 MB/sec warmup 107 sec latency 194.689 ms 2024-01-31T05:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.675+0000 7fad4493b640 1 -- 172.21.15.45:0/1799326021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad40100800 msgr2=0x7fad4010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.675+0000 7fad4493b640 1 --2- 172.21.15.45:0/1799326021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad40100800 0x7fad4010ca20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fad38009ca0 tx=0x7fad3802c0f0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.676+0000 7fad4493b640 1 -- 172.21.15.45:0/1799326021 shutdown_connections 2024-01-31T05:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.676+0000 7fad4493b640 1 --2- 172.21.15.45:0/1799326021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad40100800 0x7fad4010ca20 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.676+0000 7fad4493b640 1 --2- 172.21.15.45:0/1799326021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad400ffe40 0x7fad401002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.676+0000 7fad4493b640 1 --2- 172.21.15.45:0/1799326021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad40104b90 0x7fad40104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.676+0000 7fad4493b640 1 -- 172.21.15.45:0/1799326021 >> 172.21.15.45:0/1799326021 conn(0x7fad40076a40 msgr2=0x7fad40076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.676+0000 7fad4493b640 1 -- 172.21.15.45:0/1799326021 shutdown_connections 2024-01-31T05:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.676+0000 7fad4493b640 1 -- 172.21.15.45:0/1799326021 wait complete. 2024-01-31T05:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.676+0000 7fad4493b640 1 Processor -- start 2024-01-31T05:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.676+0000 7fad4493b640 1 -- start start 2024-01-31T05:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad4493b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad400ffe40 0x7fad40199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad4493b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad40100800 0x7fad40199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad4493b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad40104b90 0x7fad4019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad4493b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4019e2e0 con 0x7fad40104b90 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad4493b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4019e450 con 0x7fad400ffe40 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad4493b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4019e610 con 0x7fad40100800 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad3ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad40104b90 0x7fad4019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad3ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad40104b90 0x7fad4019dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43054/0 (socket says 172.21.15.45:43054) 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad40100800 0x7fad40199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad3ed9d640 1 -- 172.21.15.45:0/1116591820 learned_addr learned my addr 172.21.15.45:0/1116591820 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad3e59c640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad400ffe40 0x7fad40199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad3ed9d640 1 -- 172.21.15.45:0/1116591820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad40100800 msgr2=0x7fad40199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad3ed9d640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad40100800 0x7fad40199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad3ed9d640 1 -- 172.21.15.45:0/1116591820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad400ffe40 msgr2=0x7fad40199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad3ed9d640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad400ffe40 0x7fad40199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad3ed9d640 1 -- 172.21.15.45:0/1116591820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad38009900 con 0x7fad40104b90 2024-01-31T05:37:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad40100800 0x7fad40199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:37:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.677+0000 7fad3ed9d640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad40104b90 0x7fad4019dce0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fad3802c700 tx=0x7fad38032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.678+0000 7fad377fe640 1 -- 172.21.15.45:0/1116591820 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38032800 con 0x7fad40104b90 2024-01-31T05:37:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.678+0000 7fad377fe640 1 -- 172.21.15.45:0/1116591820 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad38003e30 con 0x7fad40104b90 2024-01-31T05:37:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.678+0000 7fad4493b640 1 -- 172.21.15.45:0/1116591820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4019e890 con 0x7fad40104b90 2024-01-31T05:37:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.678+0000 7fad377fe640 1 -- 172.21.15.45:0/1116591820 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad380047f0 con 0x7fad40104b90 2024-01-31T05:37:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.678+0000 7fad4493b640 1 -- 172.21.15.45:0/1116591820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad401a6170 con 0x7fad40104b90 2024-01-31T05:37:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.680+0000 7fad377fe640 1 -- 172.21.15.45:0/1116591820 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fad3803b040 con 0x7fad40104b90 2024-01-31T05:37:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.680+0000 7fad477d3740 1 Processor -- start 2024-01-31T05:37:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad477d3740 1 -- start start 2024-01-31T05:37:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad477d3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a060 0x5622b748a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad477d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a980 0x5622b7493210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad477d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7495b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad477d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b73f9040 con 0x5622b7493750 2024-01-31T05:37:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad477d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b73f6730 con 0x5622b748a980 2024-01-31T05:37:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad477d3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b73f2020 con 0x5622b748a060 2024-01-31T05:37:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7495b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7495b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43064/0 (socket says 172.21.15.45:43064) 2024-01-31T05:37:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad3dd9b640 1 -- 172.21.15.45:0/2625287367 learned_addr learned my addr 172.21.15.45:0/2625287367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad3e59c640 1 --2- 172.21.15.45:0/2625287367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a980 0x5622b7493210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad377fe640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fad2c0a1570 0x7fad2c0a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/2625287367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a060 0x5622b748a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.681+0000 7fad3ed9d640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fad2c0a1570 0x7fad2c0a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.682+0000 7fad3dd9b640 1 -- 172.21.15.45:0/2625287367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a060 msgr2=0x5622b748a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.682+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/2625287367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a060 0x5622b748a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.682+0000 7fad3dd9b640 1 -- 172.21.15.45:0/2625287367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a980 msgr2=0x5622b7493210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.682+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/2625287367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a980 0x5622b7493210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.682+0000 7fad3dd9b640 1 -- 172.21.15.45:0/2625287367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622b73e9e40 con 0x5622b7493750 2024-01-31T05:37:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.682+0000 7fad3e59c640 1 --2- 172.21.15.45:0/2625287367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a980 0x5622b7493210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:37:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.682+0000 7fad377fe640 1 -- 172.21.15.45:0/1116591820 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fad380ef5e0 con 0x7fad40104b90 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.682+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/2625287367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7495b40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fad28004650 tx=0x7fad28009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57888c6fad5748c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.682+0000 7fad3ed9d640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fad2c0a1570 0x7fad2c0a3a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fad380b8aa0 tx=0x7fad3804de10 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.682+0000 7fad34ff9640 1 -- 172.21.15.45:0/2625287367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2800fc50 con 0x5622b7493750 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.682+0000 7fad34ff9640 1 -- 172.21.15.45:0/2625287367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad2800fdb0 con 0x5622b7493750 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.682+0000 7fad34ff9640 1 -- 172.21.15.45:0/2625287367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2800e5d0 con 0x5622b7493750 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.683+0000 7fad477d3740 1 -- 172.21.15.45:0/2625287367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 msgr2=0x5622b7495b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.683+0000 7fad477d3740 1 --2- 172.21.15.45:0/2625287367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7495b40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fad28004650 tx=0x7fad28009b80 comp rx=0 tx=0).stop 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.683+0000 7fad477d3740 1 -- 172.21.15.45:0/2625287367 shutdown_connections 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.683+0000 7fad477d3740 1 --2- 172.21.15.45:0/2625287367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7495b40 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.683+0000 7fad477d3740 1 --2- 172.21.15.45:0/2625287367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a980 0x5622b7493210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.683+0000 7fad477d3740 1 --2- 172.21.15.45:0/2625287367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a060 0x5622b748a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.683+0000 7fad477d3740 1 -- 172.21.15.45:0/2625287367 >> 172.21.15.45:0/2625287367 conn(0x5622b7484980 msgr2=0x5622b7484d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.683+0000 7fad477d3740 1 -- 172.21.15.45:0/2625287367 shutdown_connections 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.683+0000 7fad477d3740 1 -- 172.21.15.45:0/2625287367 wait complete. 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.683+0000 7fad477d3740 1 -- start start 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.683+0000 7fad477d3740 1 Processor -- start 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad477d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a060 0x5622b7490800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad477d3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a980 0x5622b7490d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad477d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7491280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad477d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b73ab1b0 con 0x5622b7493750 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad477d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b73aae50 con 0x5622b748a060 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad477d3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b7370090 con 0x5622b748a980 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad477d3740 10 client.0 _ll_register_callbacks cb 0x5622b7395a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a060 0x5622b7490800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad3e59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a980 0x5622b7490d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7491280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad3e59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a980 0x5622b7490d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:45352/0 (socket says 172.21.15.45:45352) 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad3e59c640 1 -- 172.21.15.45:0/3761839170 learned_addr learned my addr 172.21.15.45:0/3761839170 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad3dd9b640 1 -- 172.21.15.45:0/3761839170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a980 msgr2=0x5622b7490d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/3761839170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a980 0x5622b7490d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad3dd9b640 1 -- 172.21.15.45:0/3761839170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a060 msgr2=0x5622b7490800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/3761839170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a060 0x5622b7490800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad3dd9b640 1 -- 172.21.15.45:0/3761839170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad28019040 con 0x5622b7493750 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/3761839170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7491280 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fad28001eb0 tx=0x7fad28005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5231bbbd3573568a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad25ffb640 1 -- 172.21.15.45:0/3761839170 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28005520 con 0x5622b7493750 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad25ffb640 1 -- 172.21.15.45:0/3761839170 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fad2801f440 con 0x5622b7493750 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.684+0000 7fad477d3740 1 -- 172.21.15.45:0/3761839170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622b73eb010 con 0x5622b7493750 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.685+0000 7fad25ffb640 1 -- 172.21.15.45:0/3761839170 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad2801f600 con 0x5622b7493750 2024-01-31T05:37:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.685+0000 7fad477d3740 10 client.16617 fetch_fsmap learned FSMap version 31 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.685+0000 7fad477d3740 1 -- 172.21.15.45:0/3761839170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5622b73f06a0 con 0x5622b7493750 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.685+0000 7fad25ffb640 1 -- 172.21.15.45:0/3761839170 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5622b73eb010 con 0x5622b7493750 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.685+0000 7fad25ffb640 1 -- 172.21.15.45:0/3761839170 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fad2802e450 con 0x5622b7493750 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.685+0000 7fad477d3740 10 client.16617 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.685+0000 7fad477d3740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.685+0000 7fad477d3740 20 client.16617 populate_metadata read hostname 'smithi045' 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.685+0000 7fad477d3740 1 --2- 172.21.15.45:0/3761839170 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5622b74a34a0 0x5622b74a5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.685+0000 7fad477d3740 4 client.16617 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.685+0000 7fad477d3740 1 -- 172.21.15.45:0/3761839170 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622b73ead30 con 0x5622b74a34a0 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.685+0000 7fad3e59c640 1 --2- 172.21.15.45:0/3761839170 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5622b74a34a0 0x5622b74a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad25ffb640 10 client.16617 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad3e59c640 1 --2- 172.21.15.45:0/3761839170 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5622b74a34a0 0x5622b74a5890 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0d0e8416371de6 server_cookie=dc80b99c9c34710e in_seq=0 out_seq=0 2024-01-31T05:37:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad25ffb640 1 -- 172.21.15.45:0/3761839170 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5622b73ead30 con 0x5622b74a34a0 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad25ffb640 10 client.16617 handle_command_reply: tid=0 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad477d3740 1 client.16617 shutdown 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad477d3740 1 -- 172.21.15.45:0/3761839170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 msgr2=0x5622b7491280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad477d3740 1 --2- 172.21.15.45:0/3761839170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7491280 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fad28001eb0 tx=0x7fad28005f90 comp rx=0 tx=0).stop 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad477d3740 1 -- 172.21.15.45:0/3761839170 shutdown_connections 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad477d3740 1 --2- 172.21.15.45:0/3761839170 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5622b74a34a0 0x5622b74a5890 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad477d3740 1 --2- 172.21.15.45:0/3761839170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7491280 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad477d3740 1 --2- 172.21.15.45:0/3761839170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a980 0x5622b7490d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad477d3740 1 --2- 172.21.15.45:0/3761839170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a060 0x5622b7490800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad25ffb640 0 client.16617 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad477d3740 1 -- 172.21.15.45:0/3761839170 >> 172.21.15.45:0/3761839170 conn(0x5622b7484980 msgr2=0x5622b74866d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad477d3740 1 -- 172.21.15.45:0/3761839170 shutdown_connections 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.686+0000 7fad477d3740 1 -- 172.21.15.45:0/3761839170 wait complete. 2024-01-31T05:37:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.687+0000 7fad477d3740 20 client.16617 trim_cache size 0 max 16384 2024-01-31T05:37:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad477d3740 1 Processor -- start 2024-01-31T05:37:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad477d3740 1 -- start start 2024-01-31T05:37:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad477d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a060 0x5622b7492f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad477d3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a980 0x5622b748f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad477d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b748da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad477d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b74802b0 con 0x5622b7493750 2024-01-31T05:37:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad477d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b74a0160 con 0x5622b748a060 2024-01-31T05:37:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad477d3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b747eee0 con 0x5622b748a980 2024-01-31T05:37:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a060 0x5622b7492f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad3e59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a980 0x5622b748f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a060 0x5622b7492f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:35648/0 (socket says 172.21.15.45:35648) 2024-01-31T05:37:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad3e59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a980 0x5622b748f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:45366/0 (socket says 172.21.15.45:45366) 2024-01-31T05:37:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad3dd9b640 1 -- 172.21.15.45:0/559542272 learned_addr learned my addr 172.21.15.45:0/559542272 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/559542272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b748da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad3dd9b640 1 -- 172.21.15.45:0/559542272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a980 msgr2=0x5622b748f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/559542272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a980 0x5622b748f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad3dd9b640 1 -- 172.21.15.45:0/559542272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 msgr2=0x5622b748da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/559542272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b748da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.702+0000 7fad3dd9b640 1 -- 172.21.15.45:0/559542272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622b738d030 con 0x5622b748a060 2024-01-31T05:37:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.703+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/559542272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a060 0x5622b7492f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad2801fee0 tx=0x7fad28005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc9ec6dfc5aceb6b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.703+0000 7fad27fff640 1 -- 172.21.15.45:0/559542272 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2800fc20 con 0x5622b748a060 2024-01-31T05:37:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.703+0000 7fad27fff640 1 -- 172.21.15.45:0/559542272 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad28005c50 con 0x5622b748a060 2024-01-31T05:37:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.703+0000 7fad27fff640 1 -- 172.21.15.45:0/559542272 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28005650 con 0x5622b748a060 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 -- 172.21.15.45:0/559542272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a060 msgr2=0x5622b7492f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 --2- 172.21.15.45:0/559542272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a060 0x5622b7492f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad2801fee0 tx=0x7fad28005970 comp rx=0 tx=0).stop 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 -- 172.21.15.45:0/559542272 shutdown_connections 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 --2- 172.21.15.45:0/559542272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b748da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 --2- 172.21.15.45:0/559542272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a980 0x5622b748f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 --2- 172.21.15.45:0/559542272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a060 0x5622b7492f30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 -- 172.21.15.45:0/559542272 >> 172.21.15.45:0/559542272 conn(0x5622b7484980 msgr2=0x5622b748f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 -- 172.21.15.45:0/559542272 shutdown_connections 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 -- 172.21.15.45:0/559542272 wait complete. 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 -- start start 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 Processor -- start 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a060 0x5622b749a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a980 0x5622b74989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7498f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b7499460 con 0x5622b7493750 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b74995d0 con 0x5622b748a980 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b7499790 con 0x5622b748a060 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad477d3740 10 client.0 _ll_register_callbacks cb 0x7fad28009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7498f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.704+0000 7fad3e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a980 0x5622b74989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7498f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43086/0 (socket says 172.21.15.45:43086) 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad3dd9b640 1 -- 172.21.15.45:0/4114263033 learned_addr learned my addr 172.21.15.45:0/4114263033 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a060 0x5622b749a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad3dd9b640 1 -- 172.21.15.45:0/4114263033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a060 msgr2=0x5622b749a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a060 0x5622b749a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad3dd9b640 1 -- 172.21.15.45:0/4114263033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a980 msgr2=0x5622b74989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:37:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a980 0x5622b74989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad3dd9b640 1 -- 172.21.15.45:0/4114263033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622b73f06a0 con 0x5622b7493750 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad3e59c640 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a980 0x5622b74989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad3dd9b640 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7498f20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fad2800fc60 tx=0x7fad28005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540e5f76270256ed server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad25ffb640 1 -- 172.21.15.45:0/4114263033 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2801f440 con 0x5622b7493750 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad25ffb640 1 -- 172.21.15.45:0/4114263033 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fad28000e40 con 0x5622b7493750 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad477d3740 1 -- 172.21.15.45:0/4114263033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad2800fdb0 con 0x5622b7493750 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad25ffb640 1 -- 172.21.15.45:0/4114263033 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad2801f5a0 con 0x5622b7493750 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad477d3740 10 client.16623 fetch_fsmap learned FSMap version 31 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.705+0000 7fad477d3740 1 -- 172.21.15.45:0/4114263033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5622b73b2a50 con 0x5622b7493750 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.706+0000 7fad25ffb640 1 -- 172.21.15.45:0/4114263033 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad2800fdb0 con 0x5622b7493750 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.706+0000 7fad25ffb640 1 -- 172.21.15.45:0/4114263033 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fad28003990 con 0x5622b7493750 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.706+0000 7fad477d3740 10 client.16623 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.706+0000 7fad477d3740 10 client.16623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:37:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.706+0000 7fad477d3740 20 client.16623 populate_metadata read hostname 'smithi045' 2024-01-31T05:37:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.706+0000 7fad477d3740 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5622b74faa50 0x5622b74fce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:37:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.706+0000 7fad477d3740 4 client.16623 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:37:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.706+0000 7fad477d3740 1 -- 172.21.15.45:0/4114263033 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5622b70f1600 con 0x5622b74faa50 2024-01-31T05:37:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.706+0000 7fad3e59c640 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5622b74faa50 0x5622b74fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:37:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.706+0000 7fad3e59c640 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5622b74faa50 0x5622b74fce70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d44fb3602b4e29 server_cookie=ff4127511ff7ea06 in_seq=0 out_seq=0 2024-01-31T05:37:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.706+0000 7fad25ffb640 10 client.16623 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.766+0000 7fad25ffb640 1 -- 172.21.15.45:0/4114263033 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5622b70f1600 con 0x5622b74faa50 2024-01-31T05:37:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad25ffb640 10 client.16623 handle_command_reply: tid=0 2024-01-31T05:37:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad477d3740 1 client.16623 shutdown 2024-01-31T05:37:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad477d3740 1 -- 172.21.15.45:0/4114263033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 msgr2=0x5622b7498f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad477d3740 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7498f20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fad2800fc60 tx=0x7fad28005550 comp rx=0 tx=0).stop 2024-01-31T05:37:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad477d3740 1 -- 172.21.15.45:0/4114263033 shutdown_connections 2024-01-31T05:37:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad477d3740 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x5622b74faa50 0x5622b74fce70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad25ffb640 0 client.16623 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:37:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad477d3740 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622b7493750 0x5622b7498f20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad477d3740 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622b748a980 0x5622b74989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad477d3740 1 --2- 172.21.15.45:0/4114263033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5622b748a060 0x5622b749a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad477d3740 1 -- 172.21.15.45:0/4114263033 >> 172.21.15.45:0/4114263033 conn(0x5622b7484980 msgr2=0x5622b7483190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad477d3740 1 -- 172.21.15.45:0/4114263033 shutdown_connections 2024-01-31T05:37:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad477d3740 1 -- 172.21.15.45:0/4114263033 wait complete. 2024-01-31T05:37:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad477d3740 20 client.16623 trim_cache size 0 max 16384 2024-01-31T05:37:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:37:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:37:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:37:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:37:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad4493b640 1 -- 172.21.15.45:0/1116591820 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fad2c0a1570 msgr2=0x7fad2c0a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad4493b640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fad2c0a1570 0x7fad2c0a3a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fad380b8aa0 tx=0x7fad3804de10 comp rx=0 tx=0).stop 2024-01-31T05:37:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad4493b640 1 -- 172.21.15.45:0/1116591820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad40104b90 msgr2=0x7fad4019dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:37:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.767+0000 7fad4493b640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad40104b90 0x7fad4019dce0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fad3802c700 tx=0x7fad38032fc0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.768+0000 7fad4493b640 1 -- 172.21.15.45:0/1116591820 shutdown_connections 2024-01-31T05:37:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.768+0000 7fad4493b640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fad2c0a1570 0x7fad2c0a3a30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.768+0000 7fad4493b640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad40104b90 0x7fad4019dce0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.768+0000 7fad4493b640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad40100800 0x7fad40199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.768+0000 7fad4493b640 1 --2- 172.21.15.45:0/1116591820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad400ffe40 0x7fad40199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:37:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.768+0000 7fad4493b640 1 -- 172.21.15.45:0/1116591820 >> 172.21.15.45:0/1116591820 conn(0x7fad40076a40 msgr2=0x7fad4010b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:37:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.768+0000 7fad4493b640 1 -- 172.21.15.45:0/1116591820 shutdown_connections 2024-01-31T05:37:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:37:48.768+0000 7fad4493b640 1 -- 172.21.15.45:0/1116591820 wait complete. 2024-01-31T05:37:49.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 46787 24.93 MB/sec warmup 108 sec latency 187.136 ms 2024-01-31T05:37:49.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:49 smithi156 ceph-mon[35720]: pgmap v481: 97 pgs: 97 active+clean; 876 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.4 MiB/s wr, 356 op/s 2024-01-31T05:37:50.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:49 smithi186 ceph-mon[35659]: pgmap v481: 97 pgs: 97 active+clean; 876 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.4 MiB/s wr, 356 op/s 2024-01-31T05:37:50.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:49 smithi045 ceph-mon[31585]: pgmap v481: 97 pgs: 97 active+clean; 876 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.4 MiB/s wr, 356 op/s 2024-01-31T05:37:50.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 47241 25.00 MB/sec warmup 109 sec latency 419.212 ms 2024-01-31T05:37:51.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 47551 24.93 MB/sec warmup 110 sec latency 431.697 ms 2024-01-31T05:37:51.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:51 smithi156 ceph-mon[35720]: pgmap v482: 97 pgs: 97 active+clean; 876 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.4 MiB/s wr, 356 op/s 2024-01-31T05:37:52.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:51 smithi186 ceph-mon[35659]: pgmap v482: 97 pgs: 97 active+clean; 876 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.4 MiB/s wr, 356 op/s 2024-01-31T05:37:52.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:51 smithi045 ceph-mon[31585]: pgmap v482: 97 pgs: 97 active+clean; 876 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.4 MiB/s wr, 356 op/s 2024-01-31T05:37:52.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 47940 24.90 MB/sec warmup 111 sec latency 195.684 ms 2024-01-31T05:37:53.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 48386 24.89 MB/sec warmup 112 sec latency 55.718 ms 2024-01-31T05:37:53.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:53 smithi156 ceph-mon[35720]: pgmap v483: 97 pgs: 97 active+clean; 885 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.9 MiB/s wr, 578 op/s 2024-01-31T05:37:54.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:53 smithi186 ceph-mon[35659]: pgmap v483: 97 pgs: 97 active+clean; 885 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.9 MiB/s wr, 578 op/s 2024-01-31T05:37:54.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:53 smithi045 ceph-mon[31585]: pgmap v483: 97 pgs: 97 active+clean; 885 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.9 MiB/s wr, 578 op/s 2024-01-31T05:37:54.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 48783 24.86 MB/sec warmup 113 sec latency 68.644 ms 2024-01-31T05:37:55.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 49066 24.75 MB/sec warmup 114 sec latency 250.437 ms 2024-01-31T05:37:55.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:55 smithi156 ceph-mon[35720]: pgmap v484: 97 pgs: 97 active+clean; 885 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.2 MiB/s wr, 390 op/s 2024-01-31T05:37:56.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:55 smithi186 ceph-mon[35659]: pgmap v484: 97 pgs: 97 active+clean; 885 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.2 MiB/s wr, 390 op/s 2024-01-31T05:37:56.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:55 smithi045 ceph-mon[31585]: pgmap v484: 97 pgs: 97 active+clean; 885 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.2 MiB/s wr, 390 op/s 2024-01-31T05:37:56.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 49540 24.72 MB/sec warmup 115 sec latency 265.205 ms 2024-01-31T05:37:57.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 50177 24.69 MB/sec warmup 116 sec latency 115.271 ms 2024-01-31T05:37:57.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:57 smithi156 ceph-mon[35720]: pgmap v485: 97 pgs: 97 active+clean; 904 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 567 op/s 2024-01-31T05:37:58.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:57 smithi186 ceph-mon[35659]: pgmap v485: 97 pgs: 97 active+clean; 904 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 567 op/s 2024-01-31T05:37:58.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:57 smithi045 ceph-mon[31585]: pgmap v485: 97 pgs: 97 active+clean; 904 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 567 op/s 2024-01-31T05:37:58.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 50742 24.87 MB/sec warmup 117 sec latency 174.329 ms 2024-01-31T05:37:59.646 INFO:tasks.workunit.client.0.smithi045.stdout: 10 51245 24.89 MB/sec warmup 118 sec latency 193.027 ms 2024-01-31T05:37:59.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:37:59 smithi156 ceph-mon[35720]: pgmap v486: 97 pgs: 97 active+clean; 904 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.7 MiB/s wr, 397 op/s 2024-01-31T05:38:00.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:37:59 smithi186 ceph-mon[35659]: pgmap v486: 97 pgs: 97 active+clean; 904 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.7 MiB/s wr, 397 op/s 2024-01-31T05:38:00.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:37:59 smithi045 ceph-mon[31585]: pgmap v486: 97 pgs: 97 active+clean; 904 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.7 MiB/s wr, 397 op/s 2024-01-31T05:38:00.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 51608 24.86 MB/sec warmup 119 sec latency 457.356 ms 2024-01-31T05:38:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.901+0000 7f5e2e7fc640 1 -- 172.21.15.45:0/606243718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2002cb70 con 0x7f5e30104b90 2024-01-31T05:38:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.902+0000 7f5e3611f640 1 -- 172.21.15.45:0/606243718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e30104b90 msgr2=0x7f5e30104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.902+0000 7f5e3611f640 1 --2- 172.21.15.45:0/606243718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e30104b90 0x7f5e30104f70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009900 tx=0x7f5e2002bc30 comp rx=0 tx=0).stop 2024-01-31T05:38:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.902+0000 7f5e3611f640 1 -- 172.21.15.45:0/606243718 shutdown_connections 2024-01-31T05:38:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.902+0000 7f5e3611f640 1 --2- 172.21.15.45:0/606243718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e30100800 0x7f5e3010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.902+0000 7f5e3611f640 1 --2- 172.21.15.45:0/606243718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e300ffe40 0x7f5e301002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.902+0000 7f5e3611f640 1 --2- 172.21.15.45:0/606243718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e30104b90 0x7f5e30104f70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.902+0000 7f5e3611f640 1 -- 172.21.15.45:0/606243718 >> 172.21.15.45:0/606243718 conn(0x7f5e30076a40 msgr2=0x7f5e30076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.902+0000 7f5e3611f640 1 -- 172.21.15.45:0/606243718 shutdown_connections 2024-01-31T05:38:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.902+0000 7f5e3611f640 1 -- 172.21.15.45:0/606243718 wait complete. 2024-01-31T05:38:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.903+0000 7f5e3611f640 1 Processor -- start 2024-01-31T05:38:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.904+0000 7f5e3611f640 1 -- start start 2024-01-31T05:38:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.904+0000 7f5e3611f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e300ffe40 0x7f5e30197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.904+0000 7f5e3611f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e30100800 0x7f5e301977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.904+0000 7f5e3611f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e30104b90 0x7f5e3019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.904+0000 7f5e3611f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3019c0f0 con 0x7f5e30104b90 2024-01-31T05:38:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.904+0000 7f5e3611f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3019c260 con 0x7f5e300ffe40 2024-01-31T05:38:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.905+0000 7f5e3611f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3019c420 con 0x7f5e30100800 2024-01-31T05:38:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.905+0000 7f5e2effd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e30100800 0x7f5e301977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.905+0000 7f5e2f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e300ffe40 0x7f5e30197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.905+0000 7f5e2ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e30104b90 0x7f5e3019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.905+0000 7f5e2f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e300ffe40 0x7f5e30197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:57516/0 (socket says 172.21.15.45:57516) 2024-01-31T05:38:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.905+0000 7f5e2f7fe640 1 -- 172.21.15.45:0/2292775879 learned_addr learned my addr 172.21.15.45:0/2292775879 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.905+0000 7f5e2effd640 1 -- 172.21.15.45:0/2292775879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e300ffe40 msgr2=0x7f5e30197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.905+0000 7f5e2effd640 1 --2- 172.21.15.45:0/2292775879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e300ffe40 0x7f5e30197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.905+0000 7f5e2effd640 1 -- 172.21.15.45:0/2292775879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e30104b90 msgr2=0x7f5e3019baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.905+0000 7f5e2effd640 1 --2- 172.21.15.45:0/2292775879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e30104b90 0x7f5e3019baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.905+0000 7f5e2effd640 1 -- 172.21.15.45:0/2292775879 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e20009560 con 0x7f5e30100800 2024-01-31T05:38:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.906+0000 7f5e2effd640 1 --2- 172.21.15.45:0/2292775879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e30100800 0x7f5e301977c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400e9f0 tx=0x7f5e2400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.906+0000 7f5e2cff9640 1 -- 172.21.15.45:0/2292775879 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400ccc0 con 0x7f5e30100800 2024-01-31T05:38:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.907+0000 7f5e2cff9640 1 -- 172.21.15.45:0/2292775879 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e2400ce20 con 0x7f5e30100800 2024-01-31T05:38:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.907+0000 7f5e3611f640 1 -- 172.21.15.45:0/2292775879 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3019c700 con 0x7f5e30100800 2024-01-31T05:38:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.907+0000 7f5e3611f640 1 -- 172.21.15.45:0/2292775879 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e301a3f80 con 0x7f5e30100800 2024-01-31T05:38:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.907+0000 7f5e2cff9640 1 -- 172.21.15.45:0/2292775879 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24004a00 con 0x7f5e30100800 2024-01-31T05:38:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.910+0000 7f5e38fd8740 1 Processor -- start 2024-01-31T05:38:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.911+0000 7f5e38fd8740 1 -- start start 2024-01-31T05:38:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.911+0000 7f5e2cff9640 1 -- 172.21.15.45:0/2292775879 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f5e24005020 con 0x7f5e30100800 2024-01-31T05:38:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.911+0000 7f5e38fd8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966afa96b0 0x55966afa9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.911+0000 7f5e38fd8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97410 0x55966af97890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.911+0000 7f5e38fd8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966af97dd0 0x55966afb1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.911+0000 7f5e38fd8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966af0c040 con 0x55966af97410 2024-01-31T05:38:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.911+0000 7f5e38fd8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966af09730 con 0x55966afa96b0 2024-01-31T05:38:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.911+0000 7f5e38fd8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966af05020 con 0x55966af97dd0 2024-01-31T05:38:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.911+0000 7f5e2ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966afa96b0 0x55966afa9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.911+0000 7f5e2ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966afa96b0 0x55966afa9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:57532/0 (socket says 172.21.15.45:57532) 2024-01-31T05:38:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.911+0000 7f5e2ffff640 1 -- 172.21.15.45:0/1558395593 learned_addr learned my addr 172.21.15.45:0/1558395593 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.912+0000 7f5e2effd640 1 --2- 172.21.15.45:0/1558395593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966af97dd0 0x55966afb1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.912+0000 7f5e2ffff640 1 -- 172.21.15.45:0/1558395593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966af97dd0 msgr2=0x55966afb1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.912+0000 7f5e2ffff640 1 --2- 172.21.15.45:0/1558395593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966af97dd0 0x55966afb1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.912+0000 7f5e2ffff640 1 -- 172.21.15.45:0/1558395593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97410 msgr2=0x55966af97890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.912+0000 7f5e2ffff640 1 --2- 172.21.15.45:0/1558395593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97410 0x55966af97890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.912+0000 7f5e2ffff640 1 -- 172.21.15.45:0/1558395593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55966aefce40 con 0x55966afa96b0 2024-01-31T05:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.912+0000 7f5e2effd640 1 --2- 172.21.15.45:0/1558395593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966af97dd0 0x55966afb1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.913+0000 7f5e2ffff640 1 --2- 172.21.15.45:0/1558395593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966afa96b0 0x55966afa9a90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009fc0 tx=0x7f5e280118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ac87eccb6d9a69 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.913+0000 7f5e2cff9640 1 --2- 172.21.15.45:0/2292775879 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5e180a1650 0x7f5e180a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.913+0000 7f5e15ffb640 1 -- 172.21.15.45:0/1558395593 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28004d20 con 0x55966afa96b0 2024-01-31T05:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.913+0000 7f5e15ffb640 1 -- 172.21.15.45:0/1558395593 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e28004e80 con 0x55966afa96b0 2024-01-31T05:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.913+0000 7f5e15ffb640 1 -- 172.21.15.45:0/1558395593 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28005280 con 0x55966afa96b0 2024-01-31T05:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.914+0000 7f5e2f7fe640 1 --2- 172.21.15.45:0/2292775879 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5e180a1650 0x7f5e180a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.914+0000 7f5e2cff9640 1 -- 172.21.15.45:0/2292775879 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5e24014070 con 0x7f5e30100800 2024-01-31T05:38:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.914+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1558395593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966afa96b0 msgr2=0x55966afa9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.914+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/1558395593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966afa96b0 0x55966afa9a90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009fc0 tx=0x7f5e280118d0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.914+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1558395593 shutdown_connections 2024-01-31T05:38:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.914+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/1558395593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97410 0x55966af97890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.914+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/1558395593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966af97dd0 0x55966afb1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.914+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/1558395593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966afa96b0 0x55966afa9a90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.914+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1558395593 >> 172.21.15.45:0/1558395593 conn(0x55966af99ed0 msgr2=0x55966af9a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.915+0000 7f5e2f7fe640 1 --2- 172.21.15.45:0/2292775879 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5e180a1650 0x7f5e180a3b10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5e20000c00 tx=0x7f5e20005e20 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.915+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1558395593 shutdown_connections 2024-01-31T05:38:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.915+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1558395593 wait complete. 2024-01-31T05:38:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.916+0000 7f5e38fd8740 1 -- start start 2024-01-31T05:38:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.916+0000 7f5e38fd8740 1 Processor -- start 2024-01-31T05:38:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.916+0000 7f5e38fd8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966af97410 0x55966afae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.916+0000 7f5e38fd8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 0x55966afaeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.916+0000 7f5e38fd8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966afa96b0 0x55966afaf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.916+0000 7f5e38fd8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966aebe1b0 con 0x55966afa96b0 2024-01-31T05:38:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.916+0000 7f5e38fd8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966aebde50 con 0x55966af97dd0 2024-01-31T05:38:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.916+0000 7f5e38fd8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966ae83090 con 0x55966af97410 2024-01-31T05:38:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.916+0000 7f5e38fd8740 10 client.0 _ll_register_callbacks cb 0x55966aea8a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:38:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.917+0000 7f5e2effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 0x55966afaeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.917+0000 7f5e2effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 0x55966afaeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:57536/0 (socket says 172.21.15.45:57536) 2024-01-31T05:38:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.917+0000 7f5e2effd640 1 -- 172.21.15.45:0/1909690437 learned_addr learned my addr 172.21.15.45:0/1909690437 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.917+0000 7f5e2ffff640 1 --2- 172.21.15.45:0/1909690437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966afa96b0 0x55966afaf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.918+0000 7f5e2effd640 1 -- 172.21.15.45:0/1909690437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966af97410 msgr2=0x55966afae9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-31T05:38:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.918+0000 7f5e2effd640 1 --2- 172.21.15.45:0/1909690437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966af97410 0x55966afae9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.918+0000 7f5e2effd640 1 -- 172.21.15.45:0/1909690437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966afa96b0 msgr2=0x55966afaf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.918+0000 7f5e2effd640 1 --2- 172.21.15.45:0/1909690437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966afa96b0 0x55966afaf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.918+0000 7f5e2effd640 1 -- 172.21.15.45:0/1909690437 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e28009c70 con 0x55966af97dd0 2024-01-31T05:38:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.918+0000 7f5e2effd640 1 --2- 172.21.15.45:0/1909690437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 0x55966afaeef0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400e9c0 tx=0x7f5e24095f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adf43384bef6716c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.918+0000 7f5e0effd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:38:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.918+0000 7f5e0effd640 1 -- 172.21.15.45:0/1909690437 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e240992c0 con 0x55966af97dd0 2024-01-31T05:38:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.918+0000 7f5e0effd640 1 -- 172.21.15.45:0/1909690437 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5e2409a2d0 con 0x55966af97dd0 2024-01-31T05:38:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.919+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1909690437 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55966aefdd30 con 0x55966af97dd0 2024-01-31T05:38:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.919+0000 7f5e0effd640 1 -- 172.21.15.45:0/1909690437 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e24099530 con 0x55966af97dd0 2024-01-31T05:38:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.919+0000 7f5e38fd8740 10 client.26129 fetch_fsmap learned FSMap version 31 2024-01-31T05:38:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.919+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1909690437 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55966af036a0 con 0x55966af97dd0 2024-01-31T05:38:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.920+0000 7f5e0effd640 1 -- 172.21.15.45:0/1909690437 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55966aefdd30 con 0x55966af97dd0 2024-01-31T05:38:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.920+0000 7f5e0effd640 1 -- 172.21.15.45:0/1909690437 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5e240a1ad0 con 0x55966af97dd0 2024-01-31T05:38:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.920+0000 7f5e38fd8740 10 client.26129 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:38:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.920+0000 7f5e38fd8740 10 client.26129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:38:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.920+0000 7f5e38fd8740 20 client.26129 populate_metadata read hostname 'smithi045' 2024-01-31T05:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.920+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/1909690437 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55966af9f610 0x55966afa1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.920+0000 7f5e38fd8740 4 client.26129 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.920+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1909690437 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55966ac04600 con 0x55966af9f610 2024-01-31T05:38:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.921+0000 7f5e2f7fe640 1 --2- 172.21.15.45:0/1909690437 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55966af9f610 0x55966afa1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.921+0000 7f5e2f7fe640 1 --2- 172.21.15.45:0/1909690437 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55966af9f610 0x55966afa1a30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83219220157f3e3 server_cookie=24b8f2174158689d in_seq=0 out_seq=0 2024-01-31T05:38:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.921+0000 7f5e0effd640 10 client.26129 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.922+0000 7f5e0effd640 1 -- 172.21.15.45:0/1909690437 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55966ac04600 con 0x55966af9f610 2024-01-31T05:38:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.922+0000 7f5e0effd640 10 client.26129 handle_command_reply: tid=0 2024-01-31T05:38:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.922+0000 7f5e38fd8740 1 client.26129 shutdown 2024-01-31T05:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.923+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1909690437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 msgr2=0x55966afaeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.923+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/1909690437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 0x55966afaeef0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400e9c0 tx=0x7f5e24095f10 comp rx=0 tx=0).stop 2024-01-31T05:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.923+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1909690437 shutdown_connections 2024-01-31T05:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.923+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/1909690437 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55966af9f610 0x55966afa1a30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.923+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/1909690437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966afa96b0 0x55966afaf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.923+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/1909690437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 0x55966afaeef0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.923+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/1909690437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966af97410 0x55966afae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.923+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1909690437 >> 172.21.15.45:0/1909690437 conn(0x55966af99ed0 msgr2=0x55966af99630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.923+0000 7f5e0effd640 0 client.26129 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.924+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1909690437 shutdown_connections 2024-01-31T05:38:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.924+0000 7f5e38fd8740 1 -- 172.21.15.45:0/1909690437 wait complete. 2024-01-31T05:38:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.924+0000 7f5e38fd8740 20 client.26129 trim_cache size 0 max 16384 2024-01-31T05:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.950+0000 7f5e38fd8740 1 Processor -- start 2024-01-31T05:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.950+0000 7f5e38fd8740 1 -- start start 2024-01-31T05:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.950+0000 7f5e38fd8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97410 0x55966afb10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.950+0000 7f5e38fd8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97dd0 0x55966af92c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e38fd8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966afa96b0 0x55966afa6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e38fd8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966af9afd0 con 0x55966af97dd0 2024-01-31T05:38:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e38fd8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966afacf00 con 0x55966af97410 2024-01-31T05:38:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e38fd8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966afa7510 con 0x55966afa96b0 2024-01-31T05:38:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97410 0x55966afb10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97410 0x55966afb10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:57538/0 (socket says 172.21.15.45:57538) 2024-01-31T05:38:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2f7fe640 1 -- 172.21.15.45:0/2928489727 learned_addr learned my addr 172.21.15.45:0/2928489727 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2effd640 1 --2- 172.21.15.45:0/2928489727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97dd0 0x55966af92c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2ffff640 1 --2- 172.21.15.45:0/2928489727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966afa96b0 0x55966afa6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2f7fe640 1 -- 172.21.15.45:0/2928489727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966afa96b0 msgr2=0x55966afa6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2f7fe640 1 --2- 172.21.15.45:0/2928489727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966afa96b0 0x55966afa6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2f7fe640 1 -- 172.21.15.45:0/2928489727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97dd0 msgr2=0x55966af92c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2f7fe640 1 --2- 172.21.15.45:0/2928489727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97dd0 0x55966af92c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2f7fe640 1 -- 172.21.15.45:0/2928489727 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55966aea0030 con 0x55966af97410 2024-01-31T05:38:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2effd640 1 --2- 172.21.15.45:0/2928489727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97dd0 0x55966af92c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:38:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2ffff640 1 --2- 172.21.15.45:0/2928489727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966afa96b0 0x55966afa6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:38:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.951+0000 7f5e2f7fe640 1 --2- 172.21.15.45:0/2928489727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97410 0x55966afb10e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5e20005e90 tx=0x7f5e2003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d03f9e12c056192 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.952+0000 7f5e157fa640 1 -- 172.21.15.45:0/2928489727 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2002c870 con 0x55966af97410 2024-01-31T05:38:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.952+0000 7f5e157fa640 1 -- 172.21.15.45:0/2928489727 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e20004500 con 0x55966af97410 2024-01-31T05:38:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.952+0000 7f5e157fa640 1 -- 172.21.15.45:0/2928489727 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e200056d0 con 0x55966af97410 2024-01-31T05:38:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.953+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2928489727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97410 msgr2=0x55966afb10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.953+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/2928489727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97410 0x55966afb10e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5e20005e90 tx=0x7f5e2003f000 comp rx=0 tx=0).stop 2024-01-31T05:38:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.953+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2928489727 shutdown_connections 2024-01-31T05:38:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.953+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/2928489727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97dd0 0x55966af92c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.953+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/2928489727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966afa96b0 0x55966afa6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.953+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/2928489727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97410 0x55966afb10e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.953+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2928489727 >> 172.21.15.45:0/2928489727 conn(0x55966af99ed0 msgr2=0x55966afcfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.954+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2928489727 shutdown_connections 2024-01-31T05:38:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.954+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2928489727 wait complete. 2024-01-31T05:38:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.955+0000 7f5e38fd8740 1 -- start start 2024-01-31T05:38:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.955+0000 7f5e38fd8740 1 Processor -- start 2024-01-31T05:38:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e38fd8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97410 0x55966afb5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e38fd8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 0x55966afb3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e38fd8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966afa96b0 0x55966afb41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e38fd8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966afb48c0 con 0x55966af97410 2024-01-31T05:38:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e38fd8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966afb4a30 con 0x55966af97dd0 2024-01-31T05:38:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e38fd8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966b01e540 con 0x55966afa96b0 2024-01-31T05:38:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e38fd8740 10 client.0 _ll_register_callbacks cb 0x7f5e28009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:38:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e2f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97410 0x55966afb5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e2ffff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966afa96b0 0x55966afb41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e2f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97410 0x55966afb5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51588/0 (socket says 172.21.15.45:51588) 2024-01-31T05:38:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e2f7fe640 1 -- 172.21.15.45:0/2488131850 learned_addr learned my addr 172.21.15.45:0/2488131850 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e2effd640 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 0x55966afb3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e2ffff640 1 -- 172.21.15.45:0/2488131850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 msgr2=0x55966afb3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e2ffff640 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 0x55966afb3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e2ffff640 1 -- 172.21.15.45:0/2488131850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97410 msgr2=0x55966afb5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e2ffff640 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97410 0x55966afb5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e2ffff640 1 -- 172.21.15.45:0/2488131850 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e30068370 con 0x55966afa96b0 2024-01-31T05:38:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.956+0000 7f5e2effd640 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 0x55966afb3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:38:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.957+0000 7f5e2f7fe640 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97410 0x55966afb5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:38:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.957+0000 7f5e2ffff640 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966afa96b0 0x55966afb41c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5e28011dd0 tx=0x7f5e28005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f419fb77dd9988 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.957+0000 7f5e0effd640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-01-31T05:38:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.958+0000 7f5e0effd640 1 -- 172.21.15.45:0/2488131850 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28005770 con 0x55966afa96b0 2024-01-31T05:38:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.958+0000 7f5e0effd640 1 -- 172.21.15.45:0/2488131850 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5e280058d0 con 0x55966afa96b0 2024-01-31T05:38:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.958+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2488131850 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55966aefe010 con 0x55966afa96b0 2024-01-31T05:38:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.958+0000 7f5e0effd640 1 -- 172.21.15.45:0/2488131850 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e28005df0 con 0x55966afa96b0 2024-01-31T05:38:01.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.958+0000 7f5e38fd8740 10 client.26200 fetch_fsmap learned FSMap version 31 2024-01-31T05:38:01.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.958+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2488131850 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55966aec5a50 con 0x55966afa96b0 2024-01-31T05:38:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.959+0000 7f5e0effd640 1 -- 172.21.15.45:0/2488131850 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e2800f9a0 con 0x55966afa96b0 2024-01-31T05:38:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.959+0000 7f5e0effd640 1 -- 172.21.15.45:0/2488131850 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5e28036610 con 0x55966afa96b0 2024-01-31T05:38:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.959+0000 7f5e38fd8740 10 client.26200 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:38:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.959+0000 7f5e38fd8740 10 client.26200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:38:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.960+0000 7f5e38fd8740 20 client.26200 populate_metadata read hostname 'smithi045' 2024-01-31T05:38:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.960+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55966b01fa00 0x55966b021e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.960+0000 7f5e38fd8740 4 client.26200 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:38:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.960+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2488131850 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55966aefe010 con 0x55966b01fa00 2024-01-31T05:38:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.961+0000 7f5e2f7fe640 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55966b01fa00 0x55966b021e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.963+0000 7f5e2f7fe640 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55966b01fa00 0x55966b021e20 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e34708758be04dbb server_cookie=8e54bde681ae35ab in_seq=0 out_seq=0 2024-01-31T05:38:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:01.963+0000 7f5e0effd640 10 client.26200 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.003+0000 7f5e0effd640 1 -- 172.21.15.45:0/2488131850 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55966aefe010 con 0x55966b01fa00 2024-01-31T05:38:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.003+0000 7f5e0effd640 10 client.26200 handle_command_reply: tid=0 2024-01-31T05:38:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.003+0000 7f5e38fd8740 1 client.26200 shutdown 2024-01-31T05:38:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.004+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2488131850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966afa96b0 msgr2=0x55966afb41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.004+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966afa96b0 0x55966afb41c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5e28011dd0 tx=0x7f5e28005270 comp rx=0 tx=0).stop 2024-01-31T05:38:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.004+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2488131850 shutdown_connections 2024-01-31T05:38:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.004+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55966b01fa00 0x55966b021e20 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.004+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55966afa96b0 0x55966afb41c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.004+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55966af97dd0 0x55966afb3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.004+0000 7f5e38fd8740 1 --2- 172.21.15.45:0/2488131850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55966af97410 0x55966afb5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.004+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2488131850 >> 172.21.15.45:0/2488131850 conn(0x55966af99ed0 msgr2=0x55966afad110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.004+0000 7f5e0effd640 0 client.26200 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.004+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2488131850 shutdown_connections 2024-01-31T05:38:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.004+0000 7f5e38fd8740 1 -- 172.21.15.45:0/2488131850 wait complete. 2024-01-31T05:38:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.004+0000 7f5e38fd8740 20 client.26200 trim_cache size 0 max 16384 2024-01-31T05:38:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:38:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:38:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:38:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:38:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.005+0000 7f5e3611f640 1 -- 172.21.15.45:0/2292775879 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5e180a1650 msgr2=0x7f5e180a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.005+0000 7f5e3611f640 1 --2- 172.21.15.45:0/2292775879 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5e180a1650 0x7f5e180a3b10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5e20000c00 tx=0x7f5e20005e20 comp rx=0 tx=0).stop 2024-01-31T05:38:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.005+0000 7f5e3611f640 1 -- 172.21.15.45:0/2292775879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e30100800 msgr2=0x7f5e301977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.005+0000 7f5e3611f640 1 --2- 172.21.15.45:0/2292775879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e30100800 0x7f5e301977c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400e9f0 tx=0x7f5e2400eeb0 comp rx=0 tx=0).stop 2024-01-31T05:38:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.005+0000 7f5e3611f640 1 -- 172.21.15.45:0/2292775879 shutdown_connections 2024-01-31T05:38:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.005+0000 7f5e3611f640 1 --2- 172.21.15.45:0/2292775879 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f5e180a1650 0x7f5e180a3b10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.005+0000 7f5e3611f640 1 --2- 172.21.15.45:0/2292775879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e30104b90 0x7f5e3019baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.005+0000 7f5e3611f640 1 --2- 172.21.15.45:0/2292775879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e30100800 0x7f5e301977c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.005+0000 7f5e3611f640 1 --2- 172.21.15.45:0/2292775879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e300ffe40 0x7f5e30197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.005+0000 7f5e3611f640 1 -- 172.21.15.45:0/2292775879 >> 172.21.15.45:0/2292775879 conn(0x7f5e30076a40 msgr2=0x7f5e300fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.005+0000 7f5e3611f640 1 -- 172.21.15.45:0/2292775879 shutdown_connections 2024-01-31T05:38:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:02.005+0000 7f5e3611f640 1 -- 172.21.15.45:0/2292775879 wait complete. 2024-01-31T05:38:02.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:01 smithi186 ceph-mon[35659]: pgmap v487: 97 pgs: 97 active+clean; 904 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.7 MiB/s wr, 397 op/s 2024-01-31T05:38:02.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:01 smithi045 ceph-mon[31585]: pgmap v487: 97 pgs: 97 active+clean; 904 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.7 MiB/s wr, 397 op/s 2024-01-31T05:38:02.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:01 smithi156 ceph-mon[35720]: pgmap v487: 97 pgs: 97 active+clean; 904 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.7 MiB/s wr, 397 op/s 2024-01-31T05:38:02.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 52192 14.98 MB/sec execute 1 sec latency 778.414 ms 2024-01-31T05:38:03.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 52578 23.54 MB/sec execute 2 sec latency 60.480 ms 2024-01-31T05:38:03.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:03 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:38:03.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:03 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:38:03.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:03 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:38:03.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:03 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:38:03.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:03 smithi156 ceph-mon[35720]: pgmap v488: 97 pgs: 97 active+clean; 915 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 800 op/s 2024-01-31T05:38:04.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:03 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:38:04.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:03 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:38:04.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:03 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:38:04.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:03 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:38:04.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:03 smithi186 ceph-mon[35659]: pgmap v488: 97 pgs: 97 active+clean; 915 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 800 op/s 2024-01-31T05:38:04.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:03 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:38:04.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:03 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:38:04.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:03 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:38:04.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:03 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:38:04.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:03 smithi045 ceph-mon[31585]: pgmap v488: 97 pgs: 97 active+clean; 915 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 800 op/s 2024-01-31T05:38:04.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 52870 20.21 MB/sec execute 3 sec latency 69.590 ms 2024-01-31T05:38:05.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 53249 19.19 MB/sec execute 4 sec latency 138.520 ms 2024-01-31T05:38:06.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 53673 20.32 MB/sec execute 5 sec latency 265.791 ms 2024-01-31T05:38:06.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:06 smithi156 ceph-mon[35720]: osdmap e114: 12 total, 12 up, 12 in 2024-01-31T05:38:06.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:06 smithi156 ceph-mon[35720]: pgmap v490: 97 pgs: 97 active+clean; 915 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 694 op/s 2024-01-31T05:38:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:06 smithi186 ceph-mon[35659]: osdmap e114: 12 total, 12 up, 12 in 2024-01-31T05:38:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:06 smithi186 ceph-mon[35659]: pgmap v490: 97 pgs: 97 active+clean; 915 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 694 op/s 2024-01-31T05:38:07.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:06 smithi045 ceph-mon[31585]: osdmap e114: 12 total, 12 up, 12 in 2024-01-31T05:38:07.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:06 smithi045 ceph-mon[31585]: pgmap v490: 97 pgs: 97 active+clean; 915 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 694 op/s 2024-01-31T05:38:07.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 54206 21.09 MB/sec execute 6 sec latency 179.485 ms 2024-01-31T05:38:07.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:07 smithi156 ceph-mon[35720]: pgmap v491: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.9 MiB/s wr, 708 op/s 2024-01-31T05:38:08.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:07 smithi186 ceph-mon[35659]: pgmap v491: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.9 MiB/s wr, 708 op/s 2024-01-31T05:38:08.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:07 smithi045 ceph-mon[31585]: pgmap v491: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.9 MiB/s wr, 708 op/s 2024-01-31T05:38:08.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 54579 20.64 MB/sec execute 7 sec latency 307.683 ms 2024-01-31T05:38:09.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 55076 21.58 MB/sec execute 8 sec latency 164.178 ms 2024-01-31T05:38:09.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:09 smithi156 ceph-mon[35720]: pgmap v492: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.9 MiB/s wr, 708 op/s 2024-01-31T05:38:10.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:09 smithi186 ceph-mon[35659]: pgmap v492: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.9 MiB/s wr, 708 op/s 2024-01-31T05:38:10.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:09 smithi045 ceph-mon[31585]: pgmap v492: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.9 MiB/s wr, 708 op/s 2024-01-31T05:38:10.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 55606 22.43 MB/sec execute 9 sec latency 243.024 ms 2024-01-31T05:38:11.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 55884 21.93 MB/sec execute 10 sec latency 395.656 ms 2024-01-31T05:38:12.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 56319 22.53 MB/sec execute 11 sec latency 185.800 ms 2024-01-31T05:38:12.976 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:12 smithi156 ceph-mon[35720]: pgmap v493: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.9 MiB/s wr, 708 op/s 2024-01-31T05:38:12.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:12 smithi156 ceph-mon[35720]: osdmap e115: 12 total, 12 up, 12 in 2024-01-31T05:38:13.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:12 smithi186 ceph-mon[35659]: pgmap v493: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.9 MiB/s wr, 708 op/s 2024-01-31T05:38:13.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:12 smithi186 ceph-mon[35659]: osdmap e115: 12 total, 12 up, 12 in 2024-01-31T05:38:13.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:12 smithi045 ceph-mon[31585]: pgmap v493: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.9 MiB/s wr, 708 op/s 2024-01-31T05:38:13.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:12 smithi045 ceph-mon[31585]: osdmap e115: 12 total, 12 up, 12 in 2024-01-31T05:38:13.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 56674 22.04 MB/sec execute 12 sec latency 59.467 ms 2024-01-31T05:38:14.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 57169 22.10 MB/sec execute 13 sec latency 180.229 ms 2024-01-31T05:38:14.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:14 smithi156 ceph-mon[35720]: pgmap v495: 97 pgs: 97 active+clean; 948 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 585 op/s 2024-01-31T05:38:15.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:14 smithi186 ceph-mon[35659]: pgmap v495: 97 pgs: 97 active+clean; 948 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 585 op/s 2024-01-31T05:38:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.107+0000 7f452e1ae640 1 -- 172.21.15.45:0/2377777316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528100010 msgr2=0x7f452810c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.107+0000 7f452e1ae640 1 --2- 172.21.15.45:0/2377777316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528100010 0x7f452810c1d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f452000ac00 tx=0x7f452002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.107+0000 7f452e1ae640 1 -- 172.21.15.45:0/2377777316 shutdown_connections 2024-01-31T05:38:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.107+0000 7f452e1ae640 1 --2- 172.21.15.45:0/2377777316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528100010 0x7f452810c1d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.107+0000 7f452e1ae640 1 --2- 172.21.15.45:0/2377777316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45280ff650 0x7f45280ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.107+0000 7f452e1ae640 1 --2- 172.21.15.45:0/2377777316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4528104340 0x7f4528104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.107+0000 7f452e1ae640 1 -- 172.21.15.45:0/2377777316 >> 172.21.15.45:0/2377777316 conn(0x7f4528075d80 msgr2=0x7f45280fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.107+0000 7f452e1ae640 1 -- 172.21.15.45:0/2377777316 shutdown_connections 2024-01-31T05:38:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.107+0000 7f452e1ae640 1 -- 172.21.15.45:0/2377777316 wait complete. 2024-01-31T05:38:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.107+0000 7f452e1ae640 1 Processor -- start 2024-01-31T05:38:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.108+0000 7f452e1ae640 1 -- start start 2024-01-31T05:38:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.108+0000 7f452e1ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45280ff650 0x7f4528194f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.108+0000 7f452e1ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528100010 0x7f4528195440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.108+0000 7f452e1ae640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4528104340 0x7f4528199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.108+0000 7f452e1ae640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528199d70 con 0x7f4528100010 2024-01-31T05:38:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.108+0000 7f452e1ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528199ee0 con 0x7f45280ff650 2024-01-31T05:38:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.108+0000 7f452e1ae640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452819a0a0 con 0x7f4528104340 2024-01-31T05:38:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.108+0000 7f4526ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528100010 0x7f4528195440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.108+0000 7f4526ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528100010 0x7f4528195440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38814/0 (socket says 172.21.15.45:38814) 2024-01-31T05:38:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.108+0000 7f4526ffd640 1 -- 172.21.15.45:0/3645150288 learned_addr learned my addr 172.21.15.45:0/3645150288 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.108+0000 7f4527fff640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4528104340 0x7f4528199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.108+0000 7f45277fe640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45280ff650 0x7f4528194f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.109+0000 7f4526ffd640 1 -- 172.21.15.45:0/3645150288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4528104340 msgr2=0x7f4528199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.109+0000 7f4526ffd640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4528104340 0x7f4528199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.109+0000 7f4526ffd640 1 -- 172.21.15.45:0/3645150288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45280ff650 msgr2=0x7f4528194f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.109+0000 7f4526ffd640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45280ff650 0x7f4528194f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.109+0000 7f4526ffd640 1 -- 172.21.15.45:0/3645150288 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c009490 con 0x7f4528100010 2024-01-31T05:38:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.109+0000 7f4526ffd640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528100010 0x7f4528195440 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f451c00ea60 tx=0x7f451c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.109+0000 7f4524ff9640 1 -- 172.21.15.45:0/3645150288 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c00cde0 con 0x7f4528100010 2024-01-31T05:38:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.109+0000 7f4524ff9640 1 -- 172.21.15.45:0/3645150288 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f451c0035f0 con 0x7f4528100010 2024-01-31T05:38:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.109+0000 7f452e1ae640 1 -- 172.21.15.45:0/3645150288 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451800d570 con 0x7f4528100010 2024-01-31T05:38:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.109+0000 7f4524ff9640 1 -- 172.21.15.45:0/3645150288 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c0044e0 con 0x7f4528100010 2024-01-31T05:38:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.109+0000 7f452e1ae640 1 -- 172.21.15.45:0/3645150288 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452000a860 con 0x7f4528100010 2024-01-31T05:38:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4530f73740 1 Processor -- start 2024-01-31T05:38:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4524ff9640 1 -- 172.21.15.45:0/3645150288 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f451c003a60 con 0x7f4528100010 2024-01-31T05:38:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4530f73740 1 -- start start 2024-01-31T05:38:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4530f73740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1060 0x55e4043e1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4530f73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1980 0x55e4043ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4530f73740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4530f73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e404350040 con 0x55e4043e1980 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4530f73740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40434d730 con 0x55e4043ea750 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4530f73740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e404349020 con 0x55e4043e1060 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f45277fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1980 0x55e4043ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4527fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1060 0x55e4043e1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4526ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4527fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1060 0x55e4043e1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:48696/0 (socket says 172.21.15.45:48696) 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.112+0000 7f4527fff640 1 -- 172.21.15.45:0/1771189791 learned_addr learned my addr 172.21.15.45:0/1771189791 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.113+0000 7f4524ff9640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f45100a14f0 0x7f45100a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.113+0000 7f45277fe640 1 -- 172.21.15.45:0/1771189791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1060 msgr2=0x55e4043e1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.113+0000 7f45277fe640 1 --2- 172.21.15.45:0/1771189791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1060 0x55e4043e1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.113+0000 7f45277fe640 1 -- 172.21.15.45:0/1771189791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 msgr2=0x55e4043ecb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.113+0000 7f45277fe640 1 --2- 172.21.15.45:0/1771189791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043ecb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.113+0000 7f45277fe640 1 -- 172.21.15.45:0/1771189791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e404340e40 con 0x55e4043e1980 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.113+0000 7f4527fff640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f45100a14f0 0x7f45100a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.113+0000 7f4524ff9640 1 -- 172.21.15.45:0/3645150288 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f451c014070 con 0x7f4528100010 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.113+0000 7f45277fe640 1 --2- 172.21.15.45:0/1771189791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1980 0x55e4043ea210 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4518005fd0 tx=0x7f4518009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905561f569d1510f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.113+0000 7f450dffb640 1 -- 172.21.15.45:0/1771189791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518012050 con 0x55e4043e1980 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.113+0000 7f450dffb640 1 -- 172.21.15.45:0/1771189791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4518013040 con 0x55e4043e1980 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.113+0000 7f450dffb640 1 -- 172.21.15.45:0/1771189791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518016560 con 0x55e4043e1980 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.114+0000 7f4530f73740 1 -- 172.21.15.45:0/1771189791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1980 msgr2=0x55e4043ea210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.114+0000 7f4530f73740 1 --2- 172.21.15.45:0/1771189791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1980 0x55e4043ea210 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4518005fd0 tx=0x7f4518009260 comp rx=0 tx=0).stop 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.114+0000 7f4527fff640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f45100a14f0 0x7f45100a39b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f452000ac00 tx=0x7f452000a560 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.114+0000 7f4530f73740 1 -- 172.21.15.45:0/1771189791 shutdown_connections 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.114+0000 7f4530f73740 1 --2- 172.21.15.45:0/1771189791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1980 0x55e4043ea210 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.114+0000 7f4530f73740 1 --2- 172.21.15.45:0/1771189791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.114+0000 7f4530f73740 1 --2- 172.21.15.45:0/1771189791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1060 0x55e4043e1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.114+0000 7f4530f73740 1 -- 172.21.15.45:0/1771189791 >> 172.21.15.45:0/1771189791 conn(0x55e4043db980 msgr2=0x55e4043dbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.114+0000 7f4530f73740 1 -- 172.21.15.45:0/1771189791 shutdown_connections 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.114+0000 7f4530f73740 1 -- 172.21.15.45:0/1771189791 wait complete. 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.114+0000 7f4530f73740 1 -- start start 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.114+0000 7f4530f73740 1 Processor -- start 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4530f73740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1060 0x55e4043e7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4530f73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1980 0x55e4043e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4530f73740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4530f73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4043021b0 con 0x55e4043e1980 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4530f73740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e404301e50 con 0x55e4043ea750 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4530f73740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4042c7090 con 0x55e4043e1060 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4530f73740 10 client.0 _ll_register_callbacks cb 0x55e4042eca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:38:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4527fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4526ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1980 0x55e4043e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f45277fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1060 0x55e4043e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4527fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043e80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:46976/0 (socket says 172.21.15.45:46976) 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4527fff640 1 -- 172.21.15.45:0/351063399 learned_addr learned my addr 172.21.15.45:0/351063399 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f45277fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1060 0x55e4043e7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:48706/0 (socket says 172.21.15.45:48706) 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4527fff640 1 -- 172.21.15.45:0/351063399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1060 msgr2=0x55e4043e7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4527fff640 1 --2- 172.21.15.45:0/351063399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1060 0x55e4043e7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4527fff640 1 -- 172.21.15.45:0/351063399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1980 msgr2=0x55e4043e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4527fff640 1 --2- 172.21.15.45:0/351063399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1980 0x55e4043e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4527fff640 1 -- 172.21.15.45:0/351063399 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451800d850 con 0x55e4043ea750 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.115+0000 7f4526ffd640 1 --2- 172.21.15.45:0/351063399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1980 0x55e4043e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.116+0000 7f4527fff640 1 --2- 172.21.15.45:0/351063399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043e80f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4520034c90 tx=0x7f452002c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33e3d10093f8b43 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.116+0000 7f4506ffd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.116+0000 7f4506ffd640 1 -- 172.21.15.45:0/351063399 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200419b0 con 0x55e4043ea750 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.116+0000 7f4530f73740 1 -- 172.21.15.45:0/351063399 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e404342010 con 0x55e4043ea750 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.116+0000 7f4506ffd640 1 -- 172.21.15.45:0/351063399 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4520041b10 con 0x55e4043ea750 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.116+0000 7f4506ffd640 1 -- 172.21.15.45:0/351063399 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4520003970 con 0x55e4043ea750 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.116+0000 7f4530f73740 10 client.26147 fetch_fsmap learned FSMap version 31 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.116+0000 7f4530f73740 1 -- 172.21.15.45:0/351063399 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e4043476a0 con 0x55e4043ea750 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.117+0000 7f4506ffd640 1 -- 172.21.15.45:0/351063399 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e404342010 con 0x55e4043ea750 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.117+0000 7f4506ffd640 1 -- 172.21.15.45:0/351063399 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4520052030 con 0x55e4043ea750 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.117+0000 7f4530f73740 10 client.26147 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.117+0000 7f4530f73740 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.117+0000 7f4530f73740 20 client.26147 populate_metadata read hostname 'smithi045' 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.117+0000 7f4530f73740 1 --2- 172.21.15.45:0/351063399 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e4043fa4f0 0x55e4043fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.117+0000 7f4530f73740 4 client.26147 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:38:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.117+0000 7f4530f73740 1 -- 172.21.15.45:0/351063399 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e404341d30 con 0x55e4043fa4f0 2024-01-31T05:38:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.117+0000 7f45277fe640 1 --2- 172.21.15.45:0/351063399 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e4043fa4f0 0x55e4043fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.117+0000 7f45277fe640 1 --2- 172.21.15.45:0/351063399 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e4043fa4f0 0x55e4043fc8e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f92fca957c82789 server_cookie=833fff20b00fb311 in_seq=0 out_seq=0 2024-01-31T05:38:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.117+0000 7f4506ffd640 10 client.26147 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.119+0000 7f4506ffd640 1 -- 172.21.15.45:0/351063399 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e404341d30 con 0x55e4043fa4f0 2024-01-31T05:38:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.119+0000 7f4506ffd640 10 client.26147 handle_command_reply: tid=0 2024-01-31T05:38:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.119+0000 7f4530f73740 1 client.26147 shutdown 2024-01-31T05:38:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.120+0000 7f4530f73740 1 -- 172.21.15.45:0/351063399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 msgr2=0x55e4043e80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.120+0000 7f4530f73740 1 --2- 172.21.15.45:0/351063399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043e80f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4520034c90 tx=0x7f452002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.120+0000 7f4530f73740 1 -- 172.21.15.45:0/351063399 shutdown_connections 2024-01-31T05:38:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.120+0000 7f4530f73740 1 --2- 172.21.15.45:0/351063399 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e4043fa4f0 0x55e4043fc8e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.120+0000 7f4530f73740 1 --2- 172.21.15.45:0/351063399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1980 0x55e4043e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.120+0000 7f4530f73740 1 --2- 172.21.15.45:0/351063399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043e80f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.120+0000 7f4530f73740 1 --2- 172.21.15.45:0/351063399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1060 0x55e4043e7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.120+0000 7f4506ffd640 0 client.26147 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.120+0000 7f4530f73740 1 -- 172.21.15.45:0/351063399 >> 172.21.15.45:0/351063399 conn(0x55e4043db980 msgr2=0x55e4043dd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.120+0000 7f4530f73740 1 -- 172.21.15.45:0/351063399 shutdown_connections 2024-01-31T05:38:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.120+0000 7f4530f73740 1 -- 172.21.15.45:0/351063399 wait complete. 2024-01-31T05:38:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.120+0000 7f4530f73740 20 client.26147 trim_cache size 0 max 16384 2024-01-31T05:38:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.137+0000 7f4530f73740 1 Processor -- start 2024-01-31T05:38:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f4530f73740 1 -- start start 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f4530f73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 0x55e4043e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f4530f73740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043e1980 0x55e4043e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f4530f73740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043ea750 0x55e4043e48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f4530f73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4043e25b0 con 0x55e4043e1060 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f4530f73740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4043e08a0 con 0x55e4043e1980 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f4530f73740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e404342010 con 0x55e4043ea750 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f45277fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 0x55e4043e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f45277fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 0x55e4043e9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38856/0 (socket says 172.21.15.45:38856) 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f45277fe640 1 -- 172.21.15.45:0/4110463492 learned_addr learned my addr 172.21.15.45:0/4110463492 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f4527fff640 1 --2- 172.21.15.45:0/4110463492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043ea750 0x55e4043e48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f4526ffd640 1 --2- 172.21.15.45:0/4110463492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043e1980 0x55e4043e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f45277fe640 1 -- 172.21.15.45:0/4110463492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043ea750 msgr2=0x55e4043e48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f45277fe640 1 --2- 172.21.15.45:0/4110463492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043ea750 0x55e4043e48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f45277fe640 1 -- 172.21.15.45:0/4110463492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043e1980 msgr2=0x55e4043e7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f45277fe640 1 --2- 172.21.15.45:0/4110463492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043e1980 0x55e4043e7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f45277fe640 1 -- 172.21.15.45:0/4110463492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4042e4030 con 0x55e4043e1060 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.138+0000 7f45277fe640 1 --2- 172.21.15.45:0/4110463492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 0x55e4043e9da0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4518005fd0 tx=0x7f4518009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e400e9ae6378d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.139+0000 7f450d7fa640 1 -- 172.21.15.45:0/4110463492 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518026040 con 0x55e4043e1060 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.139+0000 7f450d7fa640 1 -- 172.21.15.45:0/4110463492 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f451802a870 con 0x55e4043e1060 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.139+0000 7f450d7fa640 1 -- 172.21.15.45:0/4110463492 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518012040 con 0x55e4043e1060 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.139+0000 7f4530f73740 1 -- 172.21.15.45:0/4110463492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 msgr2=0x55e4043e9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.139+0000 7f4530f73740 1 --2- 172.21.15.45:0/4110463492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 0x55e4043e9da0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4518005fd0 tx=0x7f4518009ce0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.139+0000 7f4530f73740 1 -- 172.21.15.45:0/4110463492 shutdown_connections 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.139+0000 7f4530f73740 1 --2- 172.21.15.45:0/4110463492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043ea750 0x55e4043e48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.139+0000 7f4530f73740 1 --2- 172.21.15.45:0/4110463492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043e1980 0x55e4043e7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.139+0000 7f4530f73740 1 --2- 172.21.15.45:0/4110463492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 0x55e4043e9da0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.139+0000 7f4530f73740 1 -- 172.21.15.45:0/4110463492 >> 172.21.15.45:0/4110463492 conn(0x55e4043db980 msgr2=0x55e404401260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.139+0000 7f4530f73740 1 -- 172.21.15.45:0/4110463492 shutdown_connections 2024-01-31T05:38:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.139+0000 7f4530f73740 1 -- 172.21.15.45:0/4110463492 wait complete. 2024-01-31T05:38:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f4530f73740 1 -- start start 2024-01-31T05:38:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f4530f73740 1 Processor -- start 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f4530f73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 0x55e4043ef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f4530f73740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1980 0x55e4043ef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f4530f73740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043efb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f4530f73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40444f9e0 con 0x55e4043e1060 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f4530f73740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40444fb50 con 0x55e4043ea750 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f4530f73740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40444fd10 con 0x55e4043e1980 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f4530f73740 10 client.0 _ll_register_callbacks cb 0x7f451800d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f45277fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 0x55e4043ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f45277fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 0x55e4043ef080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38862/0 (socket says 172.21.15.45:38862) 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f45277fe640 1 -- 172.21.15.45:0/2526122648 learned_addr learned my addr 172.21.15.45:0/2526122648 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f4526ffd640 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1980 0x55e4043ef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.140+0000 7f4527fff640 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043efb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f45277fe640 1 -- 172.21.15.45:0/2526122648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1980 msgr2=0x55e4043ef5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f45277fe640 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1980 0x55e4043ef5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f45277fe640 1 -- 172.21.15.45:0/2526122648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 msgr2=0x55e4043efb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f45277fe640 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043efb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f45277fe640 1 -- 172.21.15.45:0/2526122648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e404340e40 con 0x55e4043e1060 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4527fff640 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043efb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f45277fe640 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 0x55e4043ef080 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4518005540 tx=0x7f45180092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb7ee3700136f3f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4506ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4506ffd640 1 -- 172.21.15.45:0/2526122648 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518026040 con 0x55e4043e1060 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4530f73740 1 -- 172.21.15.45:0/2526122648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4518013040 con 0x55e4043e1060 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4506ffd640 1 -- 172.21.15.45:0/2526122648 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f451802a5b0 con 0x55e4043e1060 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4506ffd640 1 -- 172.21.15.45:0/2526122648 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f451802aa90 con 0x55e4043e1060 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4530f73740 10 client.16656 fetch_fsmap learned FSMap version 31 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4530f73740 1 -- 172.21.15.45:0/2526122648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e404309a50 con 0x55e4043e1060 2024-01-31T05:38:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4506ffd640 1 -- 172.21.15.45:0/2526122648 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4518013040 con 0x55e4043e1060 2024-01-31T05:38:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4506ffd640 1 -- 172.21.15.45:0/2526122648 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4518029070 con 0x55e4043e1060 2024-01-31T05:38:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4530f73740 10 client.16656 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:38:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4530f73740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:38:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4530f73740 20 client.16656 populate_metadata read hostname 'smithi045' 2024-01-31T05:38:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4530f73740 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e404451220 0x55e404453640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4530f73740 4 client.16656 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:38:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.141+0000 7f4530f73740 1 -- 172.21.15.45:0/2526122648 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e404048600 con 0x55e404451220 2024-01-31T05:38:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.142+0000 7f4526ffd640 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e404451220 0x55e404453640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.142+0000 7f4526ffd640 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e404451220 0x55e404453640 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d3c015588097fa1 server_cookie=c693cb9487e9329b in_seq=0 out_seq=0 2024-01-31T05:38:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.142+0000 7f4506ffd640 10 client.16656 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:15.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:14 smithi045 ceph-mon[31585]: pgmap v495: 97 pgs: 97 active+clean; 948 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 585 op/s 2024-01-31T05:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.195+0000 7f4506ffd640 1 -- 172.21.15.45:0/2526122648 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e404048600 con 0x55e404451220 2024-01-31T05:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.195+0000 7f4506ffd640 10 client.16656 handle_command_reply: tid=0 2024-01-31T05:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.195+0000 7f4530f73740 1 client.16656 shutdown 2024-01-31T05:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.195+0000 7f4530f73740 1 -- 172.21.15.45:0/2526122648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 msgr2=0x55e4043ef080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.195+0000 7f4530f73740 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 0x55e4043ef080 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4518005540 tx=0x7f45180092e0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.196+0000 7f4530f73740 1 -- 172.21.15.45:0/2526122648 shutdown_connections 2024-01-31T05:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.196+0000 7f4530f73740 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e404451220 0x55e404453640 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.196+0000 7f4530f73740 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4043ea750 0x55e4043efb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.196+0000 7f4530f73740 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e4043e1980 0x55e4043ef5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.196+0000 7f4506ffd640 0 client.16656 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.196+0000 7f4530f73740 1 --2- 172.21.15.45:0/2526122648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4043e1060 0x55e4043ef080 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.196+0000 7f4530f73740 1 -- 172.21.15.45:0/2526122648 >> 172.21.15.45:0/2526122648 conn(0x55e4043db980 msgr2=0x55e4043da190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.196+0000 7f4530f73740 1 -- 172.21.15.45:0/2526122648 shutdown_connections 2024-01-31T05:38:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.196+0000 7f4530f73740 1 -- 172.21.15.45:0/2526122648 wait complete. 2024-01-31T05:38:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.196+0000 7f4530f73740 20 client.16656 trim_cache size 0 max 16384 2024-01-31T05:38:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:38:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:38:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:38:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:38:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.197+0000 7f452e1ae640 1 -- 172.21.15.45:0/3645150288 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f45100a14f0 msgr2=0x7f45100a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.197+0000 7f452e1ae640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f45100a14f0 0x7f45100a39b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f452000ac00 tx=0x7f452000a560 comp rx=0 tx=0).stop 2024-01-31T05:38:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.197+0000 7f452e1ae640 1 -- 172.21.15.45:0/3645150288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528100010 msgr2=0x7f4528195440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.197+0000 7f452e1ae640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528100010 0x7f4528195440 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f451c00ea60 tx=0x7f451c00ef20 comp rx=0 tx=0).stop 2024-01-31T05:38:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.197+0000 7f452e1ae640 1 -- 172.21.15.45:0/3645150288 shutdown_connections 2024-01-31T05:38:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.197+0000 7f452e1ae640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f45100a14f0 0x7f45100a39b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.197+0000 7f452e1ae640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528100010 0x7f4528195440 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.197+0000 7f452e1ae640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4528104340 0x7f4528199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.197+0000 7f452e1ae640 1 --2- 172.21.15.45:0/3645150288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45280ff650 0x7f4528194f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.197+0000 7f452e1ae640 1 -- 172.21.15.45:0/3645150288 >> 172.21.15.45:0/3645150288 conn(0x7f4528075d80 msgr2=0x7f4528102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.197+0000 7f452e1ae640 1 -- 172.21.15.45:0/3645150288 shutdown_connections 2024-01-31T05:38:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:15.197+0000 7f452e1ae640 1 -- 172.21.15.45:0/3645150288 wait complete. 2024-01-31T05:38:15.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 57629 22.60 MB/sec execute 14 sec latency 162.866 ms 2024-01-31T05:38:16.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 57966 22.51 MB/sec execute 15 sec latency 140.693 ms 2024-01-31T05:38:17.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:16 smithi186 ceph-mon[35659]: pgmap v496: 97 pgs: 97 active+clean; 948 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.3 MiB/s wr, 468 op/s 2024-01-31T05:38:17.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:16 smithi045 ceph-mon[31585]: pgmap v496: 97 pgs: 97 active+clean; 948 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.3 MiB/s wr, 468 op/s 2024-01-31T05:38:17.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:16 smithi156 ceph-mon[35720]: pgmap v496: 97 pgs: 97 active+clean; 948 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.3 MiB/s wr, 468 op/s 2024-01-31T05:38:17.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 58356 21.83 MB/sec execute 16 sec latency 193.707 ms 2024-01-31T05:38:18.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 58876 21.86 MB/sec execute 17 sec latency 219.965 ms 2024-01-31T05:38:19.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:18 smithi186 ceph-mon[35659]: pgmap v497: 97 pgs: 97 active+clean; 885 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.4 MiB/s wr, 483 op/s 2024-01-31T05:38:19.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:18 smithi045 ceph-mon[31585]: pgmap v497: 97 pgs: 97 active+clean; 885 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.4 MiB/s wr, 483 op/s 2024-01-31T05:38:19.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:18 smithi156 ceph-mon[35720]: pgmap v497: 97 pgs: 97 active+clean; 885 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.4 MiB/s wr, 483 op/s 2024-01-31T05:38:19.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 59453 22.78 MB/sec execute 18 sec latency 193.350 ms 2024-01-31T05:38:20.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 59955 23.35 MB/sec execute 19 sec latency 196.000 ms 2024-01-31T05:38:21.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:20 smithi186 ceph-mon[35659]: pgmap v498: 97 pgs: 97 active+clean; 885 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.4 MiB/s wr, 483 op/s 2024-01-31T05:38:21.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:20 smithi045 ceph-mon[31585]: pgmap v498: 97 pgs: 97 active+clean; 885 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.4 MiB/s wr, 483 op/s 2024-01-31T05:38:21.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:20 smithi156 ceph-mon[35720]: pgmap v498: 97 pgs: 97 active+clean; 885 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.4 MiB/s wr, 483 op/s 2024-01-31T05:38:21.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 60165 22.55 MB/sec execute 20 sec latency 657.549 ms 2024-01-31T05:38:22.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 60560 22.35 MB/sec execute 21 sec latency 251.443 ms 2024-01-31T05:38:23.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:22 smithi186 ceph-mon[35659]: pgmap v499: 97 pgs: 97 active+clean; 885 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.4 MiB/s wr, 483 op/s 2024-01-31T05:38:23.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:22 smithi045 ceph-mon[31585]: pgmap v499: 97 pgs: 97 active+clean; 885 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.4 MiB/s wr, 483 op/s 2024-01-31T05:38:23.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:22 smithi156 ceph-mon[35720]: pgmap v499: 97 pgs: 97 active+clean; 885 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.4 MiB/s wr, 483 op/s 2024-01-31T05:38:23.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 61065 22.37 MB/sec execute 22 sec latency 59.140 ms 2024-01-31T05:38:24.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 61562 23.08 MB/sec execute 23 sec latency 119.131 ms 2024-01-31T05:38:25.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:24 smithi186 ceph-mon[35659]: pgmap v500: 97 pgs: 97 active+clean; 904 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.3 MiB/s wr, 459 op/s 2024-01-31T05:38:25.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:24 smithi045 ceph-mon[31585]: pgmap v500: 97 pgs: 97 active+clean; 904 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.3 MiB/s wr, 459 op/s 2024-01-31T05:38:25.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:24 smithi156 ceph-mon[35720]: pgmap v500: 97 pgs: 97 active+clean; 904 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.3 MiB/s wr, 459 op/s 2024-01-31T05:38:25.647 INFO:tasks.workunit.client.0.smithi045.stdout: 10 61889 22.92 MB/sec execute 24 sec latency 397.095 ms 2024-01-31T05:38:26.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 62180 22.45 MB/sec execute 25 sec latency 304.572 ms 2024-01-31T05:38:27.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:26 smithi186 ceph-mon[35659]: pgmap v501: 97 pgs: 97 active+clean; 904 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.5 MiB/s wr, 420 op/s 2024-01-31T05:38:27.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:26 smithi045 ceph-mon[31585]: pgmap v501: 97 pgs: 97 active+clean; 904 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.5 MiB/s wr, 420 op/s 2024-01-31T05:38:27.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:26 smithi156 ceph-mon[35720]: pgmap v501: 97 pgs: 97 active+clean; 904 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.5 MiB/s wr, 420 op/s 2024-01-31T05:38:27.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 62674 22.40 MB/sec execute 26 sec latency 212.505 ms 2024-01-31T05:38:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.296+0000 7f89866b1640 1 -- 172.21.15.45:0/350013245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89800ffe20 msgr2=0x7f8980100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.296+0000 7f89866b1640 1 --2- 172.21.15.45:0/350013245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89800ffe20 0x7f8980100200 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8970009890 tx=0x7f897002bb70 comp rx=0 tx=0).stop 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.296+0000 7f89866b1640 1 -- 172.21.15.45:0/350013245 shutdown_connections 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.296+0000 7f89866b1640 1 --2- 172.21.15.45:0/350013245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89801007d0 0x7f8980104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.296+0000 7f89866b1640 1 --2- 172.21.15.45:0/350013245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8980104ef0 0x7f898010cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.296+0000 7f89866b1640 1 --2- 172.21.15.45:0/350013245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89800ffe20 0x7f8980100200 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.296+0000 7f89866b1640 1 -- 172.21.15.45:0/350013245 >> 172.21.15.45:0/350013245 conn(0x7f8980076d20 msgr2=0x7f8980077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.296+0000 7f89866b1640 1 -- 172.21.15.45:0/350013245 shutdown_connections 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.296+0000 7f89866b1640 1 -- 172.21.15.45:0/350013245 wait complete. 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f89866b1640 1 Processor -- start 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f89866b1640 1 -- start start 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f89866b1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89800ffe20 0x7f8980199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f89866b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980104ef0 0x7f89801998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f89866b1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898019dc40 0x7f898019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f89866b1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898019e630 con 0x7f898019dc40 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f89866b1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898019e7a0 con 0x7f8980104ef0 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f89866b1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898019e910 con 0x7f89800ffe20 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f897f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980104ef0 0x7f89801998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f897ffff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89800ffe20 0x7f8980199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f8984c4e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898019dc40 0x7f898019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f897ffff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89800ffe20 0x7f8980199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:44684/0 (socket says 172.21.15.45:44684) 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f897ffff640 1 -- 172.21.15.45:0/2683374150 learned_addr learned my addr 172.21.15.45:0/2683374150 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f897f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980104ef0 0x7f89801998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:51846/0 (socket says 172.21.15.45:51846) 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.297+0000 7f897ffff640 1 -- 172.21.15.45:0/2683374150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980104ef0 msgr2=0x7f89801998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.298+0000 7f897ffff640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980104ef0 0x7f89801998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.298+0000 7f897ffff640 1 -- 172.21.15.45:0/2683374150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898019dc40 msgr2=0x7f898019e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.298+0000 7f897ffff640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898019dc40 0x7f898019e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.298+0000 7f897ffff640 1 -- 172.21.15.45:0/2683374150 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8970009490 con 0x7f89800ffe20 2024-01-31T05:38:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.298+0000 7f8984c4e640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898019dc40 0x7f898019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:38:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.298+0000 7f897f7fe640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980104ef0 0x7f89801998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:38:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.298+0000 7f897ffff640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89800ffe20 0x7f8980199380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f897002c180 tx=0x7f89700042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.298+0000 7f897d7fa640 1 -- 172.21.15.45:0/2683374150 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8970034850 con 0x7f89800ffe20 2024-01-31T05:38:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.298+0000 7f89866b1640 1 -- 172.21.15.45:0/2683374150 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89801a5ef0 con 0x7f89800ffe20 2024-01-31T05:38:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.298+0000 7f897d7fa640 1 -- 172.21.15.45:0/2683374150 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89700044d0 con 0x7f89800ffe20 2024-01-31T05:38:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.298+0000 7f897d7fa640 1 -- 172.21.15.45:0/2683374150 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8970003d00 con 0x7f89800ffe20 2024-01-31T05:38:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.298+0000 7f89866b1640 1 -- 172.21.15.45:0/2683374150 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89801a6260 con 0x7f89800ffe20 2024-01-31T05:38:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f898947a740 1 Processor -- start 2024-01-31T05:38:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f897d7fa640 1 -- 172.21.15.45:0/2683374150 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f897003b040 con 0x7f89800ffe20 2024-01-31T05:38:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f898947a740 1 -- start start 2024-01-31T05:38:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f898947a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 0x55e446c2e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f898947a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 0x55e446c37210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f898947a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c37750 0x55e446c39b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f898947a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446b9d040 con 0x55e446c2e060 2024-01-31T05:38:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f898947a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446b9a730 con 0x55e446c2e980 2024-01-31T05:38:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f898947a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446b96020 con 0x55e446c37750 2024-01-31T05:38:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f8984c4e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 0x55e446c2e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f897ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 0x55e446c37210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f8984c4e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 0x55e446c2e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60696/0 (socket says 172.21.15.45:60696) 2024-01-31T05:38:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f8984c4e640 1 -- 172.21.15.45:0/2491320007 learned_addr learned my addr 172.21.15.45:0/2491320007 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f897f7fe640 1 --2- 172.21.15.45:0/2491320007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c37750 0x55e446c39b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f897d7fa640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f89680a14f0 0x7f89680a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f8984c4e640 1 -- 172.21.15.45:0/2491320007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c37750 msgr2=0x55e446c39b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f8984c4e640 1 --2- 172.21.15.45:0/2491320007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c37750 0x55e446c39b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f8984c4e640 1 -- 172.21.15.45:0/2491320007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 msgr2=0x55e446c37210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.301+0000 7f8984c4e640 1 --2- 172.21.15.45:0/2491320007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 0x55e446c37210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.302+0000 7f8984c4e640 1 -- 172.21.15.45:0/2491320007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e446b8de40 con 0x55e446c2e060 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.302+0000 7f897ffff640 1 --2- 172.21.15.45:0/2491320007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 0x55e446c37210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.302+0000 7f897d7fa640 1 -- 172.21.15.45:0/2683374150 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f89700ee850 con 0x7f89800ffe20 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.302+0000 7f8984c4e640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f89680a14f0 0x7f89680a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.302+0000 7f8984c4e640 1 --2- 172.21.15.45:0/2491320007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 0x55e446c2e440 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8978011790 tx=0x7f8978011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7175b3ad86f97547 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.302+0000 7f89667fc640 1 -- 172.21.15.45:0/2491320007 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978021070 con 0x55e446c2e060 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.302+0000 7f89667fc640 1 -- 172.21.15.45:0/2491320007 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f897800fe60 con 0x55e446c2e060 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.302+0000 7f89667fc640 1 -- 172.21.15.45:0/2491320007 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978011ea0 con 0x55e446c2e060 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.303+0000 7f898947a740 1 -- 172.21.15.45:0/2491320007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 msgr2=0x55e446c2e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.303+0000 7f8984c4e640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f89680a14f0 0x7f89680a39b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8978020000 tx=0x7f897801c7e0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.303+0000 7f898947a740 1 --2- 172.21.15.45:0/2491320007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 0x55e446c2e440 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8978011790 tx=0x7f8978011c50 comp rx=0 tx=0).stop 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.304+0000 7f898947a740 1 -- 172.21.15.45:0/2491320007 shutdown_connections 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.304+0000 7f898947a740 1 --2- 172.21.15.45:0/2491320007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c37750 0x55e446c39b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.304+0000 7f898947a740 1 --2- 172.21.15.45:0/2491320007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 0x55e446c37210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.304+0000 7f898947a740 1 --2- 172.21.15.45:0/2491320007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 0x55e446c2e440 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.304+0000 7f898947a740 1 -- 172.21.15.45:0/2491320007 >> 172.21.15.45:0/2491320007 conn(0x55e446c28980 msgr2=0x55e446c28d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.304+0000 7f898947a740 1 -- 172.21.15.45:0/2491320007 shutdown_connections 2024-01-31T05:38:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.304+0000 7f898947a740 1 -- 172.21.15.45:0/2491320007 wait complete. 2024-01-31T05:38:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.304+0000 7f898947a740 1 -- start start 2024-01-31T05:38:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.304+0000 7f898947a740 1 Processor -- start 2024-01-31T05:38:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f898947a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e060 0x55e446c34630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f898947a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 0x55e446c34b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f898947a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c37750 0x55e446c350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f898947a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446b4f1b0 con 0x55e446c37750 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f898947a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446b4ee50 con 0x55e446c2e980 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f898947a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446b14090 con 0x55e446c2e060 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f898947a740 10 client.0 _ll_register_callbacks cb 0x55e446b39a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f897f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 0x55e446c34b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f8984c4e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c37750 0x55e446c350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f897f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 0x55e446c34b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:51870/0 (socket says 172.21.15.45:51870) 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f897f7fe640 1 -- 172.21.15.45:0/148083589 learned_addr learned my addr 172.21.15.45:0/148083589 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f897ffff640 1 --2- 172.21.15.45:0/148083589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e060 0x55e446c34630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f897f7fe640 1 -- 172.21.15.45:0/148083589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e060 msgr2=0x55e446c34630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f897f7fe640 1 --2- 172.21.15.45:0/148083589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e060 0x55e446c34630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f897f7fe640 1 -- 172.21.15.45:0/148083589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c37750 msgr2=0x55e446c350b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f897f7fe640 1 --2- 172.21.15.45:0/148083589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c37750 0x55e446c350b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.305+0000 7f897f7fe640 1 -- 172.21.15.45:0/148083589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8978011440 con 0x55e446c2e980 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f897f7fe640 1 --2- 172.21.15.45:0/148083589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 0x55e446c34b70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8974012cd0 tx=0x7f897400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f360e026ecc185c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f895f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f895f7fe640 1 -- 172.21.15.45:0/148083589 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974017070 con 0x55e446c2e980 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f895f7fe640 1 -- 172.21.15.45:0/148083589 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8974012110 con 0x55e446c2e980 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f898947a740 1 -- 172.21.15.45:0/148083589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e446b8f010 con 0x55e446c2e980 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f895f7fe640 1 -- 172.21.15.45:0/148083589 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8974004050 con 0x55e446c2e980 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f898947a740 10 client.26165 fetch_fsmap learned FSMap version 31 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f898947a740 1 -- 172.21.15.45:0/148083589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e446b946a0 con 0x55e446c2e980 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f895f7fe640 1 -- 172.21.15.45:0/148083589 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e446b8f010 con 0x55e446c2e980 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f895f7fe640 1 -- 172.21.15.45:0/148083589 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8974025480 con 0x55e446c2e980 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f898947a740 10 client.26165 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f898947a740 10 client.26165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f898947a740 20 client.26165 populate_metadata read hostname 'smithi045' 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f898947a740 1 --2- 172.21.15.45:0/148083589 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e446c47520 0x55e446c49940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f898947a740 4 client.26165 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.306+0000 7f898947a740 1 -- 172.21.15.45:0/148083589 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e446b8ed30 con 0x55e446c47520 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.307+0000 7f897ffff640 1 --2- 172.21.15.45:0/148083589 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e446c47520 0x55e446c49940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.307+0000 7f895f7fe640 10 client.26165 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.307+0000 7f897ffff640 1 --2- 172.21.15.45:0/148083589 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e446c47520 0x55e446c49940 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e641ec04cad8451f server_cookie=50d903436b67d58a in_seq=0 out_seq=0 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f895f7fe640 1 -- 172.21.15.45:0/148083589 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e446b8ed30 con 0x55e446c47520 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f895f7fe640 10 client.26165 handle_command_reply: tid=0 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f898947a740 1 client.26165 shutdown 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f898947a740 1 -- 172.21.15.45:0/148083589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 msgr2=0x55e446c34b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f898947a740 1 --2- 172.21.15.45:0/148083589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 0x55e446c34b70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8974012cd0 tx=0x7f897400f790 comp rx=0 tx=0).stop 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f898947a740 1 -- 172.21.15.45:0/148083589 shutdown_connections 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f898947a740 1 --2- 172.21.15.45:0/148083589 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e446c47520 0x55e446c49940 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f898947a740 1 --2- 172.21.15.45:0/148083589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c37750 0x55e446c350b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f898947a740 1 --2- 172.21.15.45:0/148083589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e980 0x55e446c34b70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f898947a740 1 --2- 172.21.15.45:0/148083589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e060 0x55e446c34630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f895f7fe640 0 client.26165 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f898947a740 1 -- 172.21.15.45:0/148083589 >> 172.21.15.45:0/148083589 conn(0x55e446c28980 msgr2=0x55e446c2a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f898947a740 1 -- 172.21.15.45:0/148083589 shutdown_connections 2024-01-31T05:38:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f898947a740 1 -- 172.21.15.45:0/148083589 wait complete. 2024-01-31T05:38:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.308+0000 7f898947a740 20 client.26165 trim_cache size 0 max 16384 2024-01-31T05:38:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.324+0000 7f898947a740 1 Processor -- start 2024-01-31T05:38:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.324+0000 7f898947a740 1 -- start start 2024-01-31T05:38:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f898947a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 0x55e446c36d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f898947a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e980 0x55e446c33610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f898947a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c37750 0x55e446c34ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f898947a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446c240b0 con 0x55e446c2e060 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f898947a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446c4eb70 con 0x55e446c37750 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f898947a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446c287b0 con 0x55e446c2e980 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f897ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 0x55e446c36d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f897ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 0x55e446c36d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60718/0 (socket says 172.21.15.45:60718) 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f897ffff640 1 -- 172.21.15.45:0/405374377 learned_addr learned my addr 172.21.15.45:0/405374377 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f897f7fe640 1 --2- 172.21.15.45:0/405374377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e980 0x55e446c33610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f8984c4e640 1 --2- 172.21.15.45:0/405374377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c37750 0x55e446c34ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f897ffff640 1 -- 172.21.15.45:0/405374377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e980 msgr2=0x55e446c33610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f897ffff640 1 --2- 172.21.15.45:0/405374377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e980 0x55e446c33610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f897ffff640 1 -- 172.21.15.45:0/405374377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c37750 msgr2=0x55e446c34ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f897ffff640 1 --2- 172.21.15.45:0/405374377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c37750 0x55e446c34ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f897ffff640 1 -- 172.21.15.45:0/405374377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e446b31030 con 0x55e446c2e060 2024-01-31T05:38:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f897ffff640 1 --2- 172.21.15.45:0/405374377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 0x55e446c36d60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f89700bb6f0 tx=0x7f89700db040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9927ac151288e5b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.325+0000 7f8965ffb640 1 -- 172.21.15.45:0/405374377 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89700b7aa0 con 0x55e446c2e060 2024-01-31T05:38:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.326+0000 7f8965ffb640 1 -- 172.21.15.45:0/405374377 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89700bb530 con 0x55e446c2e060 2024-01-31T05:38:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.326+0000 7f8965ffb640 1 -- 172.21.15.45:0/405374377 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89700bbe60 con 0x55e446c2e060 2024-01-31T05:38:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.326+0000 7f898947a740 1 -- 172.21.15.45:0/405374377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 msgr2=0x55e446c36d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.326+0000 7f898947a740 1 --2- 172.21.15.45:0/405374377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 0x55e446c36d60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f89700bb6f0 tx=0x7f89700db040 comp rx=0 tx=0).stop 2024-01-31T05:38:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.326+0000 7f898947a740 1 -- 172.21.15.45:0/405374377 shutdown_connections 2024-01-31T05:38:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.326+0000 7f898947a740 1 --2- 172.21.15.45:0/405374377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c37750 0x55e446c34ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.326+0000 7f898947a740 1 --2- 172.21.15.45:0/405374377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e980 0x55e446c33610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.326+0000 7f898947a740 1 --2- 172.21.15.45:0/405374377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c2e060 0x55e446c36d60 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.326+0000 7f898947a740 1 -- 172.21.15.45:0/405374377 >> 172.21.15.45:0/405374377 conn(0x55e446c28980 msgr2=0x55e446c32f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.326+0000 7f898947a740 1 -- 172.21.15.45:0/405374377 shutdown_connections 2024-01-31T05:38:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.326+0000 7f898947a740 1 -- 172.21.15.45:0/405374377 wait complete. 2024-01-31T05:38:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f898947a740 1 -- start start 2024-01-31T05:38:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f898947a740 1 Processor -- start 2024-01-31T05:38:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f898947a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e060 0x55e446c3ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f898947a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e980 0x55e446c3d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f898947a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c37750 0x55e446c3d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f898947a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446c3ecf0 con 0x55e446c37750 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f898947a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446c9da50 con 0x55e446c2e060 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f898947a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446c9db90 con 0x55e446c2e980 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f898947a740 10 client.0 _ll_register_callbacks cb 0x7f8978009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f8984c4e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c37750 0x55e446c3d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f8984c4e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c37750 0x55e446c3d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60732/0 (socket says 172.21.15.45:60732) 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f8984c4e640 1 -- 172.21.15.45:0/2644594393 learned_addr learned my addr 172.21.15.45:0/2644594393 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f897ffff640 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e060 0x55e446c3ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.327+0000 7f897f7fe640 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e980 0x55e446c3d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f8984c4e640 1 -- 172.21.15.45:0/2644594393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e980 msgr2=0x55e446c3d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f8984c4e640 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e980 0x55e446c3d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f8984c4e640 1 -- 172.21.15.45:0/2644594393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e060 msgr2=0x55e446c3ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f8984c4e640 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e060 0x55e446c3ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f8984c4e640 1 -- 172.21.15.45:0/2644594393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89801a6260 con 0x55e446c37750 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f897ffff640 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e060 0x55e446c3ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f8984c4e640 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c37750 0x55e446c3d8f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8978011790 tx=0x7f8978016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccc5d3186c4fca7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f895f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f895f7fe640 1 -- 172.21.15.45:0/2644594393 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978021040 con 0x55e446c37750 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f898947a740 1 -- 172.21.15.45:0/2644594393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f897800fe60 con 0x55e446c37750 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f895f7fe640 1 -- 172.21.15.45:0/2644594393 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f897800fbb0 con 0x55e446c37750 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f895f7fe640 1 -- 172.21.15.45:0/2644594393 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8978011ea0 con 0x55e446c37750 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f898947a740 10 client.16674 fetch_fsmap learned FSMap version 31 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f898947a740 1 -- 172.21.15.45:0/2644594393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e446b56a50 con 0x55e446c37750 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f895f7fe640 1 -- 172.21.15.45:0/2644594393 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f897800fe60 con 0x55e446c37750 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f895f7fe640 1 -- 172.21.15.45:0/2644594393 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f89780042d0 con 0x55e446c37750 2024-01-31T05:38:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f898947a740 10 client.16674 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:38:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f898947a740 10 client.16674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:38:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f898947a740 20 client.16674 populate_metadata read hostname 'smithi045' 2024-01-31T05:38:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f898947a740 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e446c9f340 0x55e446ca1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f898947a740 4 client.16674 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:38:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.328+0000 7f898947a740 1 -- 172.21.15.45:0/2644594393 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e446895600 con 0x55e446c9f340 2024-01-31T05:38:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.329+0000 7f897ffff640 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e446c9f340 0x55e446ca1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.329+0000 7f895f7fe640 10 client.16674 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.329+0000 7f897ffff640 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e446c9f340 0x55e446ca1760 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7a73166449ed6c4 server_cookie=b9ebbdc7e044ffb6 in_seq=0 out_seq=0 2024-01-31T05:38:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f895f7fe640 1 -- 172.21.15.45:0/2644594393 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e446895600 con 0x55e446c9f340 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f895f7fe640 10 client.16674 handle_command_reply: tid=0 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f898947a740 1 client.16674 shutdown 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f898947a740 1 -- 172.21.15.45:0/2644594393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c37750 msgr2=0x55e446c3d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f898947a740 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c37750 0x55e446c3d8f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8978011790 tx=0x7f8978016000 comp rx=0 tx=0).stop 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f898947a740 1 -- 172.21.15.45:0/2644594393 shutdown_connections 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f898947a740 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55e446c9f340 0x55e446ca1760 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f898947a740 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e446c37750 0x55e446c3d8f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f898947a740 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e446c2e980 0x55e446c3d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f895f7fe640 0 client.16674 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f898947a740 1 --2- 172.21.15.45:0/2644594393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e446c2e060 0x55e446c3ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f898947a740 1 -- 172.21.15.45:0/2644594393 >> 172.21.15.45:0/2644594393 conn(0x55e446c28980 msgr2=0x55e446c27190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f898947a740 1 -- 172.21.15.45:0/2644594393 shutdown_connections 2024-01-31T05:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f898947a740 1 -- 172.21.15.45:0/2644594393 wait complete. 2024-01-31T05:38:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.390+0000 7f898947a740 20 client.16674 trim_cache size 0 max 16384 2024-01-31T05:38:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:38:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:38:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:38:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:38:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.391+0000 7f89866b1640 1 -- 172.21.15.45:0/2683374150 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f89680a14f0 msgr2=0x7f89680a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.391+0000 7f89866b1640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f89680a14f0 0x7f89680a39b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8978020000 tx=0x7f897801c7e0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.391+0000 7f89866b1640 1 -- 172.21.15.45:0/2683374150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89800ffe20 msgr2=0x7f8980199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.391+0000 7f89866b1640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89800ffe20 0x7f8980199380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f897002c180 tx=0x7f89700042e0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.391+0000 7f89866b1640 1 -- 172.21.15.45:0/2683374150 shutdown_connections 2024-01-31T05:38:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.391+0000 7f89866b1640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f89680a14f0 0x7f89680a39b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.391+0000 7f89866b1640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898019dc40 0x7f898019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.391+0000 7f89866b1640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980104ef0 0x7f89801998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.391+0000 7f89866b1640 1 --2- 172.21.15.45:0/2683374150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89800ffe20 0x7f8980199380 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.391+0000 7f89866b1640 1 -- 172.21.15.45:0/2683374150 >> 172.21.15.45:0/2683374150 conn(0x7f8980076d20 msgr2=0x7f898010b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.391+0000 7f89866b1640 1 -- 172.21.15.45:0/2683374150 shutdown_connections 2024-01-31T05:38:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:28.391+0000 7f89866b1640 1 -- 172.21.15.45:0/2683374150 wait complete. 2024-01-31T05:38:28.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 63276 22.50 MB/sec execute 27 sec latency 213.275 ms 2024-01-31T05:38:29.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:28 smithi186 ceph-mon[35659]: pgmap v502: 97 pgs: 97 active+clean; 923 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 599 op/s 2024-01-31T05:38:29.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:28 smithi045 ceph-mon[31585]: pgmap v502: 97 pgs: 97 active+clean; 923 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 599 op/s 2024-01-31T05:38:29.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:28 smithi156 ceph-mon[35720]: pgmap v502: 97 pgs: 97 active+clean; 923 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 599 op/s 2024-01-31T05:38:29.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 63742 23.11 MB/sec execute 28 sec latency 177.487 ms 2024-01-31T05:38:30.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 64234 23.10 MB/sec execute 29 sec latency 194.305 ms 2024-01-31T05:38:31.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:30 smithi156 ceph-mon[35720]: pgmap v503: 97 pgs: 97 active+clean; 923 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 399 op/s 2024-01-31T05:38:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:30 smithi186 ceph-mon[35659]: pgmap v503: 97 pgs: 97 active+clean; 923 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 399 op/s 2024-01-31T05:38:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:30 smithi045 ceph-mon[31585]: pgmap v503: 97 pgs: 97 active+clean; 923 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 399 op/s 2024-01-31T05:38:31.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 64644 23.00 MB/sec execute 30 sec latency 329.905 ms 2024-01-31T05:38:32.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 65069 23.20 MB/sec execute 31 sec latency 159.148 ms 2024-01-31T05:38:33.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:32 smithi186 ceph-mon[35659]: pgmap v504: 97 pgs: 97 active+clean; 923 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 399 op/s 2024-01-31T05:38:33.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:32 smithi045 ceph-mon[31585]: pgmap v504: 97 pgs: 97 active+clean; 923 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 399 op/s 2024-01-31T05:38:33.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:32 smithi156 ceph-mon[35720]: pgmap v504: 97 pgs: 97 active+clean; 923 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 399 op/s 2024-01-31T05:38:33.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 65487 23.22 MB/sec execute 32 sec latency 191.108 ms 2024-01-31T05:38:34.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 65794 22.85 MB/sec execute 33 sec latency 201.499 ms 2024-01-31T05:38:35.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:34 smithi186 ceph-mon[35659]: pgmap v505: 97 pgs: 97 active+clean; 941 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 596 op/s 2024-01-31T05:38:35.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:34 smithi045 ceph-mon[31585]: pgmap v505: 97 pgs: 97 active+clean; 941 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 596 op/s 2024-01-31T05:38:35.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:34 smithi156 ceph-mon[35720]: pgmap v505: 97 pgs: 97 active+clean; 941 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 596 op/s 2024-01-31T05:38:35.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 66321 22.87 MB/sec execute 34 sec latency 210.171 ms 2024-01-31T05:38:36.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 66711 23.08 MB/sec execute 35 sec latency 305.563 ms 2024-01-31T05:38:37.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:36 smithi186 ceph-mon[35659]: pgmap v506: 97 pgs: 97 active+clean; 941 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.2 MiB/s wr, 376 op/s 2024-01-31T05:38:37.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:36 smithi045 ceph-mon[31585]: pgmap v506: 97 pgs: 97 active+clean; 941 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.2 MiB/s wr, 376 op/s 2024-01-31T05:38:37.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:36 smithi156 ceph-mon[35720]: pgmap v506: 97 pgs: 97 active+clean; 941 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.2 MiB/s wr, 376 op/s 2024-01-31T05:38:37.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 67190 22.98 MB/sec execute 36 sec latency 118.123 ms 2024-01-31T05:38:38.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 67566 22.64 MB/sec execute 37 sec latency 219.888 ms 2024-01-31T05:38:39.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:38 smithi186 ceph-mon[35659]: pgmap v507: 97 pgs: 97 active+clean; 961 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 548 op/s 2024-01-31T05:38:39.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:38 smithi045 ceph-mon[31585]: pgmap v507: 97 pgs: 97 active+clean; 961 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 548 op/s 2024-01-31T05:38:39.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:38 smithi156 ceph-mon[35720]: pgmap v507: 97 pgs: 97 active+clean; 961 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 548 op/s 2024-01-31T05:38:39.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 68214 23.09 MB/sec execute 38 sec latency 176.739 ms 2024-01-31T05:38:40.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 68731 23.46 MB/sec execute 39 sec latency 230.727 ms 2024-01-31T05:38:41.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:40 smithi186 ceph-mon[35659]: pgmap v508: 97 pgs: 97 active+clean; 961 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.1 MiB/s wr, 369 op/s 2024-01-31T05:38:41.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:40 smithi045 ceph-mon[31585]: pgmap v508: 97 pgs: 97 active+clean; 961 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.1 MiB/s wr, 369 op/s 2024-01-31T05:38:41.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:40 smithi156 ceph-mon[35720]: pgmap v508: 97 pgs: 97 active+clean; 961 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.1 MiB/s wr, 369 op/s 2024-01-31T05:38:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.519+0000 7ff3c05ae640 1 -- 172.21.15.45:0/3241606480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b8115e20 msgr2=0x7ff3b8118210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.519+0000 7ff3c05ae640 1 --2- 172.21.15.45:0/3241606480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b8115e20 0x7ff3b8118210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009ca0 tx=0x7ff3b002c120 comp rx=0 tx=0).stop 2024-01-31T05:38:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.521+0000 7ff3c05ae640 1 -- 172.21.15.45:0/3241606480 shutdown_connections 2024-01-31T05:38:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.521+0000 7ff3c05ae640 1 --2- 172.21.15.45:0/3241606480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b81082c0 0x7ff3b8108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.521+0000 7ff3c05ae640 1 --2- 172.21.15.45:0/3241606480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b8115e20 0x7ff3b8118210 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.521+0000 7ff3c05ae640 1 --2- 172.21.15.45:0/3241606480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b810b4b0 0x7ff3b8107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.521+0000 7ff3c05ae640 1 -- 172.21.15.45:0/3241606480 >> 172.21.15.45:0/3241606480 conn(0x7ff3b806c220 msgr2=0x7ff3b806c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.526+0000 7ff3c05ae640 1 -- 172.21.15.45:0/3241606480 shutdown_connections 2024-01-31T05:38:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.527+0000 7ff3c05ae640 1 -- 172.21.15.45:0/3241606480 wait complete. 2024-01-31T05:38:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.528+0000 7ff3c05ae640 1 Processor -- start 2024-01-31T05:38:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.528+0000 7ff3c05ae640 1 -- start start 2024-01-31T05:38:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.528+0000 7ff3c05ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b81082c0 0x7ff3b8120a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.528+0000 7ff3c05ae640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b8115e20 0x7ff3b8118bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.528+0000 7ff3c05ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b8119110 0x7ff3b81195c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.528+0000 7ff3c05ae640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b81a9ed0 con 0x7ff3b8119110 2024-01-31T05:38:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.528+0000 7ff3c05ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b81aa040 con 0x7ff3b81082c0 2024-01-31T05:38:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.528+0000 7ff3c05ae640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b81aa200 con 0x7ff3b8115e20 2024-01-31T05:38:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.529+0000 7ff3beb4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b8119110 0x7ff3b81195c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.529+0000 7ff3be34a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b81082c0 0x7ff3b8120a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.529+0000 7ff3be34a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b81082c0 0x7ff3b8120a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:38872/0 (socket says 172.21.15.45:38872) 2024-01-31T05:38:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.529+0000 7ff3be34a640 1 -- 172.21.15.45:0/989281903 learned_addr learned my addr 172.21.15.45:0/989281903 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.531+0000 7ff3be34a640 1 -- 172.21.15.45:0/989281903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b8115e20 msgr2=0x7ff3b8118bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-31T05:38:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.531+0000 7ff3be34a640 1 --2- 172.21.15.45:0/989281903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b8115e20 0x7ff3b8118bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.531+0000 7ff3be34a640 1 -- 172.21.15.45:0/989281903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b8119110 msgr2=0x7ff3b81195c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.531+0000 7ff3be34a640 1 --2- 172.21.15.45:0/989281903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b8119110 0x7ff3b81195c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.532+0000 7ff3be34a640 1 -- 172.21.15.45:0/989281903 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b0009900 con 0x7ff3b81082c0 2024-01-31T05:38:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.533+0000 7ff3be34a640 1 --2- 172.21.15.45:0/989281903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b81082c0 0x7ff3b8120a40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800e9c0 tx=0x7ff3a800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.538+0000 7ff3af7fe640 1 -- 172.21.15.45:0/989281903 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8009700 con 0x7ff3b81082c0 2024-01-31T05:38:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.538+0000 7ff3af7fe640 1 -- 172.21.15.45:0/989281903 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3a80044e0 con 0x7ff3b81082c0 2024-01-31T05:38:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.538+0000 7ff3af7fe640 1 -- 172.21.15.45:0/989281903 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8010430 con 0x7ff3b81082c0 2024-01-31T05:38:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.538+0000 7ff3c05ae640 1 -- 172.21.15.45:0/989281903 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b81aa4e0 con 0x7ff3b81082c0 2024-01-31T05:38:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.538+0000 7ff3c05ae640 1 -- 172.21.15.45:0/989281903 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b81aa9a0 con 0x7ff3b81082c0 2024-01-31T05:38:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.542+0000 7ff3c3375740 1 Processor -- start 2024-01-31T05:38:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.548+0000 7ff3c3375740 1 -- start start 2024-01-31T05:38:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.548+0000 7ff3af7fe640 1 -- 172.21.15.45:0/989281903 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7ff3a8010590 con 0x7ff3b81082c0 2024-01-31T05:38:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.549+0000 7ff3c3375740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a665906060 0x55a665906440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.549+0000 7ff3c3375740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 0x55a66590f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.549+0000 7ff3c3375740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a66590f750 0x55a665911b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.549+0000 7ff3c3375740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a665875040 con 0x55a665906060 2024-01-31T05:38:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.549+0000 7ff3c3375740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a665872730 con 0x55a66590f750 2024-01-31T05:38:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.549+0000 7ff3c3375740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66586e020 con 0x55a665906980 2024-01-31T05:38:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.550+0000 7ff3be34a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 0x55a66590f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.550+0000 7ff3be34a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 0x55a66590f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:56796/0 (socket says 172.21.15.45:56796) 2024-01-31T05:38:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.550+0000 7ff3be34a640 1 -- 172.21.15.45:0/833688524 learned_addr learned my addr 172.21.15.45:0/833688524 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.550+0000 7ff3bdb49640 1 --2- 172.21.15.45:0/833688524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a66590f750 0x55a665911b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.550+0000 7ff3be34a640 1 -- 172.21.15.45:0/833688524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a66590f750 msgr2=0x55a665911b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.550+0000 7ff3be34a640 1 --2- 172.21.15.45:0/833688524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a66590f750 0x55a665911b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.550+0000 7ff3be34a640 1 -- 172.21.15.45:0/833688524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a665906060 msgr2=0x55a665906440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.550+0000 7ff3be34a640 1 --2- 172.21.15.45:0/833688524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a665906060 0x55a665906440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.550+0000 7ff3be34a640 1 -- 172.21.15.45:0/833688524 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a665865e40 con 0x55a665906980 2024-01-31T05:38:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.550+0000 7ff3bdb49640 1 --2- 172.21.15.45:0/833688524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a66590f750 0x55a665911b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:38:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.551+0000 7ff3be34a640 1 --2- 172.21.15.45:0/833688524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 0x55a66590f210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8003a90 tx=0x7ff3a8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7334589b16a8100 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.551+0000 7ff3acbfe640 1 -- 172.21.15.45:0/833688524 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80ce6b0 con 0x55a665906980 2024-01-31T05:38:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.551+0000 7ff3acbfe640 1 -- 172.21.15.45:0/833688524 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3a80d0320 con 0x55a665906980 2024-01-31T05:38:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.553+0000 7ff3c3375740 1 -- 172.21.15.45:0/833688524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 msgr2=0x55a66590f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.553+0000 7ff3c3375740 1 --2- 172.21.15.45:0/833688524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 0x55a66590f210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8003a90 tx=0x7ff3a8004730 comp rx=0 tx=0).stop 2024-01-31T05:38:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.553+0000 7ff3af7fe640 1 --2- 172.21.15.45:0/989281903 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff3a40a1720 0x7ff3a40a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.553+0000 7ff3af7fe640 1 -- 172.21.15.45:0/989281903 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff3a8014070 con 0x7ff3b81082c0 2024-01-31T05:38:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.553+0000 7ff3c3375740 1 -- 172.21.15.45:0/833688524 shutdown_connections 2024-01-31T05:38:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.553+0000 7ff3c3375740 1 --2- 172.21.15.45:0/833688524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a66590f750 0x55a665911b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.553+0000 7ff3c3375740 1 --2- 172.21.15.45:0/833688524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 0x55a66590f210 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.553+0000 7ff3c3375740 1 --2- 172.21.15.45:0/833688524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a665906060 0x55a665906440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.553+0000 7ff3c3375740 1 -- 172.21.15.45:0/833688524 >> 172.21.15.45:0/833688524 conn(0x55a665900980 msgr2=0x55a665900d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.554+0000 7ff3beb4b640 1 --2- 172.21.15.45:0/989281903 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff3a40a1720 0x7ff3a40a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.555+0000 7ff3beb4b640 1 --2- 172.21.15.45:0/989281903 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff3a40a1720 0x7ff3a40a3be0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff3b002c730 tx=0x7ff3b0009890 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.556+0000 7ff3c3375740 1 -- 172.21.15.45:0/833688524 shutdown_connections 2024-01-31T05:38:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.556+0000 7ff3c3375740 1 -- 172.21.15.45:0/833688524 wait complete. 2024-01-31T05:38:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.556+0000 7ff3c3375740 1 -- start start 2024-01-31T05:38:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.556+0000 7ff3c3375740 1 Processor -- start 2024-01-31T05:38:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.557+0000 7ff3c3375740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665906060 0x55a66590f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.557+0000 7ff3c3375740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a665906980 0x55a66590c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.557+0000 7ff3be34a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665906060 0x55a66590f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.557+0000 7ff3be34a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665906060 0x55a66590f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:38892/0 (socket says 172.21.15.45:38892) 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.557+0000 7ff3be34a640 1 -- 172.21.15.45:0/2119295331 learned_addr learned my addr 172.21.15.45:0/2119295331 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.557+0000 7ff3c3375740 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a66590f750 0x55a66590cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.557+0000 7ff3c3375740 1 -- 172.21.15.45:0/2119295331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6658271b0 con 0x55a665906980 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.557+0000 7ff3c3375740 1 -- 172.21.15.45:0/2119295331 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a665826e50 con 0x55a665906060 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.557+0000 7ff3c3375740 1 -- 172.21.15.45:0/2119295331 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6657ec090 con 0x55a66590f750 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.557+0000 7ff3bdb49640 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a665906980 0x55a66590c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.558+0000 7ff3c3375740 10 client.0 _ll_register_callbacks cb 0x55a665811a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.558+0000 7ff3be34a640 1 -- 172.21.15.45:0/2119295331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a66590f750 msgr2=0x55a66590cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.558+0000 7ff3beb4b640 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a66590f750 0x55a66590cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.558+0000 7ff3be34a640 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a66590f750 0x55a66590cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.558+0000 7ff3be34a640 1 -- 172.21.15.45:0/2119295331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a665906980 msgr2=0x55a66590c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.558+0000 7ff3be34a640 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a665906980 0x55a66590c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.558+0000 7ff3be34a640 1 -- 172.21.15.45:0/2119295331 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b81aa9a0 con 0x55a665906060 2024-01-31T05:38:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.558+0000 7ff3beb4b640 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a66590f750 0x55a66590cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:38:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.559+0000 7ff3be34a640 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665906060 0x55a66590f1e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800c6a0 tx=0x7ff3a8005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aba49ad8d8e7850 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.559+0000 7ff3a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:38:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.560+0000 7ff3a1ffb640 1 -- 172.21.15.45:0/2119295331 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80973e0 con 0x55a665906060 2024-01-31T05:38:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.560+0000 7ff3c3375740 1 -- 172.21.15.45:0/2119295331 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a665867010 con 0x55a665906060 2024-01-31T05:38:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.560+0000 7ff3a1ffb640 1 -- 172.21.15.45:0/2119295331 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff3a80dd450 con 0x55a665906060 2024-01-31T05:38:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.561+0000 7ff3a1ffb640 1 -- 172.21.15.45:0/2119295331 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3a809ae70 con 0x55a665906060 2024-01-31T05:38:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.562+0000 7ff3c3375740 10 client.26180 fetch_fsmap learned FSMap version 31 2024-01-31T05:38:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.562+0000 7ff3c3375740 1 -- 172.21.15.45:0/2119295331 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a66586c6a0 con 0x55a665906060 2024-01-31T05:38:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.563+0000 7ff3a1ffb640 1 -- 172.21.15.45:0/2119295331 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3a80cd070 con 0x55a665906060 2024-01-31T05:38:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.564+0000 7ff3a1ffb640 1 -- 172.21.15.45:0/2119295331 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff3a801f060 con 0x55a665906060 2024-01-31T05:38:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.564+0000 7ff3c3375740 10 client.26180 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:38:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.564+0000 7ff3c3375740 10 client.26180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:38:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.564+0000 7ff3c3375740 20 client.26180 populate_metadata read hostname 'smithi045' 2024-01-31T05:38:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.564+0000 7ff3c3375740 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a66591f000 0x55a665921480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.564+0000 7ff3c3375740 4 client.26180 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:38:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.564+0000 7ff3c3375740 1 -- 172.21.15.45:0/2119295331 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a665867010 con 0x55a66591f000 2024-01-31T05:38:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.565+0000 7ff3bdb49640 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a66591f000 0x55a665921480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.565+0000 7ff3bdb49640 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a66591f000 0x55a665921480 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b16624958c41d6df server_cookie=f5a88a57c186d38f in_seq=0 out_seq=0 2024-01-31T05:38:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.566+0000 7ff3a1ffb640 10 client.26180 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.566+0000 7ff3a1ffb640 1 -- 172.21.15.45:0/2119295331 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a665867010 con 0x55a66591f000 2024-01-31T05:38:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.566+0000 7ff3a1ffb640 10 client.26180 handle_command_reply: tid=0 2024-01-31T05:38:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.566+0000 7ff3c3375740 1 client.26180 shutdown 2024-01-31T05:38:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.568+0000 7ff3c3375740 1 -- 172.21.15.45:0/2119295331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665906060 msgr2=0x55a66590f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.568+0000 7ff3c3375740 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665906060 0x55a66590f1e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800c6a0 tx=0x7ff3a8005b20 comp rx=0 tx=0).stop 2024-01-31T05:38:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.568+0000 7ff3c3375740 1 -- 172.21.15.45:0/2119295331 shutdown_connections 2024-01-31T05:38:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.568+0000 7ff3c3375740 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a66591f000 0x55a665921480 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.568+0000 7ff3c3375740 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a665906980 0x55a66590c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.568+0000 7ff3c3375740 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a66590f750 0x55a66590cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.568+0000 7ff3c3375740 1 --2- 172.21.15.45:0/2119295331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665906060 0x55a66590f1e0 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800c6a0 tx=0x7ff3a8005b20 comp rx=0 tx=0).stop 2024-01-31T05:38:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.568+0000 7ff3c3375740 1 -- 172.21.15.45:0/2119295331 >> 172.21.15.45:0/2119295331 conn(0x55a665900980 msgr2=0x55a6659026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.569+0000 7ff3a1ffb640 0 client.26180 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.572+0000 7ff3c3375740 1 -- 172.21.15.45:0/2119295331 shutdown_connections 2024-01-31T05:38:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.572+0000 7ff3c3375740 1 -- 172.21.15.45:0/2119295331 wait complete. 2024-01-31T05:38:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.573+0000 7ff3c3375740 20 client.26180 trim_cache size 0 max 16384 2024-01-31T05:38:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3c3375740 1 Processor -- start 2024-01-31T05:38:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3c3375740 1 -- start start 2024-01-31T05:38:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3c3375740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665906980 0x55a665921010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3c3375740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 0x55a665907950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3c3375740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665921550 0x55a665925e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3c3375740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a665900800 con 0x55a66590f750 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3c3375740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66590ab00 con 0x55a665906980 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3c3375740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66590cf90 con 0x55a665921550 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3bdb49640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 0x55a665907950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3bdb49640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 0x55a665907950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41972/0 (socket says 172.21.15.45:41972) 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3bdb49640 1 -- 172.21.15.45:0/1923015655 learned_addr learned my addr 172.21.15.45:0/1923015655 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3beb4b640 1 --2- 172.21.15.45:0/1923015655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665921550 0x55a665925e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3be34a640 1 --2- 172.21.15.45:0/1923015655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665906980 0x55a665921010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3bdb49640 1 -- 172.21.15.45:0/1923015655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665921550 msgr2=0x55a665925e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3bdb49640 1 --2- 172.21.15.45:0/1923015655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665921550 0x55a665925e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3bdb49640 1 -- 172.21.15.45:0/1923015655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665906980 msgr2=0x55a665921010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3bdb49640 1 --2- 172.21.15.45:0/1923015655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665906980 0x55a665921010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.592+0000 7ff3bdb49640 1 -- 172.21.15.45:0/1923015655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a665809030 con 0x55a66590f750 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.593+0000 7ff3bdb49640 1 --2- 172.21.15.45:0/1923015655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 0x55a665907950 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4000c00 tx=0x7ff3b400abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15f44f4e5d6c57c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.593+0000 7ff3a3fff640 1 -- 172.21.15.45:0/1923015655 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4019350 con 0x55a66590f750 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.593+0000 7ff3a3fff640 1 -- 172.21.15.45:0/1923015655 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3b400c040 con 0x55a66590f750 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.593+0000 7ff3a3fff640 1 -- 172.21.15.45:0/1923015655 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4015db0 con 0x55a66590f750 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 -- 172.21.15.45:0/1923015655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 msgr2=0x55a665907950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 --2- 172.21.15.45:0/1923015655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 0x55a665907950 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4000c00 tx=0x7ff3b400abd0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 -- 172.21.15.45:0/1923015655 shutdown_connections 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 --2- 172.21.15.45:0/1923015655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 0x55a665907950 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 --2- 172.21.15.45:0/1923015655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665921550 0x55a665925e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 --2- 172.21.15.45:0/1923015655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665906980 0x55a665921010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 -- 172.21.15.45:0/1923015655 >> 172.21.15.45:0/1923015655 conn(0x55a665900980 msgr2=0x55a665900d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 -- 172.21.15.45:0/1923015655 shutdown_connections 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 -- 172.21.15.45:0/1923015655 wait complete. 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 -- start start 2024-01-31T05:38:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 Processor -- start 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 0x55a6659166b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 0x55a665914b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665921550 0x55a665915090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6659155d0 con 0x55a66590f750 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a665915740 con 0x55a665921550 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a665915900 con 0x55a665906980 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3c3375740 10 client.0 _ll_register_callbacks cb 0x7ff3a80036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3bdb49640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 0x55a665914b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3bdb49640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 0x55a665914b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41984/0 (socket says 172.21.15.45:41984) 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3bdb49640 1 -- 172.21.15.45:0/3922942677 learned_addr learned my addr 172.21.15.45:0/3922942677 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3be34a640 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 0x55a6659166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.594+0000 7ff3beb4b640 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665921550 0x55a665915090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3bdb49640 1 -- 172.21.15.45:0/3922942677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 msgr2=0x55a6659166b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3bdb49640 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 0x55a6659166b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3bdb49640 1 -- 172.21.15.45:0/3922942677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665921550 msgr2=0x55a665915090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3bdb49640 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665921550 0x55a665915090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3bdb49640 1 -- 172.21.15.45:0/3922942677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b400a6f0 con 0x55a66590f750 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3be34a640 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 0x55a6659166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3beb4b640 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665921550 0x55a665915090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3bdb49640 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 0x55a665914b50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4015a90 tx=0x7ff3b400afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5fb00f96c37e85 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3a1ffb640 1 -- 172.21.15.45:0/3922942677 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4015d30 con 0x55a66590f750 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3a1ffb640 1 -- 172.21.15.45:0/3922942677 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff3b4019150 con 0x55a66590f750 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3c3375740 1 -- 172.21.15.45:0/3922942677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3a80d0320 con 0x55a66590f750 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3a1ffb640 1 -- 172.21.15.45:0/3922942677 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3b4019310 con 0x55a66590f750 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3c3375740 10 client.16686 fetch_fsmap learned FSMap version 31 2024-01-31T05:38:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3c3375740 1 -- 172.21.15.45:0/3922942677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a66582ea50 con 0x55a66590f750 2024-01-31T05:38:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3a1ffb640 1 -- 172.21.15.45:0/3922942677 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3a80d0320 con 0x55a66590f750 2024-01-31T05:38:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3a1ffb640 1 -- 172.21.15.45:0/3922942677 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff3b4031070 con 0x55a66590f750 2024-01-31T05:38:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3c3375740 10 client.16686 fetch_fsmap finished waiting for FSMap version 31 2024-01-31T05:38:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3c3375740 10 client.16686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:38:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.595+0000 7ff3c3375740 20 client.16686 populate_metadata read hostname 'smithi045' 2024-01-31T05:38:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.596+0000 7ff3c3375740 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a6659769c0 0x55a665978de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.596+0000 7ff3c3375740 4 client.16686 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:38:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.596+0000 7ff3c3375740 1 -- 172.21.15.45:0/3922942677 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a665866d30 con 0x55a6659769c0 2024-01-31T05:38:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.596+0000 7ff3be34a640 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a6659769c0 0x55a665978de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.596+0000 7ff3be34a640 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a6659769c0 0x55a665978de0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f23460775084d39f server_cookie=6b4985b6a2aec08c in_seq=0 out_seq=0 2024-01-31T05:38:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.596+0000 7ff3a1ffb640 10 client.16686 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:41.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 69065 23.27 MB/sec execute 40 sec latency 219.116 ms 2024-01-31T05:38:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3a1ffb640 1 -- 172.21.15.45:0/3922942677 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a665866d30 con 0x55a6659769c0 2024-01-31T05:38:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3a1ffb640 10 client.16686 handle_command_reply: tid=0 2024-01-31T05:38:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3c3375740 1 client.16686 shutdown 2024-01-31T05:38:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3c3375740 1 -- 172.21.15.45:0/3922942677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 msgr2=0x55a665914b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3c3375740 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 0x55a665914b50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4015a90 tx=0x7ff3b400afb0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3c3375740 1 -- 172.21.15.45:0/3922942677 shutdown_connections 2024-01-31T05:38:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3c3375740 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55a6659769c0 0x55a665978de0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3c3375740 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a66590f750 0x55a665914b50 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3c3375740 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a665921550 0x55a665915090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3c3375740 1 --2- 172.21.15.45:0/3922942677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a665906980 0x55a6659166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3a1ffb640 0 client.16686 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3c3375740 1 -- 172.21.15.45:0/3922942677 >> 172.21.15.45:0/3922942677 conn(0x55a665900980 msgr2=0x55a665907950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3c3375740 1 -- 172.21.15.45:0/3922942677 shutdown_connections 2024-01-31T05:38:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3c3375740 1 -- 172.21.15.45:0/3922942677 wait complete. 2024-01-31T05:38:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.679+0000 7ff3c3375740 20 client.16686 trim_cache size 0 max 16384 2024-01-31T05:38:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:38:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:38:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:38:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:38:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.680+0000 7ff3c05ae640 1 -- 172.21.15.45:0/989281903 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff3a40a1720 msgr2=0x7ff3a40a3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.680+0000 7ff3c05ae640 1 --2- 172.21.15.45:0/989281903 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff3a40a1720 0x7ff3a40a3be0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff3b002c730 tx=0x7ff3b0009890 comp rx=0 tx=0).stop 2024-01-31T05:38:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.680+0000 7ff3c05ae640 1 -- 172.21.15.45:0/989281903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b81082c0 msgr2=0x7ff3b8120a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.680+0000 7ff3c05ae640 1 --2- 172.21.15.45:0/989281903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b81082c0 0x7ff3b8120a40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800e9c0 tx=0x7ff3a800ee80 comp rx=0 tx=0).stop 2024-01-31T05:38:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.680+0000 7ff3c05ae640 1 -- 172.21.15.45:0/989281903 shutdown_connections 2024-01-31T05:38:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.680+0000 7ff3c05ae640 1 --2- 172.21.15.45:0/989281903 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff3a40a1720 0x7ff3a40a3be0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.680+0000 7ff3c05ae640 1 --2- 172.21.15.45:0/989281903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b8119110 0x7ff3b81195c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.680+0000 7ff3c05ae640 1 --2- 172.21.15.45:0/989281903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b8115e20 0x7ff3b8118bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.680+0000 7ff3c05ae640 1 --2- 172.21.15.45:0/989281903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3b81082c0 0x7ff3b8120a40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.680+0000 7ff3c05ae640 1 -- 172.21.15.45:0/989281903 >> 172.21.15.45:0/989281903 conn(0x7ff3b806c220 msgr2=0x7ff3b8109fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.680+0000 7ff3c05ae640 1 -- 172.21.15.45:0/989281903 shutdown_connections 2024-01-31T05:38:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:41.680+0000 7ff3c05ae640 1 -- 172.21.15.45:0/989281903 wait complete. 2024-01-31T05:38:42.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 69391 23.10 MB/sec execute 41 sec latency 251.155 ms 2024-01-31T05:38:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:42 smithi186 ceph-mon[35659]: pgmap v509: 97 pgs: 97 active+clean; 961 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.1 MiB/s wr, 369 op/s 2024-01-31T05:38:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:42 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:38:43.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:42 smithi045 ceph-mon[31585]: pgmap v509: 97 pgs: 97 active+clean; 961 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.1 MiB/s wr, 369 op/s 2024-01-31T05:38:43.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:42 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:38:43.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:42 smithi156 ceph-mon[35720]: pgmap v509: 97 pgs: 97 active+clean; 961 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.1 MiB/s wr, 369 op/s 2024-01-31T05:38:43.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:42 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:38:43.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 69862 23.05 MB/sec execute 42 sec latency 165.976 ms 2024-01-31T05:38:44.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 70336 23.23 MB/sec execute 43 sec latency 140.528 ms 2024-01-31T05:38:45.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:44 smithi186 ceph-mon[35659]: pgmap v510: 97 pgs: 97 active+clean; 986 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.3 MiB/s wr, 559 op/s 2024-01-31T05:38:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:44 smithi045 ceph-mon[31585]: pgmap v510: 97 pgs: 97 active+clean; 986 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.3 MiB/s wr, 559 op/s 2024-01-31T05:38:45.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:44 smithi156 ceph-mon[35720]: pgmap v510: 97 pgs: 97 active+clean; 986 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.3 MiB/s wr, 559 op/s 2024-01-31T05:38:45.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 70797 23.36 MB/sec execute 44 sec latency 63.094 ms 2024-01-31T05:38:46.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 71154 23.27 MB/sec execute 45 sec latency 279.340 ms 2024-01-31T05:38:47.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:46 smithi186 ceph-mon[35659]: pgmap v511: 97 pgs: 97 active+clean; 986 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 362 op/s 2024-01-31T05:38:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:46 smithi045 ceph-mon[31585]: pgmap v511: 97 pgs: 97 active+clean; 986 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 362 op/s 2024-01-31T05:38:47.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:46 smithi156 ceph-mon[35720]: pgmap v511: 97 pgs: 97 active+clean; 986 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 362 op/s 2024-01-31T05:38:47.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 71614 23.07 MB/sec execute 46 sec latency 203.403 ms 2024-01-31T05:38:48.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 72136 23.18 MB/sec execute 47 sec latency 226.308 ms 2024-01-31T05:38:49.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:48 smithi186 ceph-mon[35659]: pgmap v512: 97 pgs: 97 active+clean; 1006 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 557 op/s 2024-01-31T05:38:49.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:48 smithi045 ceph-mon[31585]: pgmap v512: 97 pgs: 97 active+clean; 1006 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 557 op/s 2024-01-31T05:38:49.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:48 smithi156 ceph-mon[35720]: pgmap v512: 97 pgs: 97 active+clean; 1006 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 557 op/s 2024-01-31T05:38:49.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 72616 23.43 MB/sec execute 48 sec latency 195.330 ms 2024-01-31T05:38:50.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 73077 23.49 MB/sec execute 49 sec latency 204.250 ms 2024-01-31T05:38:51.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:50 smithi186 ceph-mon[35659]: pgmap v513: 97 pgs: 97 active+clean; 1006 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.9 MiB/s wr, 386 op/s 2024-01-31T05:38:51.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:50 smithi045 ceph-mon[31585]: pgmap v513: 97 pgs: 97 active+clean; 1006 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.9 MiB/s wr, 386 op/s 2024-01-31T05:38:51.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:50 smithi156 ceph-mon[35720]: pgmap v513: 97 pgs: 97 active+clean; 1006 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.9 MiB/s wr, 386 op/s 2024-01-31T05:38:51.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 73614 23.45 MB/sec execute 50 sec latency 207.369 ms 2024-01-31T05:38:52.648 INFO:tasks.workunit.client.0.smithi045.stdout: 10 73940 23.48 MB/sec execute 51 sec latency 287.251 ms 2024-01-31T05:38:53.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:52 smithi186 ceph-mon[35659]: pgmap v514: 97 pgs: 97 active+clean; 1006 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.9 MiB/s wr, 386 op/s 2024-01-31T05:38:53.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:52 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:38:53.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:52 smithi045 ceph-mon[31585]: pgmap v514: 97 pgs: 97 active+clean; 1006 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.9 MiB/s wr, 386 op/s 2024-01-31T05:38:53.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:52 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:38:53.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:52 smithi156 ceph-mon[35720]: pgmap v514: 97 pgs: 97 active+clean; 1006 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.9 MiB/s wr, 386 op/s 2024-01-31T05:38:53.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:52 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-01-31T05:38:53.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 74304 23.36 MB/sec execute 52 sec latency 97.846 ms 2024-01-31T05:38:54.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 74631 23.16 MB/sec execute 53 sec latency 206.838 ms 2024-01-31T05:38:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.789+0000 7f834b806640 1 -- 172.21.15.45:0/3472699246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8344100300 msgr2=0x7f8344100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.789+0000 7f834b806640 1 --2- 172.21.15.45:0/3472699246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8344100300 0x7f8344100780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8334009930 tx=0x7f833402bc30 comp rx=0 tx=0).stop 2024-01-31T05:38:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.789+0000 7f834b806640 1 -- 172.21.15.45:0/3472699246 shutdown_connections 2024-01-31T05:38:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.789+0000 7f834b806640 1 --2- 172.21.15.45:0/3472699246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8344100cc0 0x7f834410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.789+0000 7f834b806640 1 --2- 172.21.15.45:0/3472699246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8344100300 0x7f8344100780 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.789+0000 7f834b806640 1 --2- 172.21.15.45:0/3472699246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8344104c20 0x7f8344105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.789+0000 7f834b806640 1 -- 172.21.15.45:0/3472699246 >> 172.21.15.45:0/3472699246 conn(0x7f83440761b0 msgr2=0x7f83440fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.790+0000 7f834b806640 1 -- 172.21.15.45:0/3472699246 shutdown_connections 2024-01-31T05:38:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.790+0000 7f834b806640 1 -- 172.21.15.45:0/3472699246 wait complete. 2024-01-31T05:38:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.790+0000 7f834b806640 1 Processor -- start 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.790+0000 7f834b806640 1 -- start start 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f834b806640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8344100300 0x7f8344197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f834b806640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8344100cc0 0x7f83441984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f834b806640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8344104c20 0x7f83441a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f834b806640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8344198d40 con 0x7f8344100cc0 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f834b806640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83441a3540 con 0x7f8344104c20 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f834b806640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83441a3680 con 0x7f8344100300 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f83495a2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8344100300 0x7f8344197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f83495a2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8344100300 0x7f8344197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:57128/0 (socket says 172.21.15.45:57128) 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f83495a2640 1 -- 172.21.15.45:0/1225340863 learned_addr learned my addr 172.21.15.45:0/1225340863 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f8348da1640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8344100cc0 0x7f83441984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f8349da3640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8344104c20 0x7f83441a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f8349da3640 1 -- 172.21.15.45:0/1225340863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8344100300 msgr2=0x7f8344197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f8349da3640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8344100300 0x7f8344197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f8349da3640 1 -- 172.21.15.45:0/1225340863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8344100cc0 msgr2=0x7f83441984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f8349da3640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8344100cc0 0x7f83441984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.791+0000 7f8349da3640 1 -- 172.21.15.45:0/1225340863 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8340009490 con 0x7f8344104c20 2024-01-31T05:38:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.792+0000 7f8349da3640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8344104c20 0x7f83441a3000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f833c00e970 tx=0x7f833c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.792+0000 7f833a7fc640 1 -- 172.21.15.45:0/1225340863 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c004d20 con 0x7f8344104c20 2024-01-31T05:38:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.792+0000 7f833a7fc640 1 -- 172.21.15.45:0/1225340863 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f833c004e80 con 0x7f8344104c20 2024-01-31T05:38:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.792+0000 7f834b806640 1 -- 172.21.15.45:0/1225340863 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8334009560 con 0x7f8344104c20 2024-01-31T05:38:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.792+0000 7f833a7fc640 1 -- 172.21.15.45:0/1225340863 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c011470 con 0x7f8344104c20 2024-01-31T05:38:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.792+0000 7f834b806640 1 -- 172.21.15.45:0/1225340863 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83441a3c30 con 0x7f8344104c20 2024-01-31T05:38:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.795+0000 7f834e5c5740 1 Processor -- start 2024-01-31T05:38:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.795+0000 7f833a7fc640 1 -- 172.21.15.45:0/1225340863 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f833c00cac0 con 0x7f8344104c20 2024-01-31T05:38:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f834e5c5740 1 -- start start 2024-01-31T05:38:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f834e5c5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 0x563203774440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f834e5c5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 0x56320377d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f834e5c5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x56320377fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f834e5c5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632036e3040 con 0x563203774980 2024-01-31T05:38:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f834e5c5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632036e0730 con 0x563203774060 2024-01-31T05:38:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f834e5c5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632036dc020 con 0x56320377d750 2024-01-31T05:38:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f8348da1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 0x563203774440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f83495a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 0x56320377d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f8348da1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 0x563203774440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:58748/0 (socket says 172.21.15.45:58748) 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f83495a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 0x56320377d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53416/0 (socket says 172.21.15.45:53416) 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f8348da1640 1 -- 172.21.15.45:0/3102041972 learned_addr learned my addr 172.21.15.45:0/3102041972 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f8348da1640 1 --2- 172.21.15.45:0/3102041972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x56320377fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f833a7fc640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f83300a1570 0x7f83300a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.796+0000 7f8348da1640 1 -- 172.21.15.45:0/3102041972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 msgr2=0x56320377fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.797+0000 7f8348da1640 1 --2- 172.21.15.45:0/3102041972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x56320377fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.797+0000 7f8348da1640 1 -- 172.21.15.45:0/3102041972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 msgr2=0x56320377d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.797+0000 7f8348da1640 1 --2- 172.21.15.45:0/3102041972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 0x56320377d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.797+0000 7f8348da1640 1 -- 172.21.15.45:0/3102041972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632036d3e40 con 0x563203774060 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.797+0000 7f8349da3640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f83300a1570 0x7f83300a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.797+0000 7f833a7fc640 1 -- 172.21.15.45:0/1225340863 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f833c0ccf90 con 0x7f8344104c20 2024-01-31T05:38:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.797+0000 7f83495a2640 1 --2- 172.21.15.45:0/3102041972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 0x56320377d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:38:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.797+0000 7f8348da1640 1 --2- 172.21.15.45:0/3102041972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 0x563203774440 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f833402c680 tx=0x7f833402c610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c8814f9221c7e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.797+0000 7f832f7fe640 1 -- 172.21.15.45:0/3102041972 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833402c780 con 0x563203774060 2024-01-31T05:38:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.797+0000 7f832f7fe640 1 -- 172.21.15.45:0/3102041972 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8334004030 con 0x563203774060 2024-01-31T05:38:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.797+0000 7f832f7fe640 1 -- 172.21.15.45:0/3102041972 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83340054a0 con 0x563203774060 2024-01-31T05:38:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.798+0000 7f834e5c5740 1 -- 172.21.15.45:0/3102041972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 msgr2=0x563203774440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.798+0000 7f834e5c5740 1 --2- 172.21.15.45:0/3102041972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 0x563203774440 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f833402c680 tx=0x7f833402c610 comp rx=0 tx=0).stop 2024-01-31T05:38:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.798+0000 7f8349da3640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f83300a1570 0x7f83300a3a30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f833c0ca790 tx=0x7f833c08f5b0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.798+0000 7f834e5c5740 1 -- 172.21.15.45:0/3102041972 shutdown_connections 2024-01-31T05:38:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.798+0000 7f834e5c5740 1 --2- 172.21.15.45:0/3102041972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 0x56320377d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.798+0000 7f834e5c5740 1 --2- 172.21.15.45:0/3102041972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x56320377fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.798+0000 7f834e5c5740 1 --2- 172.21.15.45:0/3102041972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 0x563203774440 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.798+0000 7f834e5c5740 1 -- 172.21.15.45:0/3102041972 >> 172.21.15.45:0/3102041972 conn(0x56320376e980 msgr2=0x56320376ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.798+0000 7f834e5c5740 1 -- 172.21.15.45:0/3102041972 shutdown_connections 2024-01-31T05:38:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.798+0000 7f834e5c5740 1 -- 172.21.15.45:0/3102041972 wait complete. 2024-01-31T05:38:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.798+0000 7f834e5c5740 1 -- start start 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f834e5c5740 1 Processor -- start 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f834e5c5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774060 0x56320377a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f834e5c5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 0x56320377ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f834e5c5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x56320377b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f834e5c5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632036951b0 con 0x563203774060 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f834e5c5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563203694e50 con 0x563203774980 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f834e5c5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56320365a090 con 0x56320377d750 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f834e5c5740 10 client.0 _ll_register_callbacks cb 0x56320367fa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f8348da1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774060 0x56320377a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f83495a2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 0x56320377ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f83495a2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 0x56320377ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:58764/0 (socket says 172.21.15.45:58764) 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f83495a2640 1 -- 172.21.15.45:0/3496036026 learned_addr learned my addr 172.21.15.45:0/3496036026 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.799+0000 7f8348da1640 1 --2- 172.21.15.45:0/3496036026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x56320377b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.800+0000 7f83495a2640 1 -- 172.21.15.45:0/3496036026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 msgr2=0x56320377b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.800+0000 7f83495a2640 1 --2- 172.21.15.45:0/3496036026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x56320377b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.800+0000 7f83495a2640 1 -- 172.21.15.45:0/3496036026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774060 msgr2=0x56320377a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.800+0000 7f83495a2640 1 --2- 172.21.15.45:0/3496036026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774060 0x56320377a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.800+0000 7f83495a2640 1 -- 172.21.15.45:0/3496036026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834410d320 con 0x563203774980 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.800+0000 7f83495a2640 1 --2- 172.21.15.45:0/3496036026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 0x56320377ab80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83400128d0 tx=0x7f8340012d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79435ef807142051 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.800+0000 7f832cff9640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.800+0000 7f832cff9640 1 -- 172.21.15.45:0/3496036026 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340017070 con 0x563203774980 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.800+0000 7f832cff9640 1 -- 172.21.15.45:0/3496036026 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f834000f920 con 0x563203774980 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.800+0000 7f834e5c5740 1 -- 172.21.15.45:0/3496036026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632036d5010 con 0x563203774980 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.801+0000 7f832cff9640 1 -- 172.21.15.45:0/3496036026 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83400122e0 con 0x563203774980 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.801+0000 7f834e5c5740 10 client.26198 fetch_fsmap learned FSMap version 33 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.801+0000 7f834e5c5740 1 -- 172.21.15.45:0/3496036026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5632036da6a0 con 0x563203774980 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.801+0000 7f832cff9640 1 -- 172.21.15.45:0/3496036026 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632036d5010 con 0x563203774980 2024-01-31T05:38:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.801+0000 7f832cff9640 1 -- 172.21.15.45:0/3496036026 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8340022030 con 0x563203774980 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.801+0000 7f834e5c5740 10 client.26198 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.801+0000 7f834e5c5740 10 client.26198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.801+0000 7f834e5c5740 20 client.26198 populate_metadata read hostname 'smithi045' 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.801+0000 7f834e5c5740 1 --2- 172.21.15.45:0/3496036026 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56320378d4c0 0x56320378f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.801+0000 7f834e5c5740 4 client.26198 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.801+0000 7f834e5c5740 1 -- 172.21.15.45:0/3496036026 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632036d4d30 con 0x56320378d4c0 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.801+0000 7f8348da1640 1 --2- 172.21.15.45:0/3496036026 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56320378d4c0 0x56320378f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f832cff9640 10 client.26198 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f8348da1640 1 --2- 172.21.15.45:0/3496036026 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56320378d4c0 0x56320378f8e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace3837a660f347 server_cookie=5f7abe1768c261d4 in_seq=0 out_seq=0 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f832cff9640 1 -- 172.21.15.45:0/3496036026 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632036d4d30 con 0x56320378d4c0 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f832cff9640 10 client.26198 handle_command_reply: tid=0 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f834e5c5740 1 client.26198 shutdown 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f834e5c5740 1 -- 172.21.15.45:0/3496036026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 msgr2=0x56320377ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f834e5c5740 1 --2- 172.21.15.45:0/3496036026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 0x56320377ab80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83400128d0 tx=0x7f8340012d90 comp rx=0 tx=0).stop 2024-01-31T05:38:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f834e5c5740 1 -- 172.21.15.45:0/3496036026 shutdown_connections 2024-01-31T05:38:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f834e5c5740 1 --2- 172.21.15.45:0/3496036026 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56320378d4c0 0x56320378f8e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f832cff9640 0 client.26198 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f834e5c5740 1 --2- 172.21.15.45:0/3496036026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x56320377b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f834e5c5740 1 --2- 172.21.15.45:0/3496036026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 0x56320377ab80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f834e5c5740 1 --2- 172.21.15.45:0/3496036026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774060 0x56320377a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.802+0000 7f834e5c5740 1 -- 172.21.15.45:0/3496036026 >> 172.21.15.45:0/3496036026 conn(0x56320376e980 msgr2=0x5632037706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.803+0000 7f834e5c5740 1 -- 172.21.15.45:0/3496036026 shutdown_connections 2024-01-31T05:38:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.803+0000 7f834e5c5740 1 -- 172.21.15.45:0/3496036026 wait complete. 2024-01-31T05:38:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.805+0000 7f834e5c5740 20 client.26198 trim_cache size 0 max 16384 2024-01-31T05:38:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.820+0000 7f834e5c5740 1 Processor -- start 2024-01-31T05:38:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.820+0000 7f834e5c5740 1 -- start start 2024-01-31T05:38:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.820+0000 7f834e5c5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 0x563203778f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.820+0000 7f834e5c5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 0x56320377aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.820+0000 7f834e5c5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x5632037843e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.820+0000 7f834e5c5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56320377ff00 con 0x563203774980 2024-01-31T05:38:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f834e5c5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563203779450 con 0x563203774060 2024-01-31T05:38:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f834e5c5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563203794b30 con 0x56320377d750 2024-01-31T05:38:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f83495a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 0x56320377aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f83495a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 0x56320377aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53434/0 (socket says 172.21.15.45:53434) 2024-01-31T05:38:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f83495a2640 1 -- 172.21.15.45:0/2834598789 learned_addr learned my addr 172.21.15.45:0/2834598789 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f8348da1640 1 --2- 172.21.15.45:0/2834598789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 0x563203778f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f8348da1640 1 --2- 172.21.15.45:0/2834598789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x5632037843e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f83495a2640 1 -- 172.21.15.45:0/2834598789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 msgr2=0x5632037843e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f83495a2640 1 --2- 172.21.15.45:0/2834598789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x5632037843e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f83495a2640 1 -- 172.21.15.45:0/2834598789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 msgr2=0x563203778f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f83495a2640 1 --2- 172.21.15.45:0/2834598789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 0x563203778f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f83495a2640 1 -- 172.21.15.45:0/2834598789 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563203677030 con 0x563203774980 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f8348da1640 1 --2- 172.21.15.45:0/2834598789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 0x563203778f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f83495a2640 1 --2- 172.21.15.45:0/2834598789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 0x56320377aee0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8340010db0 tx=0x7f83400097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de95cad47e1ad3bb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f832effd640 1 -- 172.21.15.45:0/2834598789 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340017040 con 0x563203774980 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f832effd640 1 -- 172.21.15.45:0/2834598789 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83400122e0 con 0x563203774980 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.821+0000 7f832effd640 1 -- 172.21.15.45:0/2834598789 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83400052a0 con 0x563203774980 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.822+0000 7f834e5c5740 1 -- 172.21.15.45:0/2834598789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 msgr2=0x56320377aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.822+0000 7f834e5c5740 1 --2- 172.21.15.45:0/2834598789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 0x56320377aee0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8340010db0 tx=0x7f83400097f0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.822+0000 7f834e5c5740 1 -- 172.21.15.45:0/2834598789 shutdown_connections 2024-01-31T05:38:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.822+0000 7f834e5c5740 1 --2- 172.21.15.45:0/2834598789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774980 0x56320377aee0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.822+0000 7f834e5c5740 1 --2- 172.21.15.45:0/2834598789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x5632037843e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.822+0000 7f834e5c5740 1 --2- 172.21.15.45:0/2834598789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774060 0x563203778f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.822+0000 7f834e5c5740 1 -- 172.21.15.45:0/2834598789 >> 172.21.15.45:0/2834598789 conn(0x56320376e980 msgr2=0x56320376d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.822+0000 7f834e5c5740 1 -- 172.21.15.45:0/2834598789 shutdown_connections 2024-01-31T05:38:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.822+0000 7f834e5c5740 1 -- 172.21.15.45:0/2834598789 wait complete. 2024-01-31T05:38:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.823+0000 7f834e5c5740 1 -- start start 2024-01-31T05:38:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.823+0000 7f834e5c5740 1 Processor -- start 2024-01-31T05:38:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.823+0000 7f834e5c5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774060 0x5632037832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.823+0000 7f834e5c5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 0x563203783830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.823+0000 7f834e5c5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x5632037e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.823+0000 7f834e5c5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563203784130 con 0x563203774060 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.823+0000 7f834e5c5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632037e3f50 con 0x563203774980 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.823+0000 7f834e5c5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632037e4090 con 0x56320377d750 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.823+0000 7f834e5c5740 10 client.0 _ll_register_callbacks cb 0x7f8334009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f83495a2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 0x563203783830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f83495a2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 0x563203783830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:58780/0 (socket says 172.21.15.45:58780) 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f83495a2640 1 -- 172.21.15.45:0/405299189 learned_addr learned my addr 172.21.15.45:0/405299189 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f8348da1640 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774060 0x5632037832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f8348da1640 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x5632037e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f83495a2640 1 -- 172.21.15.45:0/405299189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 msgr2=0x5632037e3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f83495a2640 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x5632037e3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f83495a2640 1 -- 172.21.15.45:0/405299189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774060 msgr2=0x5632037832f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f83495a2640 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774060 0x5632037832f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f83495a2640 1 -- 172.21.15.45:0/405299189 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632036d3e40 con 0x563203774980 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f83495a2640 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 0x563203783830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f83400098e0 tx=0x7f8340012830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef9960bd3eda7589 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f832cff9640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f832cff9640 1 -- 172.21.15.45:0/405299189 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340017040 con 0x563203774980 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f832cff9640 1 -- 172.21.15.45:0/405299189 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f83400050f0 con 0x563203774980 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.824+0000 7f834e5c5740 1 -- 172.21.15.45:0/405299189 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8334004030 con 0x563203774980 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.825+0000 7f832cff9640 1 -- 172.21.15.45:0/405299189 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83400052d0 con 0x563203774980 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.825+0000 7f834e5c5740 10 client.26204 fetch_fsmap learned FSMap version 33 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.825+0000 7f834e5c5740 1 -- 172.21.15.45:0/405299189 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5632036da6a0 con 0x563203774980 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.825+0000 7f832cff9640 1 -- 172.21.15.45:0/405299189 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8334004030 con 0x563203774980 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.825+0000 7f832cff9640 1 -- 172.21.15.45:0/405299189 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8340010430 con 0x563203774980 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.825+0000 7f834e5c5740 10 client.26204 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.825+0000 7f834e5c5740 10 client.26204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.825+0000 7f834e5c5740 20 client.26204 populate_metadata read hostname 'smithi045' 2024-01-31T05:38:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.825+0000 7f834e5c5740 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56320378d4c0 0x5632037e7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:38:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.825+0000 7f834e5c5740 4 client.26204 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:38:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.825+0000 7f834e5c5740 1 -- 172.21.15.45:0/405299189 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5632033db600 con 0x56320378d4c0 2024-01-31T05:38:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.825+0000 7f8348da1640 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56320378d4c0 0x5632037e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:38:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.826+0000 7f832cff9640 10 client.26204 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.826+0000 7f8348da1640 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56320378d4c0 0x5632037e7540 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c5b92210e3f7f47 server_cookie=15f88e4f8aac29ab in_seq=0 out_seq=0 2024-01-31T05:38:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.891+0000 7f832cff9640 1 -- 172.21.15.45:0/405299189 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5632033db600 con 0x56320378d4c0 2024-01-31T05:38:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.891+0000 7f832cff9640 10 client.26204 handle_command_reply: tid=0 2024-01-31T05:38:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.891+0000 7f834e5c5740 1 client.26204 shutdown 2024-01-31T05:38:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.891+0000 7f834e5c5740 1 -- 172.21.15.45:0/405299189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 msgr2=0x563203783830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.891+0000 7f834e5c5740 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 0x563203783830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f83400098e0 tx=0x7f8340012830 comp rx=0 tx=0).stop 2024-01-31T05:38:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.891+0000 7f834e5c5740 1 -- 172.21.15.45:0/405299189 shutdown_connections 2024-01-31T05:38:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.891+0000 7f834e5c5740 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x56320378d4c0 0x5632037e7540 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.891+0000 7f834e5c5740 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56320377d750 0x5632037e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.891+0000 7f834e5c5740 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563203774980 0x563203783830 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.891+0000 7f834e5c5740 1 --2- 172.21.15.45:0/405299189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563203774060 0x5632037832f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.891+0000 7f832cff9640 0 client.26204 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:38:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.891+0000 7f834e5c5740 1 -- 172.21.15.45:0/405299189 >> 172.21.15.45:0/405299189 conn(0x56320376e980 msgr2=0x56320376da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834e5c5740 1 -- 172.21.15.45:0/405299189 shutdown_connections 2024-01-31T05:38:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834e5c5740 1 -- 172.21.15.45:0/405299189 wait complete. 2024-01-31T05:38:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834e5c5740 20 client.26204 trim_cache size 0 max 16384 2024-01-31T05:38:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:38:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:38:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:38:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:38:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834b806640 1 -- 172.21.15.45:0/1225340863 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f83300a1570 msgr2=0x7f83300a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834b806640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f83300a1570 0x7f83300a3a30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f833c0ca790 tx=0x7f833c08f5b0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834b806640 1 -- 172.21.15.45:0/1225340863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8344104c20 msgr2=0x7f83441a3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:38:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834b806640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8344104c20 0x7f83441a3000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f833c00e970 tx=0x7f833c00ee30 comp rx=0 tx=0).stop 2024-01-31T05:38:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834b806640 1 -- 172.21.15.45:0/1225340863 shutdown_connections 2024-01-31T05:38:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834b806640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f83300a1570 0x7f83300a3a30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834b806640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8344100cc0 0x7f83441984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834b806640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8344104c20 0x7f83441a3000 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834b806640 1 --2- 172.21.15.45:0/1225340863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8344100300 0x7f8344197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:38:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834b806640 1 -- 172.21.15.45:0/1225340863 >> 172.21.15.45:0/1225340863 conn(0x7f83440761b0 msgr2=0x7f8344102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:38:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834b806640 1 -- 172.21.15.45:0/1225340863 shutdown_connections 2024-01-31T05:38:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:38:54.892+0000 7f834b806640 1 -- 172.21.15.45:0/1225340863 wait complete. 2024-01-31T05:38:55.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:54 smithi186 ceph-mon[35659]: pgmap v515: 97 pgs: 97 active+clean; 1022 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 568 op/s 2024-01-31T05:38:55.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:54 smithi045 ceph-mon[31585]: pgmap v515: 97 pgs: 97 active+clean; 1022 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 568 op/s 2024-01-31T05:38:55.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:54 smithi156 ceph-mon[35720]: pgmap v515: 97 pgs: 97 active+clean; 1022 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 568 op/s 2024-01-31T05:38:55.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 75198 23.28 MB/sec execute 54 sec latency 210.634 ms 2024-01-31T05:38:56.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 75728 23.47 MB/sec execute 55 sec latency 162.590 ms 2024-01-31T05:38:57.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:56 smithi186 ceph-mon[35659]: pgmap v516: 97 pgs: 97 active+clean; 1022 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.3 MiB/s wr, 378 op/s 2024-01-31T05:38:57.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:56 smithi045 ceph-mon[31585]: pgmap v516: 97 pgs: 97 active+clean; 1022 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.3 MiB/s wr, 378 op/s 2024-01-31T05:38:57.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:56 smithi156 ceph-mon[35720]: pgmap v516: 97 pgs: 97 active+clean; 1022 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.3 MiB/s wr, 378 op/s 2024-01-31T05:38:57.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 76102 23.31 MB/sec execute 56 sec latency 327.408 ms 2024-01-31T05:38:58.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 76452 23.13 MB/sec execute 57 sec latency 204.402 ms 2024-01-31T05:38:59.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:38:58 smithi186 ceph-mon[35659]: pgmap v517: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 576 op/s 2024-01-31T05:38:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:38:58 smithi045 ceph-mon[31585]: pgmap v517: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 576 op/s 2024-01-31T05:38:59.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:38:58 smithi156 ceph-mon[35720]: pgmap v517: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 576 op/s 2024-01-31T05:38:59.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 77082 23.40 MB/sec execute 58 sec latency 202.875 ms 2024-01-31T05:39:00.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 77343 23.41 MB/sec execute 59 sec latency 409.425 ms 2024-01-31T05:39:01.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:00 smithi186 ceph-mon[35659]: pgmap v518: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 380 op/s 2024-01-31T05:39:01.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:00 smithi045 ceph-mon[31585]: pgmap v518: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 380 op/s 2024-01-31T05:39:01.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:00 smithi156 ceph-mon[35720]: pgmap v518: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 380 op/s 2024-01-31T05:39:01.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 77343 23.02 MB/sec execute 60 sec latency 1409.535 ms 2024-01-31T05:39:02.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 77524 22.79 MB/sec execute 61 sec latency 1857.256 ms 2024-01-31T05:39:03.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:02 smithi186 ceph-mon[35659]: pgmap v519: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 380 op/s 2024-01-31T05:39:03.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:02 smithi045 ceph-mon[31585]: pgmap v519: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 380 op/s 2024-01-31T05:39:03.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:02 smithi156 ceph-mon[35720]: pgmap v519: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 380 op/s 2024-01-31T05:39:03.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 77825 22.65 MB/sec execute 62 sec latency 382.651 ms 2024-01-31T05:39:04.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 78087 22.49 MB/sec execute 63 sec latency 156.847 ms 2024-01-31T05:39:05.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:04 smithi186 ceph-mon[35659]: pgmap v520: 97 pgs: 97 active+clean; 949 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 554 op/s 2024-01-31T05:39:05.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:04 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:39:05.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:04 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:39:05.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:04 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:39:05.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:04 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:39:05.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:04 smithi045 ceph-mon[31585]: pgmap v520: 97 pgs: 97 active+clean; 949 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 554 op/s 2024-01-31T05:39:05.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:04 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:39:05.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:04 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:39:05.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:04 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:39:05.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:04 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:39:05.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:04 smithi156 ceph-mon[35720]: pgmap v520: 97 pgs: 97 active+clean; 949 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 554 op/s 2024-01-31T05:39:05.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:04 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:39:05.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:04 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:39:05.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:04 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:39:05.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:04 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:39:05.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 78441 22.39 MB/sec execute 64 sec latency 151.286 ms 2024-01-31T05:39:06.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 78951 22.41 MB/sec execute 65 sec latency 130.117 ms 2024-01-31T05:39:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:06 smithi186 ceph-mon[35659]: pgmap v521: 97 pgs: 97 active+clean; 949 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.5 MiB/s wr, 371 op/s 2024-01-31T05:39:07.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:06 smithi186 ceph-mon[35659]: osdmap e116: 12 total, 12 up, 12 in 2024-01-31T05:39:07.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:06 smithi045 ceph-mon[31585]: pgmap v521: 97 pgs: 97 active+clean; 949 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.5 MiB/s wr, 371 op/s 2024-01-31T05:39:07.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:06 smithi045 ceph-mon[31585]: osdmap e116: 12 total, 12 up, 12 in 2024-01-31T05:39:07.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:06 smithi156 ceph-mon[35720]: pgmap v521: 97 pgs: 97 active+clean; 949 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.5 MiB/s wr, 371 op/s 2024-01-31T05:39:07.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:06 smithi156 ceph-mon[35720]: osdmap e116: 12 total, 12 up, 12 in 2024-01-31T05:39:07.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 79310 22.39 MB/sec execute 66 sec latency 262.179 ms 2024-01-31T05:39:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.014+0000 7f4a78d24640 1 -- 172.21.15.45:0/2829079473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a74104b90 msgr2=0x7f4a74104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.014+0000 7f4a78d24640 1 --2- 172.21.15.45:0/2829079473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a74104b90 0x7f4a74104f70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009900 tx=0x7f4a6402bc30 comp rx=0 tx=0).stop 2024-01-31T05:39:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.014+0000 7f4a78d24640 1 -- 172.21.15.45:0/2829079473 shutdown_connections 2024-01-31T05:39:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.014+0000 7f4a78d24640 1 --2- 172.21.15.45:0/2829079473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a74100800 0x7f4a7410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.014+0000 7f4a78d24640 1 --2- 172.21.15.45:0/2829079473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a740ffe40 0x7f4a741002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.014+0000 7f4a78d24640 1 --2- 172.21.15.45:0/2829079473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a74104b90 0x7f4a74104f70 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.014+0000 7f4a78d24640 1 -- 172.21.15.45:0/2829079473 >> 172.21.15.45:0/2829079473 conn(0x7f4a74076a40 msgr2=0x7f4a74076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.015+0000 7f4a78d24640 1 -- 172.21.15.45:0/2829079473 shutdown_connections 2024-01-31T05:39:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.015+0000 7f4a78d24640 1 -- 172.21.15.45:0/2829079473 wait complete. 2024-01-31T05:39:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.015+0000 7f4a78d24640 1 Processor -- start 2024-01-31T05:39:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.015+0000 7f4a78d24640 1 -- start start 2024-01-31T05:39:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.015+0000 7f4a78d24640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a740ffe40 0x7f4a741972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.015+0000 7f4a78d24640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a74100800 0x7f4a741977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.015+0000 7f4a78d24640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a74104b90 0x7f4a7419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.015+0000 7f4a78d24640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7419c160 con 0x7f4a740ffe40 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.015+0000 7f4a78d24640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7419c2d0 con 0x7f4a74104b90 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.015+0000 7f4a78d24640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7419c490 con 0x7f4a74100800 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a740ffe40 0x7f4a741972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a72d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a74104b90 0x7f4a7419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a71d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a74100800 0x7f4a741977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a72d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a74104b90 0x7f4a7419bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:43424/0 (socket says 172.21.15.45:43424) 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a72d9d640 1 -- 172.21.15.45:0/2701914592 learned_addr learned my addr 172.21.15.45:0/2701914592 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a7259c640 1 -- 172.21.15.45:0/2701914592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a74100800 msgr2=0x7f4a741977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a7259c640 1 --2- 172.21.15.45:0/2701914592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a74100800 0x7f4a741977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a7259c640 1 -- 172.21.15.45:0/2701914592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a74104b90 msgr2=0x7f4a7419bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a7259c640 1 --2- 172.21.15.45:0/2701914592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a74104b90 0x7f4a7419bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a7259c640 1 -- 172.21.15.45:0/2701914592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a64009560 con 0x7f4a740ffe40 2024-01-31T05:39:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a7259c640 1 --2- 172.21.15.45:0/2701914592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a740ffe40 0x7f4a741972a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4a64036040 tx=0x7f4a6402c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a6b7fe640 1 -- 172.21.15.45:0/2701914592 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64004250 con 0x7f4a740ffe40 2024-01-31T05:39:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a6b7fe640 1 -- 172.21.15.45:0/2701914592 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a640043b0 con 0x7f4a740ffe40 2024-01-31T05:39:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a78d24640 1 -- 172.21.15.45:0/2701914592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7419c710 con 0x7f4a740ffe40 2024-01-31T05:39:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.016+0000 7f4a6b7fe640 1 -- 172.21.15.45:0/2701914592 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64003bc0 con 0x7f4a740ffe40 2024-01-31T05:39:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.017+0000 7f4a78d24640 1 -- 172.21.15.45:0/2701914592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a741a3ff0 con 0x7f4a740ffe40 2024-01-31T05:39:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.019+0000 7f4a6b7fe640 1 -- 172.21.15.45:0/2701914592 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f4a64003d20 con 0x7f4a740ffe40 2024-01-31T05:39:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.019+0000 7f4a6b7fe640 1 --2- 172.21.15.45:0/2701914592 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4a600a1570 0x7f4a600a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.019+0000 7f4a71d9b640 1 --2- 172.21.15.45:0/2701914592 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4a600a1570 0x7f4a600a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.019+0000 7f4a6b7fe640 1 -- 172.21.15.45:0/2701914592 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f4a6402de40 con 0x7f4a740ffe40 2024-01-31T05:39:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.019+0000 7f4a7bbc2740 1 Processor -- start 2024-01-31T05:39:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a7bbc2740 1 -- start start 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a71d9b640 1 --2- 172.21.15.45:0/2701914592 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4a600a1570 0x7f4a600a3a30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4a740770b0 tx=0x7f4a5c005f90 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a7bbc2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5eefb00 0x559da5eefee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a7bbc2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5ef0420 0x559da5efd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a7bbc2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 0x559da5efff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a7bbc2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da5e5f040 con 0x559da5efdb60 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a7bbc2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da5e5c730 con 0x559da5ef0420 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a7bbc2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da5e58020 con 0x559da5eefb00 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a72d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 0x559da5efff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5eefb00 0x559da5eefee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a72d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 0x559da5efff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46874/0 (socket says 172.21.15.45:46874) 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a72d9d640 1 -- 172.21.15.45:0/3723004232 learned_addr learned my addr 172.21.15.45:0/3723004232 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a71d9b640 1 --2- 172.21.15.45:0/3723004232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5ef0420 0x559da5efd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a72d9d640 1 -- 172.21.15.45:0/3723004232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5eefb00 msgr2=0x559da5eefee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a72d9d640 1 --2- 172.21.15.45:0/3723004232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5eefb00 0x559da5eefee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a72d9d640 1 -- 172.21.15.45:0/3723004232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5ef0420 msgr2=0x559da5efd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a72d9d640 1 --2- 172.21.15.45:0/3723004232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5ef0420 0x559da5efd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a72d9d640 1 -- 172.21.15.45:0/3723004232 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559da5e4fe40 con 0x559da5efdb60 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a7259c640 1 --2- 172.21.15.45:0/3723004232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5eefb00 0x559da5eefee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.020+0000 7f4a71d9b640 1 --2- 172.21.15.45:0/3723004232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5ef0420 0x559da5efd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.021+0000 7f4a72d9d640 1 --2- 172.21.15.45:0/3723004232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 0x559da5efff50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0115a0 tx=0x7f4a6c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cbb5c076e567b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.021+0000 7f4a68ff9640 1 -- 172.21.15.45:0/3723004232 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c004b20 con 0x559da5efdb60 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.021+0000 7f4a68ff9640 1 -- 172.21.15.45:0/3723004232 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a6c004c80 con 0x559da5efdb60 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.021+0000 7f4a68ff9640 1 -- 172.21.15.45:0/3723004232 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c005460 con 0x559da5efdb60 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.022+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3723004232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 msgr2=0x559da5efff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.022+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/3723004232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 0x559da5efff50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0115a0 tx=0x7f4a6c011a60 comp rx=0 tx=0).stop 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.022+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3723004232 shutdown_connections 2024-01-31T05:39:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.022+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/3723004232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 0x559da5efff50 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.022+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/3723004232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5ef0420 0x559da5efd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.022+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/3723004232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5eefb00 0x559da5eefee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.022+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3723004232 >> 172.21.15.45:0/3723004232 conn(0x559da5eea980 msgr2=0x559da5eead60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.022+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3723004232 shutdown_connections 2024-01-31T05:39:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.022+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3723004232 wait complete. 2024-01-31T05:39:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.022+0000 7f4a7bbc2740 1 -- start start 2024-01-31T05:39:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.022+0000 7f4a7bbc2740 1 Processor -- start 2024-01-31T05:39:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a7bbc2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 0x559da5efd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a7bbc2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5ef0420 0x559da5efa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a7bbc2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5efdb60 0x559da5efafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a7bbc2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da5e111b0 con 0x559da5eefb00 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a7bbc2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da5e10e50 con 0x559da5efdb60 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a7bbc2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da5dd6090 con 0x559da5ef0420 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a7bbc2740 10 client.0 _ll_register_callbacks cb 0x559da5dfba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 0x559da5efd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 0x559da5efd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46890/0 (socket says 172.21.15.45:46890) 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a7259c640 1 -- 172.21.15.45:0/1975153255 learned_addr learned my addr 172.21.15.45:0/1975153255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a72d9d640 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5efdb60 0x559da5efafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a71d9b640 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5ef0420 0x559da5efa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a72d9d640 1 -- 172.21.15.45:0/1975153255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5ef0420 msgr2=0x559da5efa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a72d9d640 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5ef0420 0x559da5efa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a72d9d640 1 -- 172.21.15.45:0/1975153255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 msgr2=0x559da5efd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a72d9d640 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 0x559da5efd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a72d9d640 1 -- 172.21.15.45:0/1975153255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6c009d40 con 0x559da5efdb60 2024-01-31T05:39:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.023+0000 7f4a7259c640 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 0x559da5efd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a72d9d640 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5efdb60 0x559da5efafd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c010010 tx=0x7f4a6c00fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad5ad7a798b2845 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a59ffb640 1 -- 172.21.15.45:0/1975153255 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c004df0 con 0x559da5efdb60 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a59ffb640 1 -- 172.21.15.45:0/1975153255 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f4a6c0051c0 con 0x559da5efdb60 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/1975153255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559da5e51010 con 0x559da5efdb60 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a59ffb640 1 -- 172.21.15.45:0/1975153255 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a6c0053c0 con 0x559da5efdb60 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a7bbc2740 10 client.26216 fetch_fsmap learned FSMap version 33 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/1975153255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559da5e566a0 con 0x559da5efdb60 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a59ffb640 1 -- 172.21.15.45:0/1975153255 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559da5e51010 con 0x559da5efdb60 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a59ffb640 1 -- 172.21.15.45:0/1975153255 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4a6c022890 con 0x559da5efdb60 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a7bbc2740 10 client.26216 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a7bbc2740 10 client.26216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a7bbc2740 20 client.26216 populate_metadata read hostname 'smithi045' 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559da5f0d8b0 0x559da5f0fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a7bbc2740 4 client.26216 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.024+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/1975153255 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559da5e50d30 con 0x559da5f0d8b0 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.025+0000 7f4a7259c640 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559da5f0d8b0 0x559da5f0fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.025+0000 7f4a59ffb640 10 client.26216 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.025+0000 7f4a7259c640 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559da5f0d8b0 0x559da5f0fca0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f44efad90951d17 server_cookie=1ef808dfe03d922b in_seq=0 out_seq=0 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a59ffb640 1 -- 172.21.15.45:0/1975153255 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559da5e50d30 con 0x559da5f0d8b0 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a59ffb640 10 client.26216 handle_command_reply: tid=0 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a7bbc2740 1 client.26216 shutdown 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/1975153255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5efdb60 msgr2=0x559da5efafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5efdb60 0x559da5efafd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c010010 tx=0x7f4a6c00fc00 comp rx=0 tx=0).stop 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/1975153255 shutdown_connections 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559da5f0d8b0 0x559da5f0fca0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5efdb60 0x559da5efafd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5ef0420 0x559da5efa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a59ffb640 0 client.26216 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/1975153255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 0x559da5efd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/1975153255 >> 172.21.15.45:0/1975153255 conn(0x559da5eea980 msgr2=0x559da5eec6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/1975153255 shutdown_connections 2024-01-31T05:39:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/1975153255 wait complete. 2024-01-31T05:39:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.026+0000 7f4a7bbc2740 20 client.26216 trim_cache size 0 max 16384 2024-01-31T05:39:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.042+0000 7f4a7bbc2740 1 Processor -- start 2024-01-31T05:39:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.042+0000 7f4a7bbc2740 1 -- start start 2024-01-31T05:39:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.042+0000 7f4a7bbc2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5eefb00 0x559da5f04920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.042+0000 7f4a7bbc2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5ef0420 0x559da5f04e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.042+0000 7f4a7bbc2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 0x559da5ef9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.042+0000 7f4a7bbc2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da5f0d6b0 con 0x559da5efdb60 2024-01-31T05:39:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.042+0000 7f4a7bbc2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da5efb870 con 0x559da5eefb00 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.042+0000 7f4a7bbc2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da5eef570 con 0x559da5ef0420 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5eefb00 0x559da5f04920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5eefb00 0x559da5f04920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:43450/0 (socket says 172.21.15.45:43450) 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a71d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5ef0420 0x559da5f04e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a7259c640 1 -- 172.21.15.45:0/2255650503 learned_addr learned my addr 172.21.15.45:0/2255650503 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a72d9d640 1 --2- 172.21.15.45:0/2255650503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 0x559da5ef9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a7259c640 1 -- 172.21.15.45:0/2255650503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5ef0420 msgr2=0x559da5f04e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a7259c640 1 --2- 172.21.15.45:0/2255650503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5ef0420 0x559da5f04e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a7259c640 1 -- 172.21.15.45:0/2255650503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 msgr2=0x559da5ef9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a7259c640 1 --2- 172.21.15.45:0/2255650503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 0x559da5ef9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a7259c640 1 -- 172.21.15.45:0/2255650503 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559da5df3030 con 0x559da5eefb00 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a72d9d640 1 --2- 172.21.15.45:0/2255650503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 0x559da5ef9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a7259c640 1 --2- 172.21.15.45:0/2255650503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5eefb00 0x559da5f04920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4a640ec9c0 tx=0x7f4a64041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f8740f4538fb77 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a5bfff640 1 -- 172.21.15.45:0/2255650503 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a640bc050 con 0x559da5eefb00 2024-01-31T05:39:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a5bfff640 1 -- 172.21.15.45:0/2255650503 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a640c01a0 con 0x559da5eefb00 2024-01-31T05:39:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.043+0000 7f4a5bfff640 1 -- 172.21.15.45:0/2255650503 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a640bb540 con 0x559da5eefb00 2024-01-31T05:39:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.044+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/2255650503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5eefb00 msgr2=0x559da5f04920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.044+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/2255650503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5eefb00 0x559da5f04920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4a640ec9c0 tx=0x7f4a64041040 comp rx=0 tx=0).stop 2024-01-31T05:39:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.044+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/2255650503 shutdown_connections 2024-01-31T05:39:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.044+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/2255650503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5efdb60 0x559da5ef9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.044+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/2255650503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5ef0420 0x559da5f04e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.044+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/2255650503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5eefb00 0x559da5f04920 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.044+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/2255650503 >> 172.21.15.45:0/2255650503 conn(0x559da5eea980 msgr2=0x559da5ef5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.044+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/2255650503 shutdown_connections 2024-01-31T05:39:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.044+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/2255650503 wait complete. 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.044+0000 7f4a7bbc2740 1 -- start start 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7bbc2740 1 Processor -- start 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7bbc2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 0x559da5f04c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7bbc2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5ef0420 0x559da5f01040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7bbc2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5efdb60 0x559da5f01580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7bbc2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da5f01c80 con 0x559da5eefb00 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7bbc2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da5f01df0 con 0x559da5ef0420 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7bbc2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da5f63da0 con 0x559da5efdb60 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 0x559da5f04c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7bbc2740 10 client.0 _ll_register_callbacks cb 0x7f4a5c0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 0x559da5f04c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46922/0 (socket says 172.21.15.45:46922) 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7259c640 1 -- 172.21.15.45:0/3269867149 learned_addr learned my addr 172.21.15.45:0/3269867149 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a71d9b640 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5ef0420 0x559da5f01040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a72d9d640 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5efdb60 0x559da5f01580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7259c640 1 -- 172.21.15.45:0/3269867149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5efdb60 msgr2=0x559da5f01580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7259c640 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5efdb60 0x559da5f01580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7259c640 1 -- 172.21.15.45:0/3269867149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5ef0420 msgr2=0x559da5f01040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7259c640 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5ef0420 0x559da5f01040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7259c640 1 -- 172.21.15.45:0/3269867149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559da5e566a0 con 0x559da5eefb00 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a72d9d640 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5efdb60 0x559da5f01580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a71d9b640 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5ef0420 0x559da5f01040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a7259c640 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 0x559da5f04c70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4a640bbd50 tx=0x7f4a640058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c3fb97b2663f77 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.045+0000 7f4a59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a59ffb640 1 -- 172.21.15.45:0/3269867149 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a640bc050 con 0x559da5eefb00 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a59ffb640 1 -- 172.21.15.45:0/3269867149 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f4a640c20b0 con 0x559da5eefb00 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3269867149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a6c004c80 con 0x559da5eefb00 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a59ffb640 1 -- 172.21.15.45:0/3269867149 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a640c2290 con 0x559da5eefb00 2024-01-31T05:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a7bbc2740 10 client.16722 fetch_fsmap learned FSMap version 33 2024-01-31T05:39:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3269867149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f4a7419c710 con 0x559da5eefb00 2024-01-31T05:39:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a59ffb640 1 -- 172.21.15.45:0/3269867149 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a6c004c80 con 0x559da5eefb00 2024-01-31T05:39:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a59ffb640 1 -- 172.21.15.45:0/3269867149 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4a6403c070 con 0x559da5eefb00 2024-01-31T05:39:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a7bbc2740 10 client.16722 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:39:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a7bbc2740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:39:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a7bbc2740 20 client.16722 populate_metadata read hostname 'smithi045' 2024-01-31T05:39:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559da5f65100 0x559da5f674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a7bbc2740 4 client.16722 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:39:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3269867149 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559da5b57600 con 0x559da5f65100 2024-01-31T05:39:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.046+0000 7f4a71d9b640 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559da5f65100 0x559da5f674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.047+0000 7f4a59ffb640 10 client.16722 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.047+0000 7f4a71d9b640 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559da5f65100 0x559da5f674f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2db54bf1fc457c23 server_cookie=4618536135b40b89 in_seq=0 out_seq=0 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a59ffb640 1 -- 172.21.15.45:0/3269867149 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559da5b57600 con 0x559da5f65100 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a59ffb640 10 client.16722 handle_command_reply: tid=0 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a7bbc2740 1 client.16722 shutdown 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3269867149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 msgr2=0x559da5f04c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 0x559da5f04c70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4a640bbd50 tx=0x7f4a640058b0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3269867149 shutdown_connections 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x559da5f65100 0x559da5f674f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da5efdb60 0x559da5f01580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559da5ef0420 0x559da5f01040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a59ffb640 0 client.16722 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a7bbc2740 1 --2- 172.21.15.45:0/3269867149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da5eefb00 0x559da5f04c70 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3269867149 >> 172.21.15.45:0/3269867149 conn(0x559da5eea980 msgr2=0x559da5ef91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3269867149 shutdown_connections 2024-01-31T05:39:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a7bbc2740 1 -- 172.21.15.45:0/3269867149 wait complete. 2024-01-31T05:39:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.100+0000 7f4a7bbc2740 20 client.16722 trim_cache size 0 max 16384 2024-01-31T05:39:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:39:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:39:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:39:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.101+0000 7f4a78d24640 1 -- 172.21.15.45:0/2701914592 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4a600a1570 msgr2=0x7f4a600a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.101+0000 7f4a78d24640 1 --2- 172.21.15.45:0/2701914592 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4a600a1570 0x7f4a600a3a30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4a740770b0 tx=0x7f4a5c005f90 comp rx=0 tx=0).stop 2024-01-31T05:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.101+0000 7f4a78d24640 1 -- 172.21.15.45:0/2701914592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a740ffe40 msgr2=0x7f4a741972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.101+0000 7f4a78d24640 1 --2- 172.21.15.45:0/2701914592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a740ffe40 0x7f4a741972a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4a64036040 tx=0x7f4a6402c680 comp rx=0 tx=0).stop 2024-01-31T05:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.101+0000 7f4a78d24640 1 -- 172.21.15.45:0/2701914592 shutdown_connections 2024-01-31T05:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.101+0000 7f4a78d24640 1 --2- 172.21.15.45:0/2701914592 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4a600a1570 0x7f4a600a3a30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.101+0000 7f4a78d24640 1 --2- 172.21.15.45:0/2701914592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a74104b90 0x7f4a7419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.101+0000 7f4a78d24640 1 --2- 172.21.15.45:0/2701914592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a74100800 0x7f4a741977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.101+0000 7f4a78d24640 1 --2- 172.21.15.45:0/2701914592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a740ffe40 0x7f4a741972a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.101+0000 7f4a78d24640 1 -- 172.21.15.45:0/2701914592 >> 172.21.15.45:0/2701914592 conn(0x7f4a74076a40 msgr2=0x7f4a740685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.101+0000 7f4a78d24640 1 -- 172.21.15.45:0/2701914592 shutdown_connections 2024-01-31T05:39:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:08.101+0000 7f4a78d24640 1 -- 172.21.15.45:0/2701914592 wait complete. 2024-01-31T05:39:08.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 79807 22.58 MB/sec execute 67 sec latency 192.593 ms 2024-01-31T05:39:09.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:08 smithi186 ceph-mon[35659]: pgmap v523: 97 pgs: 97 active+clean; 964 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 611 op/s 2024-01-31T05:39:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:08 smithi045 ceph-mon[31585]: pgmap v523: 97 pgs: 97 active+clean; 964 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 611 op/s 2024-01-31T05:39:09.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:08 smithi156 ceph-mon[35720]: pgmap v523: 97 pgs: 97 active+clean; 964 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 611 op/s 2024-01-31T05:39:09.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 80171 22.50 MB/sec execute 68 sec latency 216.863 ms 2024-01-31T05:39:10.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 80682 22.49 MB/sec execute 69 sec latency 212.610 ms 2024-01-31T05:39:11.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:10 smithi186 ceph-mon[35659]: pgmap v524: 97 pgs: 97 active+clean; 964 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 611 op/s 2024-01-31T05:39:11.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:10 smithi045 ceph-mon[31585]: pgmap v524: 97 pgs: 97 active+clean; 964 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 611 op/s 2024-01-31T05:39:11.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:10 smithi156 ceph-mon[35720]: pgmap v524: 97 pgs: 97 active+clean; 964 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 611 op/s 2024-01-31T05:39:11.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 81160 22.60 MB/sec execute 70 sec latency 217.951 ms 2024-01-31T05:39:12.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 81531 22.62 MB/sec execute 71 sec latency 507.665 ms 2024-01-31T05:39:13.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:12 smithi186 ceph-mon[35659]: pgmap v525: 97 pgs: 97 active+clean; 964 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 611 op/s 2024-01-31T05:39:13.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:12 smithi186 ceph-mon[35659]: osdmap e117: 12 total, 12 up, 12 in 2024-01-31T05:39:13.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:12 smithi045 ceph-mon[31585]: pgmap v525: 97 pgs: 97 active+clean; 964 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 611 op/s 2024-01-31T05:39:13.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:12 smithi045 ceph-mon[31585]: osdmap e117: 12 total, 12 up, 12 in 2024-01-31T05:39:13.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:12 smithi156 ceph-mon[35720]: pgmap v525: 97 pgs: 97 active+clean; 964 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 611 op/s 2024-01-31T05:39:13.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:12 smithi156 ceph-mon[35720]: osdmap e117: 12 total, 12 up, 12 in 2024-01-31T05:39:13.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 81882 22.61 MB/sec execute 72 sec latency 224.926 ms 2024-01-31T05:39:14.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 82247 22.47 MB/sec execute 73 sec latency 53.753 ms 2024-01-31T05:39:15.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:14 smithi186 ceph-mon[35659]: pgmap v527: 97 pgs: 97 active+clean; 988 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 796 op/s 2024-01-31T05:39:15.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:14 smithi045 ceph-mon[31585]: pgmap v527: 97 pgs: 97 active+clean; 988 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 796 op/s 2024-01-31T05:39:15.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:14 smithi156 ceph-mon[35720]: pgmap v527: 97 pgs: 97 active+clean; 988 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 796 op/s 2024-01-31T05:39:15.649 INFO:tasks.workunit.client.0.smithi045.stdout: 10 82747 22.49 MB/sec execute 74 sec latency 125.712 ms 2024-01-31T05:39:16.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 83131 22.56 MB/sec execute 75 sec latency 269.381 ms 2024-01-31T05:39:17.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:16 smithi186 ceph-mon[35659]: pgmap v528: 97 pgs: 97 active+clean; 988 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.3 MiB/s wr, 645 op/s 2024-01-31T05:39:17.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:16 smithi045 ceph-mon[31585]: pgmap v528: 97 pgs: 97 active+clean; 988 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.3 MiB/s wr, 645 op/s 2024-01-31T05:39:17.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:16 smithi156 ceph-mon[35720]: pgmap v528: 97 pgs: 97 active+clean; 988 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.3 MiB/s wr, 645 op/s 2024-01-31T05:39:17.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 83426 22.43 MB/sec execute 76 sec latency 315.965 ms 2024-01-31T05:39:18.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 83875 22.43 MB/sec execute 77 sec latency 158.591 ms 2024-01-31T05:39:19.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:18 smithi186 ceph-mon[35659]: pgmap v529: 97 pgs: 97 active+clean; 1018 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.3 MiB/s wr, 480 op/s 2024-01-31T05:39:19.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:18 smithi045 ceph-mon[31585]: pgmap v529: 97 pgs: 97 active+clean; 1018 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.3 MiB/s wr, 480 op/s 2024-01-31T05:39:19.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:18 smithi156 ceph-mon[35720]: pgmap v529: 97 pgs: 97 active+clean; 1018 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.3 MiB/s wr, 480 op/s 2024-01-31T05:39:19.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 84403 22.43 MB/sec execute 78 sec latency 161.550 ms 2024-01-31T05:39:20.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 84984 22.54 MB/sec execute 79 sec latency 173.560 ms 2024-01-31T05:39:21.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:20 smithi186 ceph-mon[35659]: pgmap v530: 97 pgs: 97 active+clean; 1018 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.3 MiB/s wr, 480 op/s 2024-01-31T05:39:21.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:20 smithi045 ceph-mon[31585]: pgmap v530: 97 pgs: 97 active+clean; 1018 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.3 MiB/s wr, 480 op/s 2024-01-31T05:39:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.200+0000 7f36daa89640 1 -- 172.21.15.45:0/1382994858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4106de0 msgr2=0x7f36d41071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.200+0000 7f36daa89640 1 --2- 172.21.15.45:0/1382994858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4106de0 0x7f36d41071c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009ca0 tx=0x7f36c402c180 comp rx=0 tx=0).stop 2024-01-31T05:39:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.200+0000 7f36daa89640 1 -- 172.21.15.45:0/1382994858 shutdown_connections 2024-01-31T05:39:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.200+0000 7f36daa89640 1 --2- 172.21.15.45:0/1382994858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36d4068d10 0x7f36d410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.200+0000 7f36daa89640 1 --2- 172.21.15.45:0/1382994858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36d4068370 0x7f36d40687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.200+0000 7f36daa89640 1 --2- 172.21.15.45:0/1382994858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4106de0 0x7f36d41071c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.200+0000 7f36daa89640 1 -- 172.21.15.45:0/1382994858 >> 172.21.15.45:0/1382994858 conn(0x7f36d4076a40 msgr2=0x7f36d4076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.200+0000 7f36daa89640 1 -- 172.21.15.45:0/1382994858 shutdown_connections 2024-01-31T05:39:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.200+0000 7f36daa89640 1 -- 172.21.15.45:0/1382994858 wait complete. 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36daa89640 1 Processor -- start 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36daa89640 1 -- start start 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36daa89640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36d4068370 0x7f36d4197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36daa89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4068d10 0x7f36d4197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36daa89640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36d4106de0 0x7f36d419b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36daa89640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d419bfc0 con 0x7f36d4068d10 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36daa89640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d419c130 con 0x7f36d4106de0 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36daa89640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d419c2f0 con 0x7f36d4068370 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36d3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4068d10 0x7f36d4197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36d3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4068d10 0x7f36d4197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49192/0 (socket says 172.21.15.45:49192) 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36d3fff640 1 -- 172.21.15.45:0/1588133508 learned_addr learned my addr 172.21.15.45:0/1588133508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36d8825640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36d4068370 0x7f36d4197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36d9026640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36d4106de0 0x7f36d419b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36d3fff640 1 -- 172.21.15.45:0/1588133508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36d4068370 msgr2=0x7f36d4197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36d3fff640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36d4068370 0x7f36d4197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36d3fff640 1 -- 172.21.15.45:0/1588133508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36d4106de0 msgr2=0x7f36d419b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36d3fff640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36d4106de0 0x7f36d419b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.201+0000 7f36d3fff640 1 -- 172.21.15.45:0/1588133508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c4009900 con 0x7f36d4068d10 2024-01-31T05:39:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.202+0000 7f36d3fff640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4068d10 0x7f36d4197640 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00d950 tx=0x7f36bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.202+0000 7f36d1ffb640 1 -- 172.21.15.45:0/1588133508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc009c30 con 0x7f36d4068d10 2024-01-31T05:39:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.202+0000 7f36d1ffb640 1 -- 172.21.15.45:0/1588133508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36bc009d90 con 0x7f36d4068d10 2024-01-31T05:39:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.202+0000 7f36d1ffb640 1 -- 172.21.15.45:0/1588133508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc00f740 con 0x7f36d4068d10 2024-01-31T05:39:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.202+0000 7f36daa89640 1 -- 172.21.15.45:0/1588133508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d419c5d0 con 0x7f36d4068d10 2024-01-31T05:39:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.202+0000 7f36daa89640 1 -- 172.21.15.45:0/1588133508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d41a3e50 con 0x7f36d4068d10 2024-01-31T05:39:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.204+0000 7f36d1ffb640 1 -- 172.21.15.45:0/1588133508 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f36bc010a20 con 0x7f36d4068d10 2024-01-31T05:39:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36dd84a740 1 Processor -- start 2024-01-31T05:39:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36d1ffb640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f36c00a1570 0x7f36c00a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36dd84a740 1 -- start start 2024-01-31T05:39:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36d1ffb640 1 -- 172.21.15.45:0/1588133508 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f36bc0ccaa0 con 0x7f36d4068d10 2024-01-31T05:39:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36d9026640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f36c00a1570 0x7f36c00a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36dd84a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4060 0x55ffeb3f4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36dd84a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3fd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36dd84a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb3ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36dd84a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeb363040 con 0x55ffeb3f4980 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36dd84a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeb360730 con 0x55ffeb3f4060 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36dd84a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeb35c020 con 0x55ffeb3fd750 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36d3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.205+0000 7f36d8825640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4060 0x55ffeb3f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36d8825640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4060 0x55ffeb3f4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:33546/0 (socket says 172.21.15.45:33546) 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36d8825640 1 -- 172.21.15.45:0/2348069759 learned_addr learned my addr 172.21.15.45:0/2348069759 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36d9026640 1 --2- 172.21.15.45:0/2348069759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb3ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36d3fff640 1 -- 172.21.15.45:0/2348069759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 msgr2=0x55ffeb3ffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36d3fff640 1 --2- 172.21.15.45:0/2348069759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb3ffb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36d3fff640 1 -- 172.21.15.45:0/2348069759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4060 msgr2=0x55ffeb3f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36d3fff640 1 --2- 172.21.15.45:0/2348069759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4060 0x55ffeb3f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36d3fff640 1 -- 172.21.15.45:0/2348069759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffeb353e40 con 0x55ffeb3f4980 2024-01-31T05:39:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36d8825640 1 --2- 172.21.15.45:0/2348069759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4060 0x55ffeb3f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36d3fff640 1 --2- 172.21.15.45:0/2348069759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3fd210 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0ca1f0 tx=0x7f36bc099590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23892bc8770ee935 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36d9026640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f36c00a1570 0x7f36c00a3a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f36cc011440 tx=0x7f36cc011840 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36caffd640 1 -- 172.21.15.45:0/2348069759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0a1c30 con 0x55ffeb3f4980 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36caffd640 1 -- 172.21.15.45:0/2348069759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36bc0b9070 con 0x55ffeb3f4980 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.206+0000 7f36caffd640 1 -- 172.21.15.45:0/2348069759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0a19e0 con 0x55ffeb3f4980 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.207+0000 7f36dd84a740 1 -- 172.21.15.45:0/2348069759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 msgr2=0x55ffeb3fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.207+0000 7f36dd84a740 1 --2- 172.21.15.45:0/2348069759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3fd210 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0ca1f0 tx=0x7f36bc099590 comp rx=0 tx=0).stop 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.207+0000 7f36dd84a740 1 -- 172.21.15.45:0/2348069759 shutdown_connections 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.207+0000 7f36dd84a740 1 --2- 172.21.15.45:0/2348069759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3fd210 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.207+0000 7f36dd84a740 1 --2- 172.21.15.45:0/2348069759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb3ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.207+0000 7f36dd84a740 1 --2- 172.21.15.45:0/2348069759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4060 0x55ffeb3f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.207+0000 7f36dd84a740 1 -- 172.21.15.45:0/2348069759 >> 172.21.15.45:0/2348069759 conn(0x55ffeb3ee980 msgr2=0x55ffeb3eed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.207+0000 7f36dd84a740 1 -- 172.21.15.45:0/2348069759 shutdown_connections 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.207+0000 7f36dd84a740 1 -- 172.21.15.45:0/2348069759 wait complete. 2024-01-31T05:39:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.207+0000 7f36dd84a740 1 -- start start 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.207+0000 7f36dd84a740 1 Processor -- start 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36dd84a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3f4060 0x55ffeb3fa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36dd84a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3fac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36dd84a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3fd750 0x55ffeb3fb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36dd84a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeb3151b0 con 0x55ffeb3f4980 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36dd84a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeb314e50 con 0x55ffeb3fd750 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36dd84a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeb2da090 con 0x55ffeb3f4060 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36dd84a740 10 client.0 _ll_register_callbacks cb 0x55ffeb2ffa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36d3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36d3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3fac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49208/0 (socket says 172.21.15.45:49208) 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36d3fff640 1 -- 172.21.15.45:0/1114099606 learned_addr learned my addr 172.21.15.45:0/1114099606 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36d8825640 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3f4060 0x55ffeb3fa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36d9026640 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3fd750 0x55ffeb3fb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36d3fff640 1 -- 172.21.15.45:0/1114099606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3f4060 msgr2=0x55ffeb3fa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36d3fff640 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3f4060 0x55ffeb3fa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36d3fff640 1 -- 172.21.15.45:0/1114099606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3fd750 msgr2=0x55ffeb3fb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36d3fff640 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3fd750 0x55ffeb3fb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36d3fff640 1 -- 172.21.15.45:0/1114099606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d41a3e50 con 0x55ffeb3f4980 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36d9026640 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3fd750 0x55ffeb3fb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36d3fff640 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3fac00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009610 tx=0x7f36bc09dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c902c70086a62652 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36b3fff640 1 -- 172.21.15.45:0/1114099606 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc09abd0 con 0x55ffeb3f4980 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36b3fff640 1 -- 172.21.15.45:0/1114099606 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f36bc09ad30 con 0x55ffeb3f4980 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.208+0000 7f36dd84a740 1 -- 172.21.15.45:0/1114099606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffeb355010 con 0x55ffeb3f4980 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.209+0000 7f36b3fff640 1 -- 172.21.15.45:0/1114099606 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36bc0a17a0 con 0x55ffeb3f4980 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.209+0000 7f36dd84a740 10 client.16734 fetch_fsmap learned FSMap version 33 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.209+0000 7f36dd84a740 1 -- 172.21.15.45:0/1114099606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ffeb35a6a0 con 0x55ffeb3f4980 2024-01-31T05:39:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.209+0000 7f36b3fff640 1 -- 172.21.15.45:0/1114099606 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffeb355010 con 0x55ffeb3f4980 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.209+0000 7f36b3fff640 1 -- 172.21.15.45:0/1114099606 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f36bc0b5070 con 0x55ffeb3f4980 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.209+0000 7f36dd84a740 10 client.16734 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.209+0000 7f36dd84a740 10 client.16734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.209+0000 7f36dd84a740 20 client.16734 populate_metadata read hostname 'smithi045' 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.209+0000 7f36dd84a740 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ffeb40d050 0x55ffeb40f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.209+0000 7f36dd84a740 4 client.16734 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.209+0000 7f36dd84a740 1 -- 172.21.15.45:0/1114099606 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffeb354d30 con 0x55ffeb40d050 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.209+0000 7f36d8825640 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ffeb40d050 0x55ffeb40f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36b3fff640 10 client.16734 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36d8825640 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ffeb40d050 0x55ffeb40f440 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d0a85e0078d14a9 server_cookie=f36106dd0092deaa in_seq=0 out_seq=0 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36b3fff640 1 -- 172.21.15.45:0/1114099606 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffeb354d30 con 0x55ffeb40d050 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36b3fff640 10 client.16734 handle_command_reply: tid=0 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36dd84a740 1 client.16734 shutdown 2024-01-31T05:39:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36dd84a740 1 -- 172.21.15.45:0/1114099606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 msgr2=0x55ffeb3fac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36dd84a740 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3fac00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009610 tx=0x7f36bc09dab0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36dd84a740 1 -- 172.21.15.45:0/1114099606 shutdown_connections 2024-01-31T05:39:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36dd84a740 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ffeb40d050 0x55ffeb40f440 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36dd84a740 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3fac00 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36dd84a740 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3fd750 0x55ffeb3fb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36b3fff640 0 client.16734 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36dd84a740 1 --2- 172.21.15.45:0/1114099606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3f4060 0x55ffeb3fa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.210+0000 7f36dd84a740 1 -- 172.21.15.45:0/1114099606 >> 172.21.15.45:0/1114099606 conn(0x55ffeb3ee980 msgr2=0x55ffeb3f06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.211+0000 7f36dd84a740 1 -- 172.21.15.45:0/1114099606 shutdown_connections 2024-01-31T05:39:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.211+0000 7f36dd84a740 1 -- 172.21.15.45:0/1114099606 wait complete. 2024-01-31T05:39:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.211+0000 7f36dd84a740 20 client.16734 trim_cache size 0 max 16384 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36dd84a740 1 Processor -- start 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36dd84a740 1 -- start start 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36dd84a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4060 0x55ffeb3fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36dd84a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3faf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36dd84a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb3f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36dd84a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeb3ef980 con 0x55ffeb3f4980 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36dd84a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeb3ffb40 con 0x55ffeb3f4060 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36dd84a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeb3f5550 con 0x55ffeb3fd750 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36d3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36d3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3faf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49220/0 (socket says 172.21.15.45:49220) 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36d3fff640 1 -- 172.21.15.45:0/2609960661 learned_addr learned my addr 172.21.15.45:0/2609960661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36d8825640 1 --2- 172.21.15.45:0/2609960661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4060 0x55ffeb3fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.226+0000 7f36d9026640 1 --2- 172.21.15.45:0/2609960661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb3f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.227+0000 7f36d3fff640 1 -- 172.21.15.45:0/2609960661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 msgr2=0x55ffeb3f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.227+0000 7f36d3fff640 1 --2- 172.21.15.45:0/2609960661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb3f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.227+0000 7f36d3fff640 1 -- 172.21.15.45:0/2609960661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4060 msgr2=0x55ffeb3fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.227+0000 7f36d3fff640 1 --2- 172.21.15.45:0/2609960661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4060 0x55ffeb3fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.227+0000 7f36d3fff640 1 -- 172.21.15.45:0/2609960661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffeb2f7030 con 0x55ffeb3f4980 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.227+0000 7f36d9026640 1 --2- 172.21.15.45:0/2609960661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb3f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.227+0000 7f36d3fff640 1 --2- 172.21.15.45:0/2609960661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3faf60 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0ca7f0 tx=0x7f36bc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ca8a89057080a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.227+0000 7f36ca7fc640 1 -- 172.21.15.45:0/2609960661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc098c80 con 0x55ffeb3f4980 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.227+0000 7f36ca7fc640 1 -- 172.21.15.45:0/2609960661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36bc099490 con 0x55ffeb3f4980 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.227+0000 7f36ca7fc640 1 -- 172.21.15.45:0/2609960661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc09abd0 con 0x55ffeb3f4980 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 -- 172.21.15.45:0/2609960661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 msgr2=0x55ffeb3faf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 --2- 172.21.15.45:0/2609960661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3faf60 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0ca7f0 tx=0x7f36bc00bf60 comp rx=0 tx=0).stop 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 -- 172.21.15.45:0/2609960661 shutdown_connections 2024-01-31T05:39:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 --2- 172.21.15.45:0/2609960661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4980 0x55ffeb3faf60 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 --2- 172.21.15.45:0/2609960661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb3f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 --2- 172.21.15.45:0/2609960661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4060 0x55ffeb3fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 -- 172.21.15.45:0/2609960661 >> 172.21.15.45:0/2609960661 conn(0x55ffeb3ee980 msgr2=0x55ffeb3f3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 -- 172.21.15.45:0/2609960661 shutdown_connections 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 -- 172.21.15.45:0/2609960661 wait complete. 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 -- start start 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 Processor -- start 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4060 0x55ffeb402550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.228+0000 7f36dd84a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4980 0x55ffeb402a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36dd84a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb4624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36dd84a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeb462a30 con 0x55ffeb3f4060 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36dd84a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeb462b70 con 0x55ffeb3f4980 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36dd84a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeb462cb0 con 0x55ffeb3fd750 2024-01-31T05:39:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36dd84a740 10 client.0 _ll_register_callbacks cb 0x7f36bc00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36d8825640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4060 0x55ffeb402550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36d8825640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4060 0x55ffeb402550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49232/0 (socket says 172.21.15.45:49232) 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36d8825640 1 -- 172.21.15.45:0/1942293588 learned_addr learned my addr 172.21.15.45:0/1942293588 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36d9026640 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb4624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36d3fff640 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4980 0x55ffeb402a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36d8825640 1 -- 172.21.15.45:0/1942293588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 msgr2=0x55ffeb4624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36d8825640 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb4624f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36d8825640 1 -- 172.21.15.45:0/1942293588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4980 msgr2=0x55ffeb402a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36d8825640 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4980 0x55ffeb402a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36d8825640 1 -- 172.21.15.45:0/1942293588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffeb35a6a0 con 0x55ffeb3f4060 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36d8825640 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4060 0x55ffeb402550 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f36c4037040 tx=0x7f36c402c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5f6352c36a0400 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36b3fff640 1 -- 172.21.15.45:0/1942293588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4034640 con 0x55ffeb3f4060 2024-01-31T05:39:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36dd84a740 1 -- 172.21.15.45:0/1942293588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36bc0b9070 con 0x55ffeb3f4060 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36b3fff640 1 -- 172.21.15.45:0/1942293588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f36c4046040 con 0x55ffeb3f4060 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36b3fff640 1 -- 172.21.15.45:0/1942293588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36c40347a0 con 0x55ffeb3f4060 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36dd84a740 10 client.16740 fetch_fsmap learned FSMap version 33 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.229+0000 7f36dd84a740 1 -- 172.21.15.45:0/1942293588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ffeb31ca50 con 0x55ffeb3f4060 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.230+0000 7f36b3fff640 1 -- 172.21.15.45:0/1942293588 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36bc0b9070 con 0x55ffeb3f4060 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.230+0000 7f36b3fff640 1 -- 172.21.15.45:0/1942293588 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f36c4052020 con 0x55ffeb3f4060 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.230+0000 7f36dd84a740 10 client.16740 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.230+0000 7f36dd84a740 10 client.16740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.230+0000 7f36dd84a740 20 client.16740 populate_metadata read hostname 'smithi045' 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.230+0000 7f36dd84a740 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ffeb4644e0 0x55ffeb4669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.230+0000 7f36dd84a740 4 client.16740 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.230+0000 7f36dd84a740 1 -- 172.21.15.45:0/1942293588 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ffeb05b600 con 0x55ffeb4644e0 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.230+0000 7f36d3fff640 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ffeb4644e0 0x55ffeb4669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.231+0000 7f36d3fff640 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ffeb4644e0 0x55ffeb4669a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7fa7133b654cce1 server_cookie=91ae8683eb4acf22 in_seq=0 out_seq=0 2024-01-31T05:39:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.231+0000 7f36b3fff640 10 client.16740 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:21.236 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:20 smithi156 ceph-mon[35720]: pgmap v530: 97 pgs: 97 active+clean; 1018 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.3 MiB/s wr, 480 op/s 2024-01-31T05:39:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36b3fff640 1 -- 172.21.15.45:0/1942293588 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ffeb05b600 con 0x55ffeb4644e0 2024-01-31T05:39:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36b3fff640 10 client.16740 handle_command_reply: tid=0 2024-01-31T05:39:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36dd84a740 1 client.16740 shutdown 2024-01-31T05:39:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36dd84a740 1 -- 172.21.15.45:0/1942293588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4060 msgr2=0x55ffeb402550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36dd84a740 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4060 0x55ffeb402550 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f36c4037040 tx=0x7f36c402c930 comp rx=0 tx=0).stop 2024-01-31T05:39:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36dd84a740 1 -- 172.21.15.45:0/1942293588 shutdown_connections 2024-01-31T05:39:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36dd84a740 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55ffeb4644e0 0x55ffeb4669a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36dd84a740 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ffeb3fd750 0x55ffeb4624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36dd84a740 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ffeb3f4980 0x55ffeb402a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36b3fff640 0 client.16740 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36dd84a740 1 --2- 172.21.15.45:0/1942293588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ffeb3f4060 0x55ffeb402550 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36dd84a740 1 -- 172.21.15.45:0/1942293588 >> 172.21.15.45:0/1942293588 conn(0x55ffeb3ee980 msgr2=0x55ffeb3f8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36dd84a740 1 -- 172.21.15.45:0/1942293588 shutdown_connections 2024-01-31T05:39:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36dd84a740 1 -- 172.21.15.45:0/1942293588 wait complete. 2024-01-31T05:39:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.281+0000 7f36dd84a740 20 client.16740 trim_cache size 0 max 16384 2024-01-31T05:39:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:39:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:39:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:39:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:39:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.282+0000 7f36daa89640 1 -- 172.21.15.45:0/1588133508 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f36c00a1570 msgr2=0x7f36c00a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.282+0000 7f36daa89640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f36c00a1570 0x7f36c00a3a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f36cc011440 tx=0x7f36cc011840 comp rx=0 tx=0).stop 2024-01-31T05:39:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.282+0000 7f36daa89640 1 -- 172.21.15.45:0/1588133508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4068d10 msgr2=0x7f36d4197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.282+0000 7f36daa89640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4068d10 0x7f36d4197640 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00d950 tx=0x7f36bc00de10 comp rx=0 tx=0).stop 2024-01-31T05:39:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.282+0000 7f36daa89640 1 -- 172.21.15.45:0/1588133508 shutdown_connections 2024-01-31T05:39:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.282+0000 7f36daa89640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f36c00a1570 0x7f36c00a3a30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.282+0000 7f36daa89640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4068d10 0x7f36d4197640 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.282+0000 7f36daa89640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36d4106de0 0x7f36d419b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.282+0000 7f36daa89640 1 --2- 172.21.15.45:0/1588133508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36d4068370 0x7f36d4197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.282+0000 7f36daa89640 1 -- 172.21.15.45:0/1588133508 >> 172.21.15.45:0/1588133508 conn(0x7f36d4076a40 msgr2=0x7f36d40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.282+0000 7f36daa89640 1 -- 172.21.15.45:0/1588133508 shutdown_connections 2024-01-31T05:39:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:21.282+0000 7f36daa89640 1 -- 172.21.15.45:0/1588133508 wait complete. 2024-01-31T05:39:21.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 85360 22.66 MB/sec execute 80 sec latency 236.088 ms 2024-01-31T05:39:22.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 85660 22.56 MB/sec execute 81 sec latency 232.111 ms 2024-01-31T05:39:23.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:22 smithi186 ceph-mon[35659]: pgmap v531: 97 pgs: 97 active+clean; 1018 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.3 MiB/s wr, 480 op/s 2024-01-31T05:39:23.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:22 smithi045 ceph-mon[31585]: pgmap v531: 97 pgs: 97 active+clean; 1018 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.3 MiB/s wr, 480 op/s 2024-01-31T05:39:23.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:22 smithi156 ceph-mon[35720]: pgmap v531: 97 pgs: 97 active+clean; 1018 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.3 MiB/s wr, 480 op/s 2024-01-31T05:39:23.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 86007 22.49 MB/sec execute 82 sec latency 422.956 ms 2024-01-31T05:39:24.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 86479 22.52 MB/sec execute 83 sec latency 187.356 ms 2024-01-31T05:39:25.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:24 smithi186 ceph-mon[35659]: pgmap v532: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.0 MiB/s wr, 448 op/s 2024-01-31T05:39:25.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:24 smithi045 ceph-mon[31585]: pgmap v532: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.0 MiB/s wr, 448 op/s 2024-01-31T05:39:25.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:24 smithi156 ceph-mon[35720]: pgmap v532: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.0 MiB/s wr, 448 op/s 2024-01-31T05:39:25.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 86894 22.51 MB/sec execute 84 sec latency 182.036 ms 2024-01-31T05:39:26.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 87194 22.39 MB/sec execute 85 sec latency 188.911 ms 2024-01-31T05:39:27.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:26 smithi186 ceph-mon[35659]: pgmap v533: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.2 MiB/s wr, 410 op/s 2024-01-31T05:39:27.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:26 smithi045 ceph-mon[31585]: pgmap v533: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.2 MiB/s wr, 410 op/s 2024-01-31T05:39:27.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:26 smithi156 ceph-mon[35720]: pgmap v533: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.2 MiB/s wr, 410 op/s 2024-01-31T05:39:27.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 87596 22.39 MB/sec execute 86 sec latency 116.713 ms 2024-01-31T05:39:28.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 87965 22.35 MB/sec execute 87 sec latency 338.436 ms 2024-01-31T05:39:29.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:28 smithi186 ceph-mon[35659]: pgmap v534: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 576 op/s 2024-01-31T05:39:29.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:28 smithi045 ceph-mon[31585]: pgmap v534: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 576 op/s 2024-01-31T05:39:29.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:28 smithi156 ceph-mon[35720]: pgmap v534: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 576 op/s 2024-01-31T05:39:29.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 88572 22.48 MB/sec execute 88 sec latency 156.025 ms 2024-01-31T05:39:30.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 89110 22.56 MB/sec execute 89 sec latency 193.180 ms 2024-01-31T05:39:31.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:30 smithi186 ceph-mon[35659]: pgmap v535: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.5 MiB/s wr, 371 op/s 2024-01-31T05:39:31.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:30 smithi045 ceph-mon[31585]: pgmap v535: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.5 MiB/s wr, 371 op/s 2024-01-31T05:39:31.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:30 smithi156 ceph-mon[35720]: pgmap v535: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.5 MiB/s wr, 371 op/s 2024-01-31T05:39:31.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 89612 22.64 MB/sec execute 90 sec latency 203.183 ms 2024-01-31T05:39:32.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 90084 22.77 MB/sec execute 91 sec latency 202.067 ms 2024-01-31T05:39:33.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:32 smithi186 ceph-mon[35659]: pgmap v536: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.5 MiB/s wr, 371 op/s 2024-01-31T05:39:33.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:32 smithi045 ceph-mon[31585]: pgmap v536: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.5 MiB/s wr, 371 op/s 2024-01-31T05:39:33.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:32 smithi156 ceph-mon[35720]: pgmap v536: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.5 MiB/s wr, 371 op/s 2024-01-31T05:39:33.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 90416 22.67 MB/sec execute 92 sec latency 345.986 ms 2024-01-31T05:39:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.375+0000 7f813a419640 1 -- 172.21.15.45:0/2291454761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81340ffe40 msgr2=0x7f81341002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.375+0000 7f813a419640 1 --2- 172.21.15.45:0/2291454761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81340ffe40 0x7f81341002c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f811c009900 tx=0x7f811c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:39:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.375+0000 7f813a419640 1 -- 172.21.15.45:0/2291454761 shutdown_connections 2024-01-31T05:39:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.375+0000 7f813a419640 1 --2- 172.21.15.45:0/2291454761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81340ffe40 0x7f81341002c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.375+0000 7f813a419640 1 --2- 172.21.15.45:0/2291454761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8134100800 0x7f813410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.375+0000 7f813a419640 1 --2- 172.21.15.45:0/2291454761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8134104b90 0x7f8134104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.375+0000 7f813a419640 1 -- 172.21.15.45:0/2291454761 >> 172.21.15.45:0/2291454761 conn(0x7f8134076a40 msgr2=0x7f8134076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.375+0000 7f813a419640 1 -- 172.21.15.45:0/2291454761 shutdown_connections 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.375+0000 7f813a419640 1 -- 172.21.15.45:0/2291454761 wait complete. 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.375+0000 7f813a419640 1 Processor -- start 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.375+0000 7f813a419640 1 -- start start 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f813a419640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134100800 0x7f8134197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f813a419640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8134104b90 0x7f8134197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f813a419640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813419bcd0 0x7f813419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f813a419640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813419c6c0 con 0x7f8134100800 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f813a419640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813419c830 con 0x7f8134104b90 2024-01-31T05:39:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f813a419640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813419c9a0 con 0x7f813419bcd0 2024-01-31T05:39:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f8133fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134100800 0x7f8134197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f8133fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134100800 0x7f8134197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55114/0 (socket says 172.21.15.45:55114) 2024-01-31T05:39:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f8133fff640 1 -- 172.21.15.45:0/3632491372 learned_addr learned my addr 172.21.15.45:0/3632491372 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f81389b6640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813419bcd0 0x7f813419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f81337fe640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8134104b90 0x7f8134197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f8133fff640 1 -- 172.21.15.45:0/3632491372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813419bcd0 msgr2=0x7f813419c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f8133fff640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813419bcd0 0x7f813419c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f8133fff640 1 -- 172.21.15.45:0/3632491372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8134104b90 msgr2=0x7f8134197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f8133fff640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8134104b90 0x7f8134197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f8133fff640 1 -- 172.21.15.45:0/3632491372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c009560 con 0x7f8134100800 2024-01-31T05:39:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f8133fff640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134100800 0x7f8134197410 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f812400d950 tx=0x7f812400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f81317fa640 1 -- 172.21.15.45:0/3632491372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124009c30 con 0x7f8134100800 2024-01-31T05:39:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.376+0000 7f813a419640 1 -- 172.21.15.45:0/3632491372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81341a3f80 con 0x7f8134100800 2024-01-31T05:39:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.377+0000 7f81317fa640 1 -- 172.21.15.45:0/3632491372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8124009d90 con 0x7f8134100800 2024-01-31T05:39:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.377+0000 7f81317fa640 1 -- 172.21.15.45:0/3632491372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400f740 con 0x7f8134100800 2024-01-31T05:39:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.377+0000 7f813a419640 1 -- 172.21.15.45:0/3632491372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81341a43a0 con 0x7f8134100800 2024-01-31T05:39:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f81317fa640 1 -- 172.21.15.45:0/3632491372 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f8124010a20 con 0x7f8134100800 2024-01-31T05:39:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f813d1db740 1 Processor -- start 2024-01-31T05:39:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f813d1db740 1 -- start start 2024-01-31T05:39:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f81317fa640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f81200a14f0 0x7f81200a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f813d1db740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27da0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f813d1db740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 0x557c27dae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f813d1db740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27daeb60 0x557c27db0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f813d1db740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c27d10040 con 0x557c27da0a20 2024-01-31T05:39:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f813d1db740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c27d0d730 con 0x557c27daeb60 2024-01-31T05:39:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f813d1db740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c27d09020 con 0x557c27da1340 2024-01-31T05:39:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f8133fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27da0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f81389b6640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f81200a14f0 0x7f81200a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f8133fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27da0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55118/0 (socket says 172.21.15.45:55118) 2024-01-31T05:39:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f8133fff640 1 -- 172.21.15.45:0/1448618398 learned_addr learned my addr 172.21.15.45:0/1448618398 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f81337fe640 1 --2- 172.21.15.45:0/1448618398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 0x557c27dae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.379+0000 7f81317fa640 1 -- 172.21.15.45:0/3632491372 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f81240ccaf0 con 0x7f8134100800 2024-01-31T05:39:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.380+0000 7f81389b6640 1 --2- 172.21.15.45:0/1448618398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27daeb60 0x557c27db0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.380+0000 7f8133fff640 1 -- 172.21.15.45:0/1448618398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 msgr2=0x557c27dae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.380+0000 7f8133fff640 1 --2- 172.21.15.45:0/1448618398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 0x557c27dae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.380+0000 7f8133fff640 1 -- 172.21.15.45:0/1448618398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27daeb60 msgr2=0x557c27db0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.380+0000 7f8133fff640 1 --2- 172.21.15.45:0/1448618398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27daeb60 0x557c27db0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.380+0000 7f8133fff640 1 -- 172.21.15.45:0/1448618398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c27d00e40 con 0x557c27da0a20 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.380+0000 7f8133fff640 1 --2- 172.21.15.45:0/1448618398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27da0e00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8124096410 tx=0x7f8124005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b248f1355c86de08 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.380+0000 7f81389b6640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f81200a14f0 0x7f81200a39b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f812c005fd0 tx=0x7f812c013b60 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.380+0000 7f812a7fc640 1 -- 172.21.15.45:0/1448618398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124099300 con 0x557c27da0a20 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.380+0000 7f812a7fc640 1 -- 172.21.15.45:0/1448618398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f812409a0a0 con 0x557c27da0a20 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.380+0000 7f812a7fc640 1 -- 172.21.15.45:0/1448618398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812409da40 con 0x557c27da0a20 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.381+0000 7f813d1db740 1 -- 172.21.15.45:0/1448618398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 msgr2=0x557c27da0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.381+0000 7f813d1db740 1 --2- 172.21.15.45:0/1448618398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27da0e00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8124096410 tx=0x7f8124005880 comp rx=0 tx=0).stop 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.381+0000 7f813d1db740 1 -- 172.21.15.45:0/1448618398 shutdown_connections 2024-01-31T05:39:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.381+0000 7f813d1db740 1 --2- 172.21.15.45:0/1448618398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27daeb60 0x557c27db0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.381+0000 7f813d1db740 1 --2- 172.21.15.45:0/1448618398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 0x557c27dae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.381+0000 7f813d1db740 1 --2- 172.21.15.45:0/1448618398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27da0e00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.381+0000 7f813d1db740 1 -- 172.21.15.45:0/1448618398 >> 172.21.15.45:0/1448618398 conn(0x557c27d9b980 msgr2=0x557c27d9bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.381+0000 7f813d1db740 1 -- 172.21.15.45:0/1448618398 shutdown_connections 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.381+0000 7f813d1db740 1 -- 172.21.15.45:0/1448618398 wait complete. 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.381+0000 7f813d1db740 1 -- start start 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.381+0000 7f813d1db740 1 Processor -- start 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f813d1db740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27daba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f813d1db740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27da1340 0x557c27dabfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f813d1db740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27daeb60 0x557c27dac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f813d1db740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c27cc21b0 con 0x557c27da0a20 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f813d1db740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c27cc1e50 con 0x557c27da1340 2024-01-31T05:39:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f813d1db740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c27c87090 con 0x557c27daeb60 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f813d1db740 10 client.0 _ll_register_callbacks cb 0x557c27caca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f8133fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27daba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f8133fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27daba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55126/0 (socket says 172.21.15.45:55126) 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f8133fff640 1 -- 172.21.15.45:0/1819417375 learned_addr learned my addr 172.21.15.45:0/1819417375 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f81389b6640 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27daeb60 0x557c27dac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f81337fe640 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27da1340 0x557c27dabfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f8133fff640 1 -- 172.21.15.45:0/1819417375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27daeb60 msgr2=0x557c27dac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f8133fff640 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27daeb60 0x557c27dac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f8133fff640 1 -- 172.21.15.45:0/1819417375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27da1340 msgr2=0x557c27dabfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f8133fff640 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27da1340 0x557c27dabfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f8133fff640 1 -- 172.21.15.45:0/1819417375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81341a43a0 con 0x557c27da0a20 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f81337fe640 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27da1340 0x557c27dabfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f8133fff640 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27daba60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8124095ea0 tx=0x7f8124000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dafb7ff9ff4a71c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f81137fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f81137fe640 1 -- 172.21.15.45:0/1819417375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124099300 con 0x557c27da0a20 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.382+0000 7f813d1db740 1 -- 172.21.15.45:0/1819417375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c27d02010 con 0x557c27da0a20 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f81137fe640 1 -- 172.21.15.45:0/1819417375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f81240b90e0 con 0x557c27da0a20 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f81137fe640 1 -- 172.21.15.45:0/1819417375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81240a1c80 con 0x557c27da0a20 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f813d1db740 10 client.16752 fetch_fsmap learned FSMap version 33 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f813d1db740 1 -- 172.21.15.45:0/1819417375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557c27d076a0 con 0x557c27da0a20 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f81137fe640 1 -- 172.21.15.45:0/1819417375 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c27d02010 con 0x557c27da0a20 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f81137fe640 1 -- 172.21.15.45:0/1819417375 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f81240a1880 con 0x557c27da0a20 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f813d1db740 10 client.16752 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f813d1db740 10 client.16752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f813d1db740 20 client.16752 populate_metadata read hostname 'smithi045' 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f813d1db740 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557c27dbe410 0x557c27dc0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f813d1db740 4 client.16752 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:39:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f813d1db740 1 -- 172.21.15.45:0/1819417375 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c27d01d30 con 0x557c27dbe410 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.383+0000 7f81337fe640 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557c27dbe410 0x557c27dc0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.384+0000 7f81337fe640 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557c27dbe410 0x557c27dc0800 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7192189a970cd6e server_cookie=e84383f8b0100fb6 in_seq=0 out_seq=0 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.384+0000 7f81137fe640 10 client.16752 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.384+0000 7f81137fe640 1 -- 172.21.15.45:0/1819417375 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c27d01d30 con 0x557c27dbe410 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.384+0000 7f81137fe640 10 client.16752 handle_command_reply: tid=0 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.384+0000 7f813d1db740 1 client.16752 shutdown 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.385+0000 7f813d1db740 1 -- 172.21.15.45:0/1819417375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 msgr2=0x557c27daba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.385+0000 7f813d1db740 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27daba60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8124095ea0 tx=0x7f8124000bc0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.385+0000 7f813d1db740 1 -- 172.21.15.45:0/1819417375 shutdown_connections 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.385+0000 7f813d1db740 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557c27dbe410 0x557c27dc0800 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.385+0000 7f813d1db740 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27daeb60 0x557c27dac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.385+0000 7f813d1db740 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27da1340 0x557c27dabfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.385+0000 7f813d1db740 1 --2- 172.21.15.45:0/1819417375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27daba60 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.385+0000 7f81137fe640 0 client.16752 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.385+0000 7f813d1db740 1 -- 172.21.15.45:0/1819417375 >> 172.21.15.45:0/1819417375 conn(0x557c27d9b980 msgr2=0x557c27d9d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.385+0000 7f813d1db740 1 -- 172.21.15.45:0/1819417375 shutdown_connections 2024-01-31T05:39:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.385+0000 7f813d1db740 1 -- 172.21.15.45:0/1819417375 wait complete. 2024-01-31T05:39:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.385+0000 7f813d1db740 20 client.16752 trim_cache size 0 max 16384 2024-01-31T05:39:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f813d1db740 1 Processor -- start 2024-01-31T05:39:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f813d1db740 1 -- start start 2024-01-31T05:39:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f813d1db740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27da0a20 0x557c27dae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f813d1db740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 0x557c27dac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f813d1db740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27daeb60 0x557c27db57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f813d1db740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c27d01d30 con 0x557c27daeb60 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f813d1db740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c27d02010 con 0x557c27da0a20 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f813d1db740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c27d9c9d0 con 0x557c27da1340 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f81389b6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27daeb60 0x557c27db57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f81389b6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27daeb60 0x557c27db57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55136/0 (socket says 172.21.15.45:55136) 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f81389b6640 1 -- 172.21.15.45:0/2766095035 learned_addr learned my addr 172.21.15.45:0/2766095035 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f81337fe640 1 --2- 172.21.15.45:0/2766095035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 0x557c27dac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f8133fff640 1 --2- 172.21.15.45:0/2766095035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27da0a20 0x557c27dae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f81389b6640 1 -- 172.21.15.45:0/2766095035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 msgr2=0x557c27dac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.399+0000 7f81389b6640 1 --2- 172.21.15.45:0/2766095035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 0x557c27dac340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.400+0000 7f81389b6640 1 -- 172.21.15.45:0/2766095035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27da0a20 msgr2=0x557c27dae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.400+0000 7f81389b6640 1 --2- 172.21.15.45:0/2766095035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27da0a20 0x557c27dae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.400+0000 7f81389b6640 1 -- 172.21.15.45:0/2766095035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c27ca4030 con 0x557c27daeb60 2024-01-31T05:39:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.400+0000 7f8133fff640 1 --2- 172.21.15.45:0/2766095035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27da0a20 0x557c27dae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:39:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.400+0000 7f81389b6640 1 --2- 172.21.15.45:0/2766095035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27daeb60 0x557c27db57f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f812c00fb20 tx=0x7f812c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6c301487e3c7a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.400+0000 7f8129ffb640 1 -- 172.21.15.45:0/2766095035 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c003b70 con 0x557c27daeb60 2024-01-31T05:39:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.400+0000 7f8129ffb640 1 -- 172.21.15.45:0/2766095035 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f812c003cd0 con 0x557c27daeb60 2024-01-31T05:39:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.400+0000 7f8129ffb640 1 -- 172.21.15.45:0/2766095035 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c005020 con 0x557c27daeb60 2024-01-31T05:39:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 -- 172.21.15.45:0/2766095035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27daeb60 msgr2=0x557c27db57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 --2- 172.21.15.45:0/2766095035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27daeb60 0x557c27db57f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f812c00fb20 tx=0x7f812c009500 comp rx=0 tx=0).stop 2024-01-31T05:39:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 -- 172.21.15.45:0/2766095035 shutdown_connections 2024-01-31T05:39:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 --2- 172.21.15.45:0/2766095035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27daeb60 0x557c27db57f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 --2- 172.21.15.45:0/2766095035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 0x557c27dac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 --2- 172.21.15.45:0/2766095035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27da0a20 0x557c27dae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 -- 172.21.15.45:0/2766095035 >> 172.21.15.45:0/2766095035 conn(0x557c27d9b980 msgr2=0x557c27d9bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 -- 172.21.15.45:0/2766095035 shutdown_connections 2024-01-31T05:39:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 -- 172.21.15.45:0/2766095035 wait complete. 2024-01-31T05:39:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 -- start start 2024-01-31T05:39:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 Processor -- start 2024-01-31T05:39:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27e0e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 0x557c27e0e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27daeb60 0x557c27e08830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c27e08e40 con 0x557c27da0a20 2024-01-31T05:39:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c27e08fb0 con 0x557c27daeb60 2024-01-31T05:39:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.401+0000 7f813d1db740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c27e09170 con 0x557c27da1340 2024-01-31T05:39:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f813d1db740 10 client.0 _ll_register_callbacks cb 0x7f811c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:39:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f8133fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27e0e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f8133fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27e0e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55150/0 (socket says 172.21.15.45:55150) 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f8133fff640 1 -- 172.21.15.45:0/3933327115 learned_addr learned my addr 172.21.15.45:0/3933327115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f81337fe640 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 0x557c27e0e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f81389b6640 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27daeb60 0x557c27e08830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f8133fff640 1 -- 172.21.15.45:0/3933327115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 msgr2=0x557c27e0e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f8133fff640 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 0x557c27e0e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f8133fff640 1 -- 172.21.15.45:0/3933327115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27daeb60 msgr2=0x557c27e08830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f8133fff640 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27daeb60 0x557c27e08830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f8133fff640 1 -- 172.21.15.45:0/3933327115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c012270 con 0x557c27da0a20 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f8133fff640 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27e0e0f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f81240ca840 tx=0x7f8124096360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3409c5868ae7a50e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f81137fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f81137fe640 1 -- 172.21.15.45:0/3933327115 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81240a64b0 con 0x557c27da0a20 2024-01-31T05:39:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f81137fe640 1 -- 172.21.15.45:0/3933327115 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f81240a1c80 con 0x557c27da0a20 2024-01-31T05:39:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f813d1db740 1 -- 172.21.15.45:0/3933327115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f812409a0a0 con 0x557c27da0a20 2024-01-31T05:39:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f81137fe640 1 -- 172.21.15.45:0/3933327115 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81240a6650 con 0x557c27da0a20 2024-01-31T05:39:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f813d1db740 10 client.16758 fetch_fsmap learned FSMap version 33 2024-01-31T05:39:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.402+0000 7f813d1db740 1 -- 172.21.15.45:0/3933327115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557c27cc9a50 con 0x557c27da0a20 2024-01-31T05:39:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.403+0000 7f81137fe640 1 -- 172.21.15.45:0/3933327115 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f812409a0a0 con 0x557c27da0a20 2024-01-31T05:39:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.403+0000 7f81137fe640 1 -- 172.21.15.45:0/3933327115 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f81240a17f0 con 0x557c27da0a20 2024-01-31T05:39:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.403+0000 7f813d1db740 10 client.16758 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:39:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.403+0000 7f813d1db740 10 client.16758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:39:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.403+0000 7f813d1db740 20 client.16758 populate_metadata read hostname 'smithi045' 2024-01-31T05:39:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.403+0000 7f813d1db740 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557c27e15af0 0x557c27e17ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.403+0000 7f813d1db740 4 client.16758 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:39:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.403+0000 7f813d1db740 1 -- 172.21.15.45:0/3933327115 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c27a08600 con 0x557c27e15af0 2024-01-31T05:39:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.403+0000 7f81337fe640 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557c27e15af0 0x557c27e17ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.403+0000 7f81137fe640 10 client.16758 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.403+0000 7f81337fe640 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557c27e15af0 0x557c27e17ee0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbd0bcc31af26f73 server_cookie=7fb77787cd37c2c in_seq=0 out_seq=0 2024-01-31T05:39:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.467+0000 7f81137fe640 1 -- 172.21.15.45:0/3933327115 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c27a08600 con 0x557c27e15af0 2024-01-31T05:39:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.467+0000 7f81137fe640 10 client.16758 handle_command_reply: tid=0 2024-01-31T05:39:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.467+0000 7f813d1db740 1 client.16758 shutdown 2024-01-31T05:39:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.468+0000 7f813d1db740 1 -- 172.21.15.45:0/3933327115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 msgr2=0x557c27e0e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.468+0000 7f813d1db740 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27e0e0f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f81240ca840 tx=0x7f8124096360 comp rx=0 tx=0).stop 2024-01-31T05:39:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.468+0000 7f813d1db740 1 -- 172.21.15.45:0/3933327115 shutdown_connections 2024-01-31T05:39:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.468+0000 7f813d1db740 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x557c27e15af0 0x557c27e17ee0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.468+0000 7f813d1db740 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c27daeb60 0x557c27e08830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.468+0000 7f813d1db740 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c27da1340 0x557c27e0e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.468+0000 7f81137fe640 0 client.16758 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.468+0000 7f813d1db740 1 --2- 172.21.15.45:0/3933327115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c27da0a20 0x557c27e0e0f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.468+0000 7f813d1db740 1 -- 172.21.15.45:0/3933327115 >> 172.21.15.45:0/3933327115 conn(0x557c27d9b980 msgr2=0x557c27da6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.468+0000 7f813d1db740 1 -- 172.21.15.45:0/3933327115 shutdown_connections 2024-01-31T05:39:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.468+0000 7f813d1db740 1 -- 172.21.15.45:0/3933327115 wait complete. 2024-01-31T05:39:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.468+0000 7f813d1db740 20 client.16758 trim_cache size 0 max 16384 2024-01-31T05:39:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:39:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:39:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:39:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:39:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.470+0000 7f813a419640 1 -- 172.21.15.45:0/3632491372 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f81200a14f0 msgr2=0x7f81200a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.470+0000 7f813a419640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f81200a14f0 0x7f81200a39b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f812c005fd0 tx=0x7f812c013b60 comp rx=0 tx=0).stop 2024-01-31T05:39:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.470+0000 7f813a419640 1 -- 172.21.15.45:0/3632491372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134100800 msgr2=0x7f8134197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.470+0000 7f813a419640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134100800 0x7f8134197410 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f812400d950 tx=0x7f812400de10 comp rx=0 tx=0).stop 2024-01-31T05:39:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.470+0000 7f813a419640 1 -- 172.21.15.45:0/3632491372 shutdown_connections 2024-01-31T05:39:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.470+0000 7f813a419640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f81200a14f0 0x7f81200a39b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.470+0000 7f813a419640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813419bcd0 0x7f813419c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.470+0000 7f813a419640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8134104b90 0x7f8134197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.470+0000 7f813a419640 1 --2- 172.21.15.45:0/3632491372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8134100800 0x7f8134197410 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.470+0000 7f813a419640 1 -- 172.21.15.45:0/3632491372 >> 172.21.15.45:0/3632491372 conn(0x7f8134076a40 msgr2=0x7f8134068710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.471+0000 7f813a419640 1 -- 172.21.15.45:0/3632491372 shutdown_connections 2024-01-31T05:39:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:34.471+0000 7f813a419640 1 -- 172.21.15.45:0/3632491372 wait complete. 2024-01-31T05:39:34.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 90778 22.67 MB/sec execute 93 sec latency 166.359 ms 2024-01-31T05:39:35.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:34 smithi186 ceph-mon[35659]: pgmap v537: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 552 op/s 2024-01-31T05:39:35.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:34 smithi045 ceph-mon[31585]: pgmap v537: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 552 op/s 2024-01-31T05:39:35.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:34 smithi156 ceph-mon[35720]: pgmap v537: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 552 op/s 2024-01-31T05:39:35.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 91141 22.57 MB/sec execute 94 sec latency 62.537 ms 2024-01-31T05:39:36.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 91556 22.50 MB/sec execute 95 sec latency 193.299 ms 2024-01-31T05:39:37.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:36 smithi186 ceph-mon[35659]: pgmap v538: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.8 MiB/s wr, 347 op/s 2024-01-31T05:39:37.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:36 smithi045 ceph-mon[31585]: pgmap v538: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.8 MiB/s wr, 347 op/s 2024-01-31T05:39:37.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:36 smithi156 ceph-mon[35720]: pgmap v538: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.8 MiB/s wr, 347 op/s 2024-01-31T05:39:37.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 92014 22.55 MB/sec execute 96 sec latency 187.158 ms 2024-01-31T05:39:38.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 92338 22.52 MB/sec execute 97 sec latency 334.262 ms 2024-01-31T05:39:39.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:38 smithi186 ceph-mon[35659]: pgmap v539: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.4 MiB/s wr, 506 op/s 2024-01-31T05:39:39.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:38 smithi045 ceph-mon[31585]: pgmap v539: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.4 MiB/s wr, 506 op/s 2024-01-31T05:39:39.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:38 smithi156 ceph-mon[35720]: pgmap v539: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.4 MiB/s wr, 506 op/s 2024-01-31T05:39:39.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 92820 22.52 MB/sec execute 98 sec latency 159.566 ms 2024-01-31T05:39:40.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 93336 22.50 MB/sec execute 99 sec latency 192.814 ms 2024-01-31T05:39:41.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:40 smithi186 ceph-mon[35659]: pgmap v540: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.9 MiB/s wr, 341 op/s 2024-01-31T05:39:41.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:40 smithi045 ceph-mon[31585]: pgmap v540: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.9 MiB/s wr, 341 op/s 2024-01-31T05:39:41.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:40 smithi156 ceph-mon[35720]: pgmap v540: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.9 MiB/s wr, 341 op/s 2024-01-31T05:39:41.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 93828 22.59 MB/sec execute 100 sec latency 314.168 ms 2024-01-31T05:39:42.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 94258 22.67 MB/sec execute 101 sec latency 445.988 ms 2024-01-31T05:39:43.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:42 smithi186 ceph-mon[35659]: pgmap v541: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.9 MiB/s wr, 341 op/s 2024-01-31T05:39:43.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:42 smithi045 ceph-mon[31585]: pgmap v541: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.9 MiB/s wr, 341 op/s 2024-01-31T05:39:43.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:42 smithi156 ceph-mon[35720]: pgmap v541: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.9 MiB/s wr, 341 op/s 2024-01-31T05:39:43.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 94544 22.64 MB/sec execute 102 sec latency 508.917 ms 2024-01-31T05:39:44.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 94932 22.59 MB/sec execute 103 sec latency 236.194 ms 2024-01-31T05:39:45.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:44 smithi045 ceph-mon[31585]: pgmap v542: 97 pgs: 97 active+clean; 1002 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.6 MiB/s wr, 608 op/s 2024-01-31T05:39:45.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:44 smithi156 ceph-mon[35720]: pgmap v542: 97 pgs: 97 active+clean; 1002 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.6 MiB/s wr, 608 op/s 2024-01-31T05:39:45.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:44 smithi186 ceph-mon[35659]: pgmap v542: 97 pgs: 97 active+clean; 1002 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.6 MiB/s wr, 608 op/s 2024-01-31T05:39:45.650 INFO:tasks.workunit.client.0.smithi045.stdout: 10 95336 22.56 MB/sec execute 104 sec latency 193.355 ms 2024-01-31T05:39:46.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 95777 22.52 MB/sec execute 105 sec latency 203.441 ms 2024-01-31T05:39:47.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:46 smithi045 ceph-mon[31585]: pgmap v543: 97 pgs: 97 active+clean; 1002 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 426 op/s 2024-01-31T05:39:47.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:46 smithi156 ceph-mon[35720]: pgmap v543: 97 pgs: 97 active+clean; 1002 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 426 op/s 2024-01-31T05:39:47.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:46 smithi186 ceph-mon[35659]: pgmap v543: 97 pgs: 97 active+clean; 1002 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 426 op/s 2024-01-31T05:39:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.577+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/1162685864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d48100f40 msgr2=0x7f4d48101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.577+0000 7f4d4ec5e640 1 --2- 172.21.15.45:0/1162685864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d48100f40 0x7f4d48101320 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009930 tx=0x7f4d3c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:39:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.577+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/1162685864 shutdown_connections 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.577+0000 7f4d4ec5e640 1 --2- 172.21.15.45:0/1162685864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d48106010 0x7f4d4810dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.577+0000 7f4d4ec5e640 1 --2- 172.21.15.45:0/1162685864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d481018f0 0x7f4d48105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.577+0000 7f4d4ec5e640 1 --2- 172.21.15.45:0/1162685864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d48100f40 0x7f4d48101320 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.577+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/1162685864 >> 172.21.15.45:0/1162685864 conn(0x7f4d480fc7a0 msgr2=0x7f4d480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.577+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/1162685864 shutdown_connections 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.577+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/1162685864 wait complete. 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4ec5e640 1 Processor -- start 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4ec5e640 1 -- start start 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4ec5e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d48100f40 0x7f4d48197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4ec5e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d48106010 0x7f4d48197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4ec5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d4819baa0 0x7f4d4819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4ec5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4819c490 con 0x7f4d4819baa0 2024-01-31T05:39:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4ec5e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4819c600 con 0x7f4d48100f40 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4ec5e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4819c770 con 0x7f4d48106010 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4d1fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d4819baa0 0x7f4d4819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4c9fa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d48100f40 0x7f4d48197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d47fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d48106010 0x7f4d48197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4c9fa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d48100f40 0x7f4d48197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:60048/0 (socket says 172.21.15.45:60048) 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4d1fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d4819baa0 0x7f4d4819bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58304/0 (socket says 172.21.15.45:58304) 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.578+0000 7f4d4c9fa640 1 -- 172.21.15.45:0/2889865862 learned_addr learned my addr 172.21.15.45:0/2889865862 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.579+0000 7f4d4c9fa640 1 -- 172.21.15.45:0/2889865862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d48106010 msgr2=0x7f4d48197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.579+0000 7f4d4c9fa640 1 --2- 172.21.15.45:0/2889865862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d48106010 0x7f4d48197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.579+0000 7f4d4c9fa640 1 -- 172.21.15.45:0/2889865862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d4819baa0 msgr2=0x7f4d4819bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.579+0000 7f4d4c9fa640 1 --2- 172.21.15.45:0/2889865862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d4819baa0 0x7f4d4819bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.579+0000 7f4d4c9fa640 1 -- 172.21.15.45:0/2889865862 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c009560 con 0x7f4d48100f40 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.579+0000 7f4d4c9fa640 1 --2- 172.21.15.45:0/2889865862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d48100f40 0x7f4d48197230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c005940 tx=0x7f4d3c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.579+0000 7f4d45ffb640 1 -- 172.21.15.45:0/2889865862 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c004e50 con 0x7f4d48100f40 2024-01-31T05:39:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.579+0000 7f4d45ffb640 1 -- 172.21.15.45:0/2889865862 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d3c0054c0 con 0x7f4d48100f40 2024-01-31T05:39:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.579+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/2889865862 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d481a3d50 con 0x7f4d48100f40 2024-01-31T05:39:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.579+0000 7f4d45ffb640 1 -- 172.21.15.45:0/2889865862 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c003b20 con 0x7f4d48100f40 2024-01-31T05:39:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.579+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/2889865862 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d481a41b0 con 0x7f4d48100f40 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.581+0000 7f4d45ffb640 1 -- 172.21.15.45:0/2889865862 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f4d3c003c80 con 0x7f4d48100f40 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d51a25740 1 Processor -- start 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d51a25740 1 -- start start 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d51a25740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d911ffa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d45ffb640 1 --2- 172.21.15.45:0/2889865862 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4d380a1570 0x7f4d380a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d51a25740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 0x55d912003210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d51a25740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d912005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d51a25740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d911f69040 con 0x55d912003750 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d51a25740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d911f66730 con 0x55d911ffa060 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d51a25740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d911f62020 con 0x55d911ffa980 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d4d1fb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d911ffa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d4c9fa640 1 --2- 172.21.15.45:0/2889865862 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4d380a1570 0x7f4d380a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d4d1fb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d911ffa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:60062/0 (socket says 172.21.15.45:60062) 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d47fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 0x55d912003210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d4d1fb640 1 -- 172.21.15.45:0/2697376079 learned_addr learned my addr 172.21.15.45:0/2697376079 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/2697376079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d912005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d45ffb640 1 -- 172.21.15.45:0/2889865862 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4d3c0ef450 con 0x7f4d48100f40 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d4d1fb640 1 -- 172.21.15.45:0/2697376079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 msgr2=0x55d912003210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/2697376079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 0x55d912003210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d4d1fb640 1 -- 172.21.15.45:0/2697376079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 msgr2=0x55d912005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.582+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/2697376079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d912005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.583+0000 7f4d4d1fb640 1 -- 172.21.15.45:0/2697376079 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d911f59e40 con 0x55d911ffa060 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.583+0000 7f4d4c9fa640 1 --2- 172.21.15.45:0/2889865862 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4d380a1570 0x7f4d380a3a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0ecd60 tx=0x7f4d3c0b1ad0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.583+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/2697376079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d911ffa440 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4d40005c40 tx=0x7f4d40005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ff6ad14f7e236b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.583+0000 7f4d32ffd640 1 -- 172.21.15.45:0/2697376079 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d40011130 con 0x55d911ffa060 2024-01-31T05:39:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.583+0000 7f4d32ffd640 1 -- 172.21.15.45:0/2697376079 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d40011290 con 0x55d911ffa060 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.583+0000 7f4d32ffd640 1 -- 172.21.15.45:0/2697376079 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000fab0 con 0x55d911ffa060 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.584+0000 7f4d51a25740 1 -- 172.21.15.45:0/2697376079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 msgr2=0x55d911ffa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.584+0000 7f4d51a25740 1 --2- 172.21.15.45:0/2697376079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d911ffa440 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4d40005c40 tx=0x7f4d40005bd0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.584+0000 7f4d51a25740 1 -- 172.21.15.45:0/2697376079 shutdown_connections 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.584+0000 7f4d51a25740 1 --2- 172.21.15.45:0/2697376079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d912005b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.584+0000 7f4d51a25740 1 --2- 172.21.15.45:0/2697376079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 0x55d912003210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.584+0000 7f4d51a25740 1 --2- 172.21.15.45:0/2697376079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d911ffa440 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.584+0000 7f4d51a25740 1 -- 172.21.15.45:0/2697376079 >> 172.21.15.45:0/2697376079 conn(0x55d911ff4980 msgr2=0x55d911ff4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.584+0000 7f4d51a25740 1 -- 172.21.15.45:0/2697376079 shutdown_connections 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.584+0000 7f4d51a25740 1 -- 172.21.15.45:0/2697376079 wait complete. 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.584+0000 7f4d51a25740 1 -- start start 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.584+0000 7f4d51a25740 1 Processor -- start 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.584+0000 7f4d51a25740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d912000680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d51a25740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 0x55d912000bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d51a25740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d912001100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d51a25740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d911f1b1b0 con 0x55d912003750 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d51a25740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d911f1ae50 con 0x55d911ffa060 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d51a25740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d911ee0090 con 0x55d911ffa980 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d51a25740 10 client.0 _ll_register_callbacks cb 0x55d911f05a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d4d1fb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d912000680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d47fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 0x55d912000bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d4d1fb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d912000680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:60066/0 (socket says 172.21.15.45:60066) 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d4d1fb640 1 -- 172.21.15.45:0/4171082962 learned_addr learned my addr 172.21.15.45:0/4171082962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/4171082962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d912001100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d4d1fb640 1 -- 172.21.15.45:0/4171082962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 msgr2=0x55d912000bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/4171082962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 0x55d912000bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d4d1fb640 1 -- 172.21.15.45:0/4171082962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 msgr2=0x55d912001100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/4171082962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d912001100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d4d1fb640 1 -- 172.21.15.45:0/4171082962 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d40019040 con 0x55d911ffa060 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/4171082962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d912000680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009560 tx=0x7f4d40009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d41f28eb431b730 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d27fff640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d27fff640 1 -- 172.21.15.45:0/4171082962 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000f850 con 0x55d911ffa060 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d27fff640 1 -- 172.21.15.45:0/4171082962 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4d4000f9b0 con 0x55d911ffa060 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.585+0000 7f4d51a25740 1 -- 172.21.15.45:0/4171082962 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d911f5b010 con 0x55d911ffa060 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.586+0000 7f4d27fff640 1 -- 172.21.15.45:0/4171082962 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d4000fb70 con 0x55d911ffa060 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.586+0000 7f4d51a25740 10 client.26270 fetch_fsmap learned FSMap version 33 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.586+0000 7f4d51a25740 1 -- 172.21.15.45:0/4171082962 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d911f606a0 con 0x55d911ffa060 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.586+0000 7f4d27fff640 1 -- 172.21.15.45:0/4171082962 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d911f5b010 con 0x55d911ffa060 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.586+0000 7f4d27fff640 1 -- 172.21.15.45:0/4171082962 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d4001e3e0 con 0x55d911ffa060 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.586+0000 7f4d51a25740 10 client.26270 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.586+0000 7f4d51a25740 10 client.26270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.586+0000 7f4d51a25740 20 client.26270 populate_metadata read hostname 'smithi045' 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.586+0000 7f4d51a25740 1 --2- 172.21.15.45:0/4171082962 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d912013420 0x55d912015810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.586+0000 7f4d51a25740 4 client.26270 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:39:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.586+0000 7f4d51a25740 1 -- 172.21.15.45:0/4171082962 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d911f5ad30 con 0x55d912013420 2024-01-31T05:39:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.586+0000 7f4d47fff640 1 --2- 172.21.15.45:0/4171082962 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d912013420 0x55d912015810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.587+0000 7f4d47fff640 1 --2- 172.21.15.45:0/4171082962 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d912013420 0x55d912015810 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1113836bec1cbd7 server_cookie=c5f49f71e65086d9 in_seq=0 out_seq=0 2024-01-31T05:39:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.587+0000 7f4d27fff640 10 client.26270 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.587+0000 7f4d27fff640 1 -- 172.21.15.45:0/4171082962 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d911f5ad30 con 0x55d912013420 2024-01-31T05:39:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.587+0000 7f4d27fff640 10 client.26270 handle_command_reply: tid=0 2024-01-31T05:39:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.587+0000 7f4d51a25740 1 client.26270 shutdown 2024-01-31T05:39:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.588+0000 7f4d51a25740 1 -- 172.21.15.45:0/4171082962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 msgr2=0x55d912000680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.588+0000 7f4d51a25740 1 --2- 172.21.15.45:0/4171082962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d912000680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009560 tx=0x7f4d40009230 comp rx=0 tx=0).stop 2024-01-31T05:39:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.588+0000 7f4d51a25740 1 -- 172.21.15.45:0/4171082962 shutdown_connections 2024-01-31T05:39:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.588+0000 7f4d51a25740 1 --2- 172.21.15.45:0/4171082962 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d912013420 0x55d912015810 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.588+0000 7f4d51a25740 1 --2- 172.21.15.45:0/4171082962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d912001100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.588+0000 7f4d51a25740 1 --2- 172.21.15.45:0/4171082962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 0x55d912000bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.588+0000 7f4d27fff640 0 client.26270 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.588+0000 7f4d51a25740 1 --2- 172.21.15.45:0/4171082962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d912000680 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.588+0000 7f4d51a25740 1 -- 172.21.15.45:0/4171082962 >> 172.21.15.45:0/4171082962 conn(0x55d911ff4980 msgr2=0x55d911ff66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.588+0000 7f4d51a25740 1 -- 172.21.15.45:0/4171082962 shutdown_connections 2024-01-31T05:39:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.588+0000 7f4d51a25740 1 -- 172.21.15.45:0/4171082962 wait complete. 2024-01-31T05:39:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.588+0000 7f4d51a25740 20 client.26270 trim_cache size 0 max 16384 2024-01-31T05:39:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.601+0000 7f4d51a25740 1 Processor -- start 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d51a25740 1 -- start start 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d51a25740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d912002db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d51a25740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 0x55d911fff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d51a25740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d911ffd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d51a25740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d912013220 con 0x55d912003750 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d51a25740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d91201aa20 con 0x55d911ffa060 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d51a25740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d912005ec0 con 0x55d911ffa980 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d4d1fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d911ffd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d4d1fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d911ffd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58324/0 (socket says 172.21.15.45:58324) 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d4d1fb640 1 -- 172.21.15.45:0/150632462 learned_addr learned my addr 172.21.15.45:0/150632462 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d47fff640 1 --2- 172.21.15.45:0/150632462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 0x55d911fff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/150632462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d912002db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d4d1fb640 1 -- 172.21.15.45:0/150632462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 msgr2=0x55d911fff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/150632462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 0x55d911fff660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d4d1fb640 1 -- 172.21.15.45:0/150632462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 msgr2=0x55d912002db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/150632462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d912002db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d4d1fb640 1 -- 172.21.15.45:0/150632462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d911efd030 con 0x55d912003750 2024-01-31T05:39:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.602+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/150632462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d911ffd8e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4d4001ec20 tx=0x7f4d400117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f71156314dc3e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.603+0000 7f4d327fc640 1 -- 172.21.15.45:0/150632462 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d40011130 con 0x55d912003750 2024-01-31T05:39:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.603+0000 7f4d327fc640 1 -- 172.21.15.45:0/150632462 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d4000fed0 con 0x55d912003750 2024-01-31T05:39:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.603+0000 7f4d327fc640 1 -- 172.21.15.45:0/150632462 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d40009b70 con 0x55d912003750 2024-01-31T05:39:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.603+0000 7f4d51a25740 1 -- 172.21.15.45:0/150632462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 msgr2=0x55d911ffd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.603+0000 7f4d51a25740 1 --2- 172.21.15.45:0/150632462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d911ffd8e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4d4001ec20 tx=0x7f4d400117a0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.603+0000 7f4d51a25740 1 -- 172.21.15.45:0/150632462 shutdown_connections 2024-01-31T05:39:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.603+0000 7f4d51a25740 1 --2- 172.21.15.45:0/150632462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d912003750 0x55d911ffd8e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.603+0000 7f4d51a25740 1 --2- 172.21.15.45:0/150632462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa980 0x55d911fff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.603+0000 7f4d51a25740 1 --2- 172.21.15.45:0/150632462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d911ffa060 0x55d912002db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.603+0000 7f4d51a25740 1 -- 172.21.15.45:0/150632462 >> 172.21.15.45:0/150632462 conn(0x55d911ff4980 msgr2=0x55d911ffef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.603+0000 7f4d51a25740 1 -- 172.21.15.45:0/150632462 shutdown_connections 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.603+0000 7f4d51a25740 1 -- 172.21.15.45:0/150632462 wait complete. 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d51a25740 1 -- start start 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d51a25740 1 Processor -- start 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d51a25740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa060 0x55d91200a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d51a25740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d911ffa980 0x55d912008a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d51a25740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d912003750 0x55d912008f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d51a25740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d912009490 con 0x55d911ffa980 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d51a25740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d912009600 con 0x55d912003750 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d51a25740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9120097c0 con 0x55d911ffa060 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d51a25740 10 client.0 _ll_register_callbacks cb 0x7f4d40009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:39:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d47fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d911ffa980 0x55d912008a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d47fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d911ffa980 0x55d912008a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58334/0 (socket says 172.21.15.45:58334) 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d47fff640 1 -- 172.21.15.45:0/2538525829 learned_addr learned my addr 172.21.15.45:0/2538525829 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa060 0x55d91200a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.604+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d912003750 0x55d912008f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d47fff640 1 -- 172.21.15.45:0/2538525829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa060 msgr2=0x55d91200a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d47fff640 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa060 0x55d91200a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d47fff640 1 -- 172.21.15.45:0/2538525829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d912003750 msgr2=0x55d912008f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d47fff640 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d912003750 0x55d912008f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d47fff640 1 -- 172.21.15.45:0/2538525829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d911f606a0 con 0x55d911ffa980 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d912003750 0x55d912008f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d47fff640 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d911ffa980 0x55d912008a10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009dd0 tx=0x7f4d34009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77cd815c4a9b7d5f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d27fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d27fff640 1 -- 172.21.15.45:0/2538525829 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34017040 con 0x55d911ffa980 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d27fff640 1 -- 172.21.15.45:0/2538525829 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4d34003ac0 con 0x55d911ffa980 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d51a25740 1 -- 172.21.15.45:0/2538525829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d40011290 con 0x55d911ffa980 2024-01-31T05:39:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d27fff640 1 -- 172.21.15.45:0/2538525829 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d34003c80 con 0x55d911ffa980 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d51a25740 10 client.16776 fetch_fsmap learned FSMap version 33 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d51a25740 1 -- 172.21.15.45:0/2538525829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d911f22a50 con 0x55d911ffa980 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.605+0000 7f4d27fff640 1 -- 172.21.15.45:0/2538525829 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d40011290 con 0x55d911ffa980 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.606+0000 7f4d27fff640 1 -- 172.21.15.45:0/2538525829 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d34010430 con 0x55d911ffa980 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.606+0000 7f4d51a25740 10 client.16776 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.606+0000 7f4d51a25740 10 client.16776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.606+0000 7f4d51a25740 20 client.16776 populate_metadata read hostname 'smithi045' 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.606+0000 7f4d51a25740 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d91206af00 0x55d91206d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.606+0000 7f4d51a25740 4 client.16776 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.606+0000 7f4d51a25740 1 -- 172.21.15.45:0/2538525829 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d911c61600 con 0x55d91206af00 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.606+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d91206af00 0x55d91206d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.606+0000 7f4d4d1fb640 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d91206af00 0x55d91206d320 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3284e4a19b69724d server_cookie=6ac7d0159d3edc7d in_seq=0 out_seq=0 2024-01-31T05:39:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.606+0000 7f4d27fff640 10 client.16776 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:47.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 96222 22.57 MB/sec execute 106 sec latency 51.870 ms 2024-01-31T05:39:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.673+0000 7f4d27fff640 1 -- 172.21.15.45:0/2538525829 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d911c61600 con 0x55d91206af00 2024-01-31T05:39:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.673+0000 7f4d27fff640 10 client.16776 handle_command_reply: tid=0 2024-01-31T05:39:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d51a25740 1 client.16776 shutdown 2024-01-31T05:39:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d51a25740 1 -- 172.21.15.45:0/2538525829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d911ffa980 msgr2=0x55d912008a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d51a25740 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d911ffa980 0x55d912008a10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009dd0 tx=0x7f4d34009c40 comp rx=0 tx=0).stop 2024-01-31T05:39:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d51a25740 1 -- 172.21.15.45:0/2538525829 shutdown_connections 2024-01-31T05:39:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d51a25740 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55d91206af00 0x55d91206d320 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d51a25740 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d911ffa980 0x55d912008a10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d27fff640 0 client.16776 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:39:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d51a25740 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d912003750 0x55d912008f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d51a25740 1 --2- 172.21.15.45:0/2538525829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d911ffa060 0x55d91200a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d51a25740 1 -- 172.21.15.45:0/2538525829 >> 172.21.15.45:0/2538525829 conn(0x55d911ff4980 msgr2=0x55d911ff3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d51a25740 1 -- 172.21.15.45:0/2538525829 shutdown_connections 2024-01-31T05:39:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d51a25740 1 -- 172.21.15.45:0/2538525829 wait complete. 2024-01-31T05:39:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d51a25740 20 client.16776 trim_cache size 0 max 16384 2024-01-31T05:39:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:39:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:39:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:39:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:39:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/2889865862 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4d380a1570 msgr2=0x7f4d380a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.674+0000 7f4d4ec5e640 1 --2- 172.21.15.45:0/2889865862 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4d380a1570 0x7f4d380a3a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0ecd60 tx=0x7f4d3c0b1ad0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.675+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/2889865862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d48100f40 msgr2=0x7f4d48197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:39:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.675+0000 7f4d4ec5e640 1 --2- 172.21.15.45:0/2889865862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d48100f40 0x7f4d48197230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c005940 tx=0x7f4d3c005d50 comp rx=0 tx=0).stop 2024-01-31T05:39:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.675+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/2889865862 shutdown_connections 2024-01-31T05:39:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.675+0000 7f4d4ec5e640 1 --2- 172.21.15.45:0/2889865862 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f4d380a1570 0x7f4d380a3a30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.675+0000 7f4d4ec5e640 1 --2- 172.21.15.45:0/2889865862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d4819baa0 0x7f4d4819bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.675+0000 7f4d4ec5e640 1 --2- 172.21.15.45:0/2889865862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d48106010 0x7f4d48197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.675+0000 7f4d4ec5e640 1 --2- 172.21.15.45:0/2889865862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d48100f40 0x7f4d48197230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:39:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.675+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/2889865862 >> 172.21.15.45:0/2889865862 conn(0x7f4d480fc7a0 msgr2=0x7f4d480fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:39:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.675+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/2889865862 shutdown_connections 2024-01-31T05:39:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:39:47.675+0000 7f4d4ec5e640 1 -- 172.21.15.45:0/2889865862 wait complete. 2024-01-31T05:39:48.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 96633 22.54 MB/sec execute 107 sec latency 254.223 ms 2024-01-31T05:39:49.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:48 smithi045 ceph-mon[31585]: pgmap v544: 97 pgs: 97 active+clean; 1021 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 615 op/s 2024-01-31T05:39:49.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:48 smithi156 ceph-mon[35720]: pgmap v544: 97 pgs: 97 active+clean; 1021 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 615 op/s 2024-01-31T05:39:49.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:48 smithi186 ceph-mon[35659]: pgmap v544: 97 pgs: 97 active+clean; 1021 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 615 op/s 2024-01-31T05:39:49.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 97136 22.64 MB/sec execute 108 sec latency 163.016 ms 2024-01-31T05:39:50.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 97750 22.66 MB/sec execute 109 sec latency 187.321 ms 2024-01-31T05:39:51.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:50 smithi045 ceph-mon[31585]: pgmap v545: 97 pgs: 97 active+clean; 1021 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.5 MiB/s wr, 455 op/s 2024-01-31T05:39:51.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:50 smithi156 ceph-mon[35720]: pgmap v545: 97 pgs: 97 active+clean; 1021 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.5 MiB/s wr, 455 op/s 2024-01-31T05:39:51.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:50 smithi186 ceph-mon[35659]: pgmap v545: 97 pgs: 97 active+clean; 1021 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.5 MiB/s wr, 455 op/s 2024-01-31T05:39:51.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 98249 22.76 MB/sec execute 110 sec latency 182.155 ms 2024-01-31T05:39:52.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 98729 22.80 MB/sec execute 111 sec latency 234.314 ms 2024-01-31T05:39:53.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:52 smithi045 ceph-mon[31585]: pgmap v546: 97 pgs: 97 active+clean; 1021 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.5 MiB/s wr, 455 op/s 2024-01-31T05:39:53.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:52 smithi156 ceph-mon[35720]: pgmap v546: 97 pgs: 97 active+clean; 1021 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.5 MiB/s wr, 455 op/s 2024-01-31T05:39:53.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:52 smithi186 ceph-mon[35659]: pgmap v546: 97 pgs: 97 active+clean; 1021 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.5 MiB/s wr, 455 op/s 2024-01-31T05:39:53.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 99025 22.78 MB/sec execute 112 sec latency 402.507 ms 2024-01-31T05:39:54.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 99429 22.76 MB/sec execute 113 sec latency 226.529 ms 2024-01-31T05:39:55.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:54 smithi045 ceph-mon[31585]: pgmap v547: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 656 op/s 2024-01-31T05:39:55.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:54 smithi156 ceph-mon[35720]: pgmap v547: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 656 op/s 2024-01-31T05:39:55.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:54 smithi186 ceph-mon[35659]: pgmap v547: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 656 op/s 2024-01-31T05:39:55.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 99789 22.72 MB/sec execute 114 sec latency 54.069 ms 2024-01-31T05:39:56.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 100228 22.66 MB/sec execute 115 sec latency 54.287 ms 2024-01-31T05:39:57.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:56 smithi045 ceph-mon[31585]: pgmap v548: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.9 MiB/s wr, 388 op/s 2024-01-31T05:39:57.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:56 smithi156 ceph-mon[35720]: pgmap v548: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.9 MiB/s wr, 388 op/s 2024-01-31T05:39:57.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:56 smithi186 ceph-mon[35659]: pgmap v548: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.9 MiB/s wr, 388 op/s 2024-01-31T05:39:57.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 100677 22.70 MB/sec execute 116 sec latency 167.452 ms 2024-01-31T05:39:58.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 101006 22.63 MB/sec execute 117 sec latency 378.679 ms 2024-01-31T05:39:59.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:39:58 smithi045 ceph-mon[31585]: pgmap v549: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 573 op/s 2024-01-31T05:39:59.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:39:58 smithi156 ceph-mon[35720]: pgmap v549: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 573 op/s 2024-01-31T05:39:59.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:39:58 smithi186 ceph-mon[35659]: pgmap v549: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 573 op/s 2024-01-31T05:39:59.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 101491 22.69 MB/sec execute 118 sec latency 199.159 ms 2024-01-31T05:40:00.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 101971 22.67 MB/sec execute 119 sec latency 192.390 ms 2024-01-31T05:40:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 -- 172.21.15.45:0/1329715185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39001081f0 msgr2=0x7f390010fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 --2- 172.21.15.45:0/1329715185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39001081f0 0x7f390010fdc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009930 tx=0x7f38f802bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 -- 172.21.15.45:0/1329715185 shutdown_connections 2024-01-31T05:40:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 --2- 172.21.15.45:0/1329715185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39001081f0 0x7f390010fdc0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 --2- 172.21.15.45:0/1329715185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3900103ad0 0x7f3900107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 --2- 172.21.15.45:0/1329715185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3900103120 0x7f3900103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 -- 172.21.15.45:0/1329715185 >> 172.21.15.45:0/1329715185 conn(0x7f39000fe960 msgr2=0x7f3900100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 -- 172.21.15.45:0/1329715185 shutdown_connections 2024-01-31T05:40:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 -- 172.21.15.45:0/1329715185 wait complete. 2024-01-31T05:40:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 Processor -- start 2024-01-31T05:40:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 -- start start 2024-01-31T05:40:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3900103120 0x7f39001993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.774+0000 7f3906f7c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39001081f0 0x7f3900199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f3906f7c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f390019dca0 0x7f390019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f3906f7c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390019e690 con 0x7f39001081f0 2024-01-31T05:40:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f3906f7c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390019e800 con 0x7f3900103120 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f3906f7c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390019e970 con 0x7f390019dca0 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f38fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39001081f0 0x7f3900199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f38fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39001081f0 0x7f3900199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42402/0 (socket says 172.21.15.45:42402) 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f38fffff640 1 -- 172.21.15.45:0/1122569692 learned_addr learned my addr 172.21.15.45:0/1122569692 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f3904d18640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3900103120 0x7f39001993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f3905519640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f390019dca0 0x7f390019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f38fffff640 1 -- 172.21.15.45:0/1122569692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f390019dca0 msgr2=0x7f390019e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f38fffff640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f390019dca0 0x7f390019e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f38fffff640 1 -- 172.21.15.45:0/1122569692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3900103120 msgr2=0x7f39001993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f38fffff640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3900103120 0x7f39001993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f38fffff640 1 -- 172.21.15.45:0/1122569692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f0009840 con 0x7f39001081f0 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f38fffff640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39001081f0 0x7f3900199920 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f38f000efd0 tx=0x7f38f000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f38fdffb640 1 -- 172.21.15.45:0/1122569692 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0009b20 con 0x7f39001081f0 2024-01-31T05:40:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f38fdffb640 1 -- 172.21.15.45:0/1122569692 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38f0003b70 con 0x7f39001081f0 2024-01-31T05:40:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f3906f7c640 1 -- 172.21.15.45:0/1122569692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f8009560 con 0x7f39001081f0 2024-01-31T05:40:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.775+0000 7f38fdffb640 1 -- 172.21.15.45:0/1122569692 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0018930 con 0x7f39001081f0 2024-01-31T05:40:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.776+0000 7f3906f7c640 1 -- 172.21.15.45:0/1122569692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39001a6260 con 0x7f39001081f0 2024-01-31T05:40:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.777+0000 7f38fdffb640 1 -- 172.21.15.45:0/1122569692 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f38f0003ce0 con 0x7f39001081f0 2024-01-31T05:40:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f38fdffb640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f38ec0a1570 0x7f38ec0a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f3909e1c740 1 Processor -- start 2024-01-31T05:40:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f3904d18640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f38ec0a1570 0x7f38ec0a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f3909e1c740 1 -- start start 2024-01-31T05:40:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f38fdffb640 1 -- 172.21.15.45:0/1122569692 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f38f0014070 con 0x7f39001081f0 2024-01-31T05:40:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f3909e1c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda454d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f3909e1c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda4556210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f3909e1c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda4558b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f3909e1c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda44bc040 con 0x55bda454d980 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f3909e1c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda44b9730 con 0x55bda4556750 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f3909e1c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda44b5020 con 0x55bda454d060 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f38fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda4556210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f38fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda4556210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42408/0 (socket says 172.21.15.45:42408) 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f38fffff640 1 -- 172.21.15.45:0/2912890511 learned_addr learned my addr 172.21.15.45:0/2912890511 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f3904d18640 1 --2- 172.21.15.45:0/2912890511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda454d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.778+0000 7f3905519640 1 --2- 172.21.15.45:0/2912890511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda4558b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.779+0000 7f3904d18640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f38ec0a1570 0x7f38ec0a3a30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009c20 tx=0x7f38e8011810 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.779+0000 7f38fffff640 1 -- 172.21.15.45:0/2912890511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 msgr2=0x55bda454d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.779+0000 7f38fffff640 1 --2- 172.21.15.45:0/2912890511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda454d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.779+0000 7f38fffff640 1 -- 172.21.15.45:0/2912890511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 msgr2=0x55bda4558b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.779+0000 7f38fffff640 1 --2- 172.21.15.45:0/2912890511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda4558b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.779+0000 7f38fffff640 1 -- 172.21.15.45:0/2912890511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bda44ace40 con 0x55bda454d980 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.779+0000 7f3905519640 1 --2- 172.21.15.45:0/2912890511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda4558b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:40:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.779+0000 7f38fffff640 1 --2- 172.21.15.45:0/2912890511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda4556210 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f38f00a2410 tx=0x7f38f0005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a210112e117234b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.779+0000 7f38f6ffd640 1 -- 172.21.15.45:0/2912890511 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f009a5b0 con 0x55bda454d980 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.779+0000 7f38f6ffd640 1 -- 172.21.15.45:0/2912890511 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38f009e700 con 0x55bda454d980 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.779+0000 7f38f6ffd640 1 -- 172.21.15.45:0/2912890511 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0099aa0 con 0x55bda454d980 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.780+0000 7f3909e1c740 1 -- 172.21.15.45:0/2912890511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 msgr2=0x55bda4556210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.780+0000 7f3909e1c740 1 --2- 172.21.15.45:0/2912890511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda4556210 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f38f00a2410 tx=0x7f38f0005a50 comp rx=0 tx=0).stop 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.780+0000 7f3909e1c740 1 -- 172.21.15.45:0/2912890511 shutdown_connections 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.780+0000 7f3909e1c740 1 --2- 172.21.15.45:0/2912890511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda4556210 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.780+0000 7f3909e1c740 1 --2- 172.21.15.45:0/2912890511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda4558b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.780+0000 7f3909e1c740 1 --2- 172.21.15.45:0/2912890511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda454d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.780+0000 7f3909e1c740 1 -- 172.21.15.45:0/2912890511 >> 172.21.15.45:0/2912890511 conn(0x55bda4547980 msgr2=0x55bda4547d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.780+0000 7f3909e1c740 1 -- 172.21.15.45:0/2912890511 shutdown_connections 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.780+0000 7f3909e1c740 1 -- 172.21.15.45:0/2912890511 wait complete. 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.780+0000 7f3909e1c740 1 -- start start 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.780+0000 7f3909e1c740 1 Processor -- start 2024-01-31T05:40:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f3909e1c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda4553780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f3909e1c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda4553cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f3909e1c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda4554200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f3909e1c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda446e1b0 con 0x55bda454d980 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f3909e1c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda446de50 con 0x55bda4556750 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f3909e1c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda4433090 con 0x55bda454d060 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f3909e1c740 10 client.0 _ll_register_callbacks cb 0x55bda4458a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f38fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda4553cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f3904d18640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda4553780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f38fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda4553cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42410/0 (socket says 172.21.15.45:42410) 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f38fffff640 1 -- 172.21.15.45:0/1227929723 learned_addr learned my addr 172.21.15.45:0/1227929723 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f3905519640 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda4554200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f38fffff640 1 -- 172.21.15.45:0/1227929723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 msgr2=0x55bda4553780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f38fffff640 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda4553780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f38fffff640 1 -- 172.21.15.45:0/1227929723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 msgr2=0x55bda4554200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f38fffff640 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda4554200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f38fffff640 1 -- 172.21.15.45:0/1227929723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39001a6260 con 0x55bda454d980 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.781+0000 7f3904d18640 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda4553780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.782+0000 7f38fffff640 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda4553cc0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f38f0096ab0 tx=0x7f38f009a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdda348e29fb3eb0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.782+0000 7f38dffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.782+0000 7f38dffff640 1 -- 172.21.15.45:0/1227929723 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0099cb0 con 0x55bda454d980 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.782+0000 7f38dffff640 1 -- 172.21.15.45:0/1227929723 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f38f00a24b0 con 0x55bda454d980 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.782+0000 7f3909e1c740 1 -- 172.21.15.45:0/1227929723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bda44ae010 con 0x55bda454d980 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.782+0000 7f38dffff640 1 -- 172.21.15.45:0/1227929723 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38f00a2670 con 0x55bda454d980 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.782+0000 7f3909e1c740 10 client.16788 fetch_fsmap learned FSMap version 33 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.782+0000 7f3909e1c740 1 -- 172.21.15.45:0/1227929723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55bda44b36a0 con 0x55bda454d980 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.782+0000 7f38dffff640 1 -- 172.21.15.45:0/1227929723 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38f007e070 con 0x55bda454d980 2024-01-31T05:40:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.783+0000 7f38dffff640 1 -- 172.21.15.45:0/1227929723 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f38f009e200 con 0x55bda454d980 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.783+0000 7f3909e1c740 10 client.16788 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.783+0000 7f3909e1c740 10 client.16788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.783+0000 7f3909e1c740 20 client.16788 populate_metadata read hostname 'smithi045' 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.783+0000 7f3909e1c740 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55bda4566220 0x55bda4568640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.783+0000 7f3909e1c740 4 client.16788 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.783+0000 7f3909e1c740 1 -- 172.21.15.45:0/1227929723 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bda44add30 con 0x55bda4566220 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.783+0000 7f3904d18640 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55bda4566220 0x55bda4568640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.783+0000 7f38dffff640 10 client.16788 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.783+0000 7f3904d18640 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55bda4566220 0x55bda4568640 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eccca49e2b6834b4 server_cookie=55069a44f4db44c2 in_seq=0 out_seq=0 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f38dffff640 1 -- 172.21.15.45:0/1227929723 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bda44add30 con 0x55bda4566220 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f38dffff640 10 client.16788 handle_command_reply: tid=0 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f3909e1c740 1 client.16788 shutdown 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f3909e1c740 1 -- 172.21.15.45:0/1227929723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 msgr2=0x55bda4553cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f3909e1c740 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda4553cc0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f38f0096ab0 tx=0x7f38f009a410 comp rx=0 tx=0).stop 2024-01-31T05:40:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f3909e1c740 1 -- 172.21.15.45:0/1227929723 shutdown_connections 2024-01-31T05:40:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f3909e1c740 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55bda4566220 0x55bda4568640 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f3909e1c740 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda4553cc0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f3909e1c740 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda4554200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f38dffff640 0 client.16788 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:40:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f3909e1c740 1 --2- 172.21.15.45:0/1227929723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda4553780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f3909e1c740 1 -- 172.21.15.45:0/1227929723 >> 172.21.15.45:0/1227929723 conn(0x55bda4547980 msgr2=0x55bda45496d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.784+0000 7f3909e1c740 1 -- 172.21.15.45:0/1227929723 shutdown_connections 2024-01-31T05:40:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.785+0000 7f3909e1c740 1 -- 172.21.15.45:0/1227929723 wait complete. 2024-01-31T05:40:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.785+0000 7f3909e1c740 20 client.16788 trim_cache size 0 max 16384 2024-01-31T05:40:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.800+0000 7f3909e1c740 1 Processor -- start 2024-01-31T05:40:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.800+0000 7f3909e1c740 1 -- start start 2024-01-31T05:40:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.800+0000 7f3909e1c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d060 0x55bda4555eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.800+0000 7f3909e1c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda454d980 0x55bda45509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.800+0000 7f3909e1c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda4556750 0x55bda4550f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.800+0000 7f3909e1c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda4541f10 con 0x55bda454d060 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.800+0000 7f3909e1c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda454c420 con 0x55bda454d980 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.800+0000 7f3909e1c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda4559200 con 0x55bda4556750 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f3904d18640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d060 0x55bda4555eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f3904d18640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d060 0x55bda4555eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42416/0 (socket says 172.21.15.45:42416) 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f3904d18640 1 -- 172.21.15.45:0/3679620756 learned_addr learned my addr 172.21.15.45:0/3679620756 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f38fffff640 1 --2- 172.21.15.45:0/3679620756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda454d980 0x55bda45509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f3905519640 1 --2- 172.21.15.45:0/3679620756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda4556750 0x55bda4550f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f3904d18640 1 -- 172.21.15.45:0/3679620756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda4556750 msgr2=0x55bda4550f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f3904d18640 1 --2- 172.21.15.45:0/3679620756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda4556750 0x55bda4550f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f3904d18640 1 -- 172.21.15.45:0/3679620756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda454d980 msgr2=0x55bda45509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f3904d18640 1 --2- 172.21.15.45:0/3679620756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda454d980 0x55bda45509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f3904d18640 1 -- 172.21.15.45:0/3679620756 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bda4450030 con 0x55bda454d060 2024-01-31T05:40:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f38fffff640 1 --2- 172.21.15.45:0/3679620756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda454d980 0x55bda45509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:40:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f3905519640 1 --2- 172.21.15.45:0/3679620756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda4556750 0x55bda4550f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:40:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f3904d18640 1 --2- 172.21.15.45:0/3679620756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d060 0x55bda4555eb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f38e8011d10 tx=0x7f38e801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d7d0eb5e4eb7f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f38f67fc640 1 -- 172.21.15.45:0/3679620756 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8023040 con 0x55bda454d060 2024-01-31T05:40:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f38f67fc640 1 -- 172.21.15.45:0/3679620756 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e80298e0 con 0x55bda454d060 2024-01-31T05:40:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.801+0000 7f38f67fc640 1 -- 172.21.15.45:0/3679620756 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e800fc50 con 0x55bda454d060 2024-01-31T05:40:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.802+0000 7f3909e1c740 1 -- 172.21.15.45:0/3679620756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d060 msgr2=0x55bda4555eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.802+0000 7f3909e1c740 1 --2- 172.21.15.45:0/3679620756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d060 0x55bda4555eb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f38e8011d10 tx=0x7f38e801d000 comp rx=0 tx=0).stop 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.802+0000 7f3909e1c740 1 -- 172.21.15.45:0/3679620756 shutdown_connections 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.802+0000 7f3909e1c740 1 --2- 172.21.15.45:0/3679620756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda4556750 0x55bda4550f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.802+0000 7f3909e1c740 1 --2- 172.21.15.45:0/3679620756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda454d980 0x55bda45509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.802+0000 7f3909e1c740 1 --2- 172.21.15.45:0/3679620756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d060 0x55bda4555eb0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.802+0000 7f3909e1c740 1 -- 172.21.15.45:0/3679620756 >> 172.21.15.45:0/3679620756 conn(0x55bda4547980 msgr2=0x55bda4552050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f3909e1c740 1 -- 172.21.15.45:0/3679620756 shutdown_connections 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f3909e1c740 1 -- 172.21.15.45:0/3679620756 wait complete. 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f3909e1c740 1 -- start start 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f3909e1c740 1 Processor -- start 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f3909e1c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda455c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f3909e1c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda455cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f3909e1c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda45bc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f3909e1c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda455d3a0 con 0x55bda454d980 2024-01-31T05:40:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f3909e1c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda45bccc0 con 0x55bda4556750 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f3909e1c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda45bce80 con 0x55bda454d060 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f3909e1c740 10 client.0 _ll_register_callbacks cb 0x7f38f0096230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f38fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda455cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f38fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda455cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42420/0 (socket says 172.21.15.45:42420) 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f38fffff640 1 -- 172.21.15.45:0/771547527 learned_addr learned my addr 172.21.15.45:0/771547527 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.803+0000 7f3905519640 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda45bc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f3904d18640 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda455c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f38fffff640 1 -- 172.21.15.45:0/771547527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 msgr2=0x55bda455c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f38fffff640 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda455c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f38fffff640 1 -- 172.21.15.45:0/771547527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 msgr2=0x55bda45bc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f38fffff640 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda45bc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f38fffff640 1 -- 172.21.15.45:0/771547527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bda44b36a0 con 0x55bda454d980 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f38fffff640 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda455cc20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f38f009a2b0 tx=0x7f38f0005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c784c978a3b19 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f38dffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:40:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f38dffff640 1 -- 172.21.15.45:0/771547527 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00a2940 con 0x55bda454d980 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f38dffff640 1 -- 172.21.15.45:0/771547527 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f38f00a24b0 con 0x55bda454d980 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f3909e1c740 1 -- 172.21.15.45:0/771547527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38f009e700 con 0x55bda454d980 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f38dffff640 1 -- 172.21.15.45:0/771547527 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38f00a2670 con 0x55bda454d980 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f3909e1c740 10 client.16794 fetch_fsmap learned FSMap version 33 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.804+0000 7f3909e1c740 1 -- 172.21.15.45:0/771547527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f38e801e760 con 0x55bda454d980 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.805+0000 7f38dffff640 1 -- 172.21.15.45:0/771547527 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38f009e700 con 0x55bda454d980 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.805+0000 7f38dffff640 1 -- 172.21.15.45:0/771547527 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f38f00a4cd0 con 0x55bda454d980 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.805+0000 7f3909e1c740 10 client.16794 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.805+0000 7f3909e1c740 10 client.16794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.805+0000 7f3909e1c740 20 client.16794 populate_metadata read hostname 'smithi045' 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.805+0000 7f3909e1c740 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55bda4566220 0x55bda45c0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.805+0000 7f3909e1c740 4 client.16794 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.805+0000 7f3909e1c740 1 -- 172.21.15.45:0/771547527 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bda41b4600 con 0x55bda4566220 2024-01-31T05:40:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.805+0000 7f3904d18640 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55bda4566220 0x55bda45c0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.805+0000 7f3904d18640 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55bda4566220 0x55bda45c0290 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8aaff684223ccdb server_cookie=7686a16d278773fa in_seq=0 out_seq=0 2024-01-31T05:40:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:00.805+0000 7f38dffff640 10 client.16794 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:40:01.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:00 smithi045 ceph-mon[31585]: pgmap v550: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.7 MiB/s wr, 385 op/s 2024-01-31T05:40:01.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:00 smithi045 ceph-mon[31585]: overall HEALTH_OK 2024-01-31T05:40:01.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:00 smithi186 ceph-mon[35659]: pgmap v550: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.7 MiB/s wr, 385 op/s 2024-01-31T05:40:01.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:00 smithi186 ceph-mon[35659]: overall HEALTH_OK 2024-01-31T05:40:01.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:00 smithi156 ceph-mon[35720]: pgmap v550: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.7 MiB/s wr, 385 op/s 2024-01-31T05:40:01.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:00 smithi156 ceph-mon[35720]: overall HEALTH_OK 2024-01-31T05:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f38dffff640 1 -- 172.21.15.45:0/771547527 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bda41b4600 con 0x55bda4566220 2024-01-31T05:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f38dffff640 10 client.16794 handle_command_reply: tid=0 2024-01-31T05:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f3909e1c740 1 client.16794 shutdown 2024-01-31T05:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f3909e1c740 1 -- 172.21.15.45:0/771547527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 msgr2=0x55bda455cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f3909e1c740 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda455cc20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f38f009a2b0 tx=0x7f38f0005b80 comp rx=0 tx=0).stop 2024-01-31T05:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f3909e1c740 1 -- 172.21.15.45:0/771547527 shutdown_connections 2024-01-31T05:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f3909e1c740 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55bda4566220 0x55bda45c0290 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f3909e1c740 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bda454d980 0x55bda455cc20 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f38dffff640 0 client.16794 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:40:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f3909e1c740 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bda4556750 0x55bda45bc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f3909e1c740 1 --2- 172.21.15.45:0/771547527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bda454d060 0x55bda455c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f3909e1c740 1 -- 172.21.15.45:0/771547527 >> 172.21.15.45:0/771547527 conn(0x55bda4547980 msgr2=0x55bda4553740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f3909e1c740 1 -- 172.21.15.45:0/771547527 shutdown_connections 2024-01-31T05:40:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f3909e1c740 1 -- 172.21.15.45:0/771547527 wait complete. 2024-01-31T05:40:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.410+0000 7f3909e1c740 20 client.16794 trim_cache size 0 max 16384 2024-01-31T05:40:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:40:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:40:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:40:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.411+0000 7f3906f7c640 1 -- 172.21.15.45:0/1122569692 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f38ec0a1570 msgr2=0x7f38ec0a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.411+0000 7f3906f7c640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f38ec0a1570 0x7f38ec0a3a30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009c20 tx=0x7f38e8011810 comp rx=0 tx=0).stop 2024-01-31T05:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.411+0000 7f3906f7c640 1 -- 172.21.15.45:0/1122569692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39001081f0 msgr2=0x7f3900199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.411+0000 7f3906f7c640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39001081f0 0x7f3900199920 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f38f000efd0 tx=0x7f38f000c6a0 comp rx=0 tx=0).stop 2024-01-31T05:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.411+0000 7f3906f7c640 1 -- 172.21.15.45:0/1122569692 shutdown_connections 2024-01-31T05:40:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.411+0000 7f3906f7c640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f38ec0a1570 0x7f38ec0a3a30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.411+0000 7f3906f7c640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39001081f0 0x7f3900199920 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.411+0000 7f3906f7c640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f390019dca0 0x7f390019e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.411+0000 7f3906f7c640 1 --2- 172.21.15.45:0/1122569692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3900103120 0x7f39001993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.411+0000 7f3906f7c640 1 -- 172.21.15.45:0/1122569692 >> 172.21.15.45:0/1122569692 conn(0x7f39000fe960 msgr2=0x7f3900100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.411+0000 7f3906f7c640 1 -- 172.21.15.45:0/1122569692 shutdown_connections 2024-01-31T05:40:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:01.411+0000 7f3906f7c640 1 -- 172.21.15.45:0/1122569692 wait complete. 2024-01-31T05:40:01.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 102382 22.64 MB/sec execute 120 sec latency 150.374 ms 2024-01-31T05:40:02.165 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: *** Caught signal (Segmentation fault) ** 2024-01-31T05:40:02.165 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: in thread 7f40beef5700 thread_name:md_log_replay 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f40cfab5d20] 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 2: /usr/bin/ceph-mds(+0x5040a2) [0x55ddfb9610a2] 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ddfb9632f3] 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 4: (MDLog::_replay_thread()+0x647) [0x55ddfb9bf697] 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ddfb9c8b31] 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f40cfaab1ca] 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 7: clone() 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 2024-01-31T05:40:01.869+0000 7f40beef5700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: in thread 7f40beef5700 thread_name:md_log_replay 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f40cfab5d20] 2024-01-31T05:40:02.166 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 2: /usr/bin/ceph-mds(+0x5040a2) [0x55ddfb9610a2] 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ddfb9632f3] 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 4: (MDLog::_replay_thread()+0x647) [0x55ddfb9bf697] 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ddfb9c8b31] 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f40cfaab1ca] 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 7: clone() 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 0> 2024-01-31T05:40:01.869+0000 7f40beef5700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: in thread 7f40beef5700 thread_name:md_log_replay 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f40cfab5d20] 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 2: /usr/bin/ceph-mds(+0x5040a2) [0x55ddfb9610a2] 2024-01-31T05:40:02.167 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ddfb9632f3] 2024-01-31T05:40:02.168 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 4: (MDLog::_replay_thread()+0x647) [0x55ddfb9bf697] 2024-01-31T05:40:02.168 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ddfb9c8b31] 2024-01-31T05:40:02.168 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f40cfaab1ca] 2024-01-31T05:40:02.168 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 7: clone() 2024-01-31T05:40:02.168 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:40:02.168 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 2024-01-31T05:40:02.168 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 0> 2024-01-31T05:40:01.869+0000 7f40beef5700 -1 *** Caught signal (Segmentation fault) ** 2024-01-31T05:40:02.168 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: in thread 7f40beef5700 thread_name:md_log_replay 2024-01-31T05:40:02.168 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 2024-01-31T05:40:02.168 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-31T05:40:02.168 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f40cfab5d20] 2024-01-31T05:40:02.169 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 2: /usr/bin/ceph-mds(+0x5040a2) [0x55ddfb9610a2] 2024-01-31T05:40:02.169 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ddfb9632f3] 2024-01-31T05:40:02.169 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 4: (MDLog::_replay_thread()+0x647) [0x55ddfb9bf697] 2024-01-31T05:40:02.169 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ddfb9c8b31] 2024-01-31T05:40:02.169 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f40cfaab1ca] 2024-01-31T05:40:02.169 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 7: clone() 2024-01-31T05:40:02.169 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-31T05:40:02.169 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h[72779]: 2024-01-31T05:40:02.169 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:01 smithi156 podman[74085]: 2024-01-31 05:40:01.948309343 +0000 UTC m=+0.025076841 container died e5de3b395006d3a1707ed5c1f90e5629996b111d9c7f624d7c8e9ca59c80b004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-01-31T05:40:02.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 102820 22.73 MB/sec execute 121 sec latency 167.879 ms 2024-01-31T05:40:02.953 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:02 smithi156 ceph-mon[35720]: pgmap v551: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.7 MiB/s wr, 385 op/s 2024-01-31T05:40:02.954 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:02 smithi156 podman[74085]: 2024-01-31 05:40:02.700828291 +0000 UTC m=+0.777595768 container remove e5de3b395006d3a1707ed5c1f90e5629996b111d9c7f624d7c8e9ca59c80b004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-h, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3) 2024-01-31T05:40:02.954 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:02 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-01-31T05:40:03.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:02 smithi045 ceph-mon[31585]: pgmap v551: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.7 MiB/s wr, 385 op/s 2024-01-31T05:40:03.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:02 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-01-31T05:40:03.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:02 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Consumed 2.090s CPU time. 2024-01-31T05:40:03.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:02 smithi186 ceph-mon[35659]: pgmap v551: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.7 MiB/s wr, 385 op/s 2024-01-31T05:40:03.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 103130 22.71 MB/sec execute 122 sec latency 288.362 ms 2024-01-31T05:40:04.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 103455 22.67 MB/sec execute 123 sec latency 315.501 ms 2024-01-31T05:40:05.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:04 smithi045 ceph-mon[31585]: pgmap v552: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 728 op/s 2024-01-31T05:40:05.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:04 smithi156 ceph-mon[35720]: pgmap v552: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 728 op/s 2024-01-31T05:40:05.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:04 smithi186 ceph-mon[35659]: pgmap v552: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 728 op/s 2024-01-31T05:40:05.652 INFO:tasks.workunit.client.0.smithi045.stdout: 10 103795 22.63 MB/sec execute 124 sec latency 245.524 ms 2024-01-31T05:40:06.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:05 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:40:06.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:05 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:40:06.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:05 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:40:06.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:05 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:40:06.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:05 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:40:06.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:05 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:40:06.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:05 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:40:06.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:05 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:40:06.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:05 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-31T05:40:06.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:05 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-31T05:40:06.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:05 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-31T05:40:06.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:05 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' 2024-01-31T05:40:06.651 INFO:tasks.workunit.client.0.smithi045.stdout: 10 104023 22.52 MB/sec execute 125 sec latency 1156.446 ms 2024-01-31T05:40:07.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:06 smithi045 ceph-mon[31585]: pgmap v553: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.6 MiB/s wr, 528 op/s 2024-01-31T05:40:07.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:06 smithi045 ceph-mon[31585]: osdmap e118: 12 total, 12 up, 12 in 2024-01-31T05:40:07.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:06 smithi156 ceph-mon[35720]: pgmap v553: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.6 MiB/s wr, 528 op/s 2024-01-31T05:40:07.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:06 smithi156 ceph-mon[35720]: osdmap e118: 12 total, 12 up, 12 in 2024-01-31T05:40:07.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:06 smithi186 ceph-mon[35659]: pgmap v553: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.6 MiB/s wr, 528 op/s 2024-01-31T05:40:07.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:06 smithi186 ceph-mon[35659]: osdmap e118: 12 total, 12 up, 12 in 2024-01-31T05:40:07.652 INFO:tasks.workunit.client.0.smithi045.stdout: 10 104457 22.47 MB/sec execute 126 sec latency 2156.484 ms 2024-01-31T05:40:08.652 INFO:tasks.workunit.client.0.smithi045.stdout: 10 104876 22.51 MB/sec execute 127 sec latency 3156.533 ms 2024-01-31T05:40:09.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:08 smithi045 ceph-mon[31585]: pgmap v555: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.6 MiB/s wr, 631 op/s 2024-01-31T05:40:09.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:08 smithi156 ceph-mon[35720]: pgmap v555: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.6 MiB/s wr, 631 op/s 2024-01-31T05:40:09.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:08 smithi186 ceph-mon[35659]: pgmap v555: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.6 MiB/s wr, 631 op/s 2024-01-31T05:40:09.505 INFO:tasks.workunit.client.0.smithi045.stdout:[86701] write failed on handle 11412 (Resource temporarily unavailable) 2024-01-31T05:40:09.505 INFO:tasks.workunit.client.0.smithi045.stdout:Child failed with status 1 2024-01-31T05:40:09.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:40:11.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:10 smithi045 ceph-mon[31585]: pgmap v556: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.6 MiB/s wr, 631 op/s 2024-01-31T05:40:11.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:10 smithi156 ceph-mon[35720]: pgmap v556: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.6 MiB/s wr, 631 op/s 2024-01-31T05:40:11.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:10 smithi186 ceph-mon[35659]: pgmap v556: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.6 MiB/s wr, 631 op/s 2024-01-31T05:40:13.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:12 smithi045 ceph-mon[31585]: pgmap v557: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.6 MiB/s wr, 631 op/s 2024-01-31T05:40:13.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:12 smithi045 ceph-mon[31585]: osdmap e119: 12 total, 12 up, 12 in 2024-01-31T05:40:13.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:12 smithi156 ceph-mon[35720]: pgmap v557: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.6 MiB/s wr, 631 op/s 2024-01-31T05:40:13.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:12 smithi156 ceph-mon[35720]: osdmap e119: 12 total, 12 up, 12 in 2024-01-31T05:40:13.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:12 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 6. 2024-01-31T05:40:13.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:12 smithi156 systemd[1]: Stopped Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:40:13.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:12 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Consumed 2.090s CPU time. 2024-01-31T05:40:13.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:12 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Start request repeated too quickly. 2024-01-31T05:40:13.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:12 smithi156 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-01-31T05:40:13.228 INFO:journalctl@ceph.mds.h.smithi156.stdout:Jan 31 05:40:12 smithi156 systemd[1]: Failed to start Ceph mds.h for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:40:13.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:12 smithi186 ceph-mon[35659]: pgmap v557: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.6 MiB/s wr, 631 op/s 2024-01-31T05:40:13.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:12 smithi186 ceph-mon[35659]: osdmap e119: 12 total, 12 up, 12 in 2024-01-31T05:40:14.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.513+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/571770332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec105fe0 msgr2=0x7f0cec10dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:14.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.513+0000 7f0cf1a5e640 1 --2- 172.21.15.45:0/571770332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec105fe0 0x7f0cec10dbb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009cd0 tx=0x7f0ce402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.513+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/571770332 shutdown_connections 2024-01-31T05:40:14.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.513+0000 7f0cf1a5e640 1 --2- 172.21.15.45:0/571770332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec105fe0 0x7f0cec10dbb0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.513+0000 7f0cf1a5e640 1 --2- 172.21.15.45:0/571770332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cec1018c0 0x7f0cec105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.513+0000 7f0cf1a5e640 1 --2- 172.21.15.45:0/571770332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cec100f10 0x7f0cec1012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.513+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/571770332 >> 172.21.15.45:0/571770332 conn(0x7f0cec0fc7b0 msgr2=0x7f0cec0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:14.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.514+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/571770332 shutdown_connections 2024-01-31T05:40:14.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.514+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/571770332 wait complete. 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.514+0000 7f0cf1a5e640 1 Processor -- start 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.514+0000 7f0cf1a5e640 1 -- start start 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.514+0000 7f0cf1a5e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cec100f10 0x7f0cec1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.514+0000 7f0cf1a5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec105fe0 0x7f0cec1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.514+0000 7f0cf1a5e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cec19ba60 0x7f0cec19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.514+0000 7f0cf1a5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec19c450 con 0x7f0cec105fe0 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.514+0000 7f0cf1a5e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec19c5c0 con 0x7f0cec19ba60 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.514+0000 7f0cf1a5e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec19c730 con 0x7f0cec100f10 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.514+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec105fe0 0x7f0cec1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec105fe0 0x7f0cec1976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55090/0 (socket says 172.21.15.45:55090) 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0ce3fff640 1 -- 172.21.15.45:0/379709038 learned_addr learned my addr 172.21.15.45:0/379709038 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0ceb7fe640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cec19ba60 0x7f0cec19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0ceaffd640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cec100f10 0x7f0cec1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0ce3fff640 1 -- 172.21.15.45:0/379709038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cec100f10 msgr2=0x7f0cec1971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cec100f10 0x7f0cec1971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0ce3fff640 1 -- 172.21.15.45:0/379709038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cec19ba60 msgr2=0x7f0cec19bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cec19ba60 0x7f0cec19bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0ce3fff640 1 -- 172.21.15.45:0/379709038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4009900 con 0x7f0cec105fe0 2024-01-31T05:40:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec105fe0 0x7f0cec1976e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4005fd0 tx=0x7f0cd400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0cf0a5c640 1 -- 172.21.15.45:0/379709038 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4009c30 con 0x7f0cec105fe0 2024-01-31T05:40:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0cf0a5c640 1 -- 172.21.15.45:0/379709038 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd4009d90 con 0x7f0cec105fe0 2024-01-31T05:40:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/379709038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cec1a3d10 con 0x7f0cec105fe0 2024-01-31T05:40:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0cf0a5c640 1 -- 172.21.15.45:0/379709038 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4003e80 con 0x7f0cec105fe0 2024-01-31T05:40:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.515+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/379709038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cec1a41c0 con 0x7f0cec105fe0 2024-01-31T05:40:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.518+0000 7f0cf0a5c640 1 -- 172.21.15.45:0/379709038 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f0cd4004000 con 0x7f0cec105fe0 2024-01-31T05:40:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.518+0000 7f0cf4825740 1 Processor -- start 2024-01-31T05:40:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.518+0000 7f0cf4825740 1 -- start start 2024-01-31T05:40:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.518+0000 7f0cf0a5c640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0cd80a14f0 0x7f0cd80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.518+0000 7f0cf4825740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eafb80 0x55acf3eaff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.518+0000 7f0cf4825740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eb04a0 0x55acf3ebd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.518+0000 7f0cf4825740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3ebdb60 0x55acf3ebff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.518+0000 7f0cf4825740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3e1f040 con 0x55acf3eb04a0 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.518+0000 7f0cf4825740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3e1c730 con 0x55acf3ebdb60 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.518+0000 7f0cf4825740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3e18020 con 0x55acf3eafb80 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.518+0000 7f0ceb7fe640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0cd80a14f0 0x7f0cd80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.518+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eb04a0 0x55acf3ebd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eb04a0 0x55acf3ebd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55106/0 (socket says 172.21.15.45:55106) 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ce3fff640 1 -- 172.21.15.45:0/1054205099 learned_addr learned my addr 172.21.15.45:0/1054205099 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0cf0a5c640 1 -- 172.21.15.45:0/379709038 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0cd40ccbd0 con 0x7f0cec105fe0 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ceaffd640 1 --2- 172.21.15.45:0/1054205099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eafb80 0x55acf3eaff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ceb7fe640 1 --2- 172.21.15.45:0/1054205099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3ebdb60 0x55acf3ebff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ce3fff640 1 -- 172.21.15.45:0/1054205099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eafb80 msgr2=0x55acf3eaff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/1054205099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eafb80 0x55acf3eaff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ce3fff640 1 -- 172.21.15.45:0/1054205099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3ebdb60 msgr2=0x55acf3ebff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/1054205099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3ebdb60 0x55acf3ebff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ce3fff640 1 -- 172.21.15.45:0/1054205099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf3e0fe40 con 0x55acf3eb04a0 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ceb7fe640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0cd80a14f0 0x7f0cd80a39b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4005cc0 tx=0x7f0ce4032340 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/1054205099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eb04a0 0x55acf3ebd620 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400bfd0 tx=0x7f0cd4095dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6ea538828a7841 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ce1ffb640 1 -- 172.21.15.45:0/1054205099 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4099330 con 0x55acf3eb04a0 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ce1ffb640 1 -- 172.21.15.45:0/1054205099 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd40035f0 con 0x55acf3eb04a0 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.519+0000 7f0ce1ffb640 1 -- 172.21.15.45:0/1054205099 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4097e00 con 0x55acf3eb04a0 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.520+0000 7f0cf4825740 1 -- 172.21.15.45:0/1054205099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eb04a0 msgr2=0x55acf3ebd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.520+0000 7f0cf4825740 1 --2- 172.21.15.45:0/1054205099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eb04a0 0x55acf3ebd620 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400bfd0 tx=0x7f0cd4095dd0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.520+0000 7f0cf4825740 1 -- 172.21.15.45:0/1054205099 shutdown_connections 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.520+0000 7f0cf4825740 1 --2- 172.21.15.45:0/1054205099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eb04a0 0x55acf3ebd620 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.520+0000 7f0cf4825740 1 --2- 172.21.15.45:0/1054205099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3ebdb60 0x55acf3ebff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.520+0000 7f0cf4825740 1 --2- 172.21.15.45:0/1054205099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eafb80 0x55acf3eaff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.520+0000 7f0cf4825740 1 -- 172.21.15.45:0/1054205099 >> 172.21.15.45:0/1054205099 conn(0x55acf3eaa980 msgr2=0x55acf3eaad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.520+0000 7f0cf4825740 1 -- 172.21.15.45:0/1054205099 shutdown_connections 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.520+0000 7f0cf4825740 1 -- 172.21.15.45:0/1054205099 wait complete. 2024-01-31T05:40:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.520+0000 7f0cf4825740 1 -- start start 2024-01-31T05:40:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0cf4825740 1 Processor -- start 2024-01-31T05:40:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0cf4825740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 0x55acf3ebab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0cf4825740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3eb04a0 0x55acf3ebb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0cf4825740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3ebdb60 0x55acf3ebb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0cf4825740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3dd11b0 con 0x55acf3eafb80 2024-01-31T05:40:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0cf4825740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3dd0e50 con 0x55acf3eb04a0 2024-01-31T05:40:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0cf4825740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3d96090 con 0x55acf3ebdb60 2024-01-31T05:40:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0cf4825740 10 client.0 _ll_register_callbacks cb 0x55acf3dbba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:40:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0ceaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 0x55acf3ebab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0ceaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 0x55acf3ebab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55116/0 (socket says 172.21.15.45:55116) 2024-01-31T05:40:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0ceaffd640 1 -- 172.21.15.45:0/4200284686 learned_addr learned my addr 172.21.15.45:0/4200284686 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0ceb7fe640 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3ebdb60 0x55acf3ebb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3eb04a0 0x55acf3ebb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0ceaffd640 1 -- 172.21.15.45:0/4200284686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3ebdb60 msgr2=0x55acf3ebb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0ceaffd640 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3ebdb60 0x55acf3ebb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0ceaffd640 1 -- 172.21.15.45:0/4200284686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3eb04a0 msgr2=0x55acf3ebb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0ceaffd640 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3eb04a0 0x55acf3ebb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0ceaffd640 1 -- 172.21.15.45:0/4200284686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cec1a41c0 con 0x55acf3eafb80 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.521+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3eb04a0 0x55acf3ebb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0ceaffd640 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 0x55acf3ebab30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00bf10 tx=0x7f0cdc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d9e85ca6e57e79 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0ccaffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0ccaffd640 1 -- 172.21.15.45:0/4200284686 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0108a0 con 0x55acf3eafb80 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0ccaffd640 1 -- 172.21.15.45:0/4200284686 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0cdc010a00 con 0x55acf3eafb80 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0cf4825740 1 -- 172.21.15.45:0/4200284686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acf3e11010 con 0x55acf3eafb80 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0ccaffd640 1 -- 172.21.15.45:0/4200284686 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cdc010bc0 con 0x55acf3eafb80 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0cf4825740 10 client.16806 fetch_fsmap learned FSMap version 33 2024-01-31T05:40:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0cf4825740 1 -- 172.21.15.45:0/4200284686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55acf3e166a0 con 0x55acf3eafb80 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0ccaffd640 1 -- 172.21.15.45:0/4200284686 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cdc027080 con 0x55acf3eafb80 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0ccaffd640 1 -- 172.21.15.45:0/4200284686 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0cdc021030 con 0x55acf3eafb80 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0cf4825740 10 client.16806 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0cf4825740 10 client.16806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0cf4825740 20 client.16806 populate_metadata read hostname 'smithi045' 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0cf4825740 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55acf3ecda00 0x55acf3ecfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0cf4825740 4 client.16806 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.522+0000 7f0cf4825740 1 -- 172.21.15.45:0/4200284686 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acf3e10d30 con 0x55acf3ecda00 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.523+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55acf3ecda00 0x55acf3ecfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.523+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55acf3ecda00 0x55acf3ecfe40 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ecb82460a9ad5c server_cookie=6d35258a43943027 in_seq=0 out_seq=0 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.523+0000 7f0ccaffd640 10 client.16806 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0ccaffd640 1 -- 172.21.15.45:0/4200284686 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55acf3e10d30 con 0x55acf3ecda00 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0ccaffd640 10 client.16806 handle_command_reply: tid=0 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0cf4825740 1 client.16806 shutdown 2024-01-31T05:40:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0cf4825740 1 -- 172.21.15.45:0/4200284686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 msgr2=0x55acf3ebab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0cf4825740 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 0x55acf3ebab30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00bf10 tx=0x7f0cdc012820 comp rx=0 tx=0).stop 2024-01-31T05:40:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0cf4825740 1 -- 172.21.15.45:0/4200284686 shutdown_connections 2024-01-31T05:40:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0cf4825740 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55acf3ecda00 0x55acf3ecfe40 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0cf4825740 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3ebdb60 0x55acf3ebb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0cf4825740 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3eb04a0 0x55acf3ebb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0cf4825740 1 --2- 172.21.15.45:0/4200284686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 0x55acf3ebab30 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0ccaffd640 0 client.16806 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:40:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0cf4825740 1 -- 172.21.15.45:0/4200284686 >> 172.21.15.45:0/4200284686 conn(0x55acf3eaa980 msgr2=0x55acf3eac6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0cf4825740 1 -- 172.21.15.45:0/4200284686 shutdown_connections 2024-01-31T05:40:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0cf4825740 1 -- 172.21.15.45:0/4200284686 wait complete. 2024-01-31T05:40:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.524+0000 7f0cf4825740 20 client.16806 trim_cache size 0 max 16384 2024-01-31T05:40:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.537+0000 7f0cf4825740 1 Processor -- start 2024-01-31T05:40:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0cf4825740 1 -- start start 2024-01-31T05:40:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0cf4825740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 0x55acf3ebd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0cf4825740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eb04a0 0x55acf3ebb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0cf4825740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3ebdb60 0x55acf3eb7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0cf4825740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3ea60b0 con 0x55acf3eafb80 2024-01-31T05:40:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0cf4825740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3ea4e30 con 0x55acf3ebdb60 2024-01-31T05:40:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0cf4825740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3eaf340 con 0x55acf3eb04a0 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ceaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 0x55acf3ebd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ceaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 0x55acf3ebd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55130/0 (socket says 172.21.15.45:55130) 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ceaffd640 1 -- 172.21.15.45:0/829377120 learned_addr learned my addr 172.21.15.45:0/829377120 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ceb7fe640 1 --2- 172.21.15.45:0/829377120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3ebdb60 0x55acf3eb7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/829377120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eb04a0 0x55acf3ebb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ceaffd640 1 -- 172.21.15.45:0/829377120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eb04a0 msgr2=0x55acf3ebb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ceaffd640 1 --2- 172.21.15.45:0/829377120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eb04a0 0x55acf3ebb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ceaffd640 1 -- 172.21.15.45:0/829377120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3ebdb60 msgr2=0x55acf3eb7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ceaffd640 1 --2- 172.21.15.45:0/829377120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3ebdb60 0x55acf3eb7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ceaffd640 1 -- 172.21.15.45:0/829377120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf3db3030 con 0x55acf3eafb80 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ceb7fe640 1 --2- 172.21.15.45:0/829377120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3ebdb60 0x55acf3eb7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/829377120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eb04a0 0x55acf3ebb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:40:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ceaffd640 1 --2- 172.21.15.45:0/829377120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 0x55acf3ebd260 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc023500 tx=0x7f0cdc00b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88c3dea9c3cb65e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.538+0000 7f0ce17fa640 1 -- 172.21.15.45:0/829377120 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc00bad0 con 0x55acf3eafb80 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.539+0000 7f0ce17fa640 1 -- 172.21.15.45:0/829377120 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55acf3e11010 con 0x55acf3eafb80 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.539+0000 7f0ce17fa640 1 -- 172.21.15.45:0/829377120 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc004cc0 con 0x55acf3eafb80 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.539+0000 7f0cf4825740 1 -- 172.21.15.45:0/829377120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 msgr2=0x55acf3ebd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.539+0000 7f0cf4825740 1 --2- 172.21.15.45:0/829377120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 0x55acf3ebd260 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc023500 tx=0x7f0cdc00b540 comp rx=0 tx=0).stop 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.539+0000 7f0cf4825740 1 -- 172.21.15.45:0/829377120 shutdown_connections 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.539+0000 7f0cf4825740 1 --2- 172.21.15.45:0/829377120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3ebdb60 0x55acf3eb7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.539+0000 7f0cf4825740 1 --2- 172.21.15.45:0/829377120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eb04a0 0x55acf3ebb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.539+0000 7f0cf4825740 1 --2- 172.21.15.45:0/829377120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3eafb80 0x55acf3ebd260 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.539+0000 7f0cf4825740 1 -- 172.21.15.45:0/829377120 >> 172.21.15.45:0/829377120 conn(0x55acf3eaa980 msgr2=0x55acf3eb5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.539+0000 7f0cf4825740 1 -- 172.21.15.45:0/829377120 shutdown_connections 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.539+0000 7f0cf4825740 1 -- 172.21.15.45:0/829377120 wait complete. 2024-01-31T05:40:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.539+0000 7f0cf4825740 1 -- start start 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0cf4825740 1 Processor -- start 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0cf4825740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eafb80 0x55acf3ec3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0cf4825740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3eb04a0 0x55acf3ec4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0cf4825740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3ebdb60 0x55acf3f1df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0cf4825740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3f1e480 con 0x55acf3ebdb60 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0cf4825740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3f1e5c0 con 0x55acf3eb04a0 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0cf4825740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3f1e780 con 0x55acf3eafb80 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0cf4825740 10 client.0 _ll_register_callbacks cb 0x7f0cd4009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0ceb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3ebdb60 0x55acf3f1df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0ceb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3ebdb60 0x55acf3f1df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55138/0 (socket says 172.21.15.45:55138) 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0ceaffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eafb80 0x55acf3ec3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0ceb7fe640 1 -- 172.21.15.45:0/1480800531 learned_addr learned my addr 172.21.15.45:0/1480800531 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0ce3fff640 1 --2- 172.21.15.45:0/1480800531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3eb04a0 0x55acf3ec4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0ceb7fe640 1 -- 172.21.15.45:0/1480800531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eafb80 msgr2=0x55acf3ec3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0ceb7fe640 1 --2- 172.21.15.45:0/1480800531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eafb80 0x55acf3ec3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0ceb7fe640 1 -- 172.21.15.45:0/1480800531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3eb04a0 msgr2=0x55acf3ec4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0ceb7fe640 1 --2- 172.21.15.45:0/1480800531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3eb04a0 0x55acf3ec4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.540+0000 7f0ceb7fe640 1 -- 172.21.15.45:0/1480800531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acf3e166a0 con 0x55acf3ebdb60 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0ccaffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0ceb7fe640 1 --2- 172.21.15.45:0/1480800531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3ebdb60 0x55acf3f1df40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009cd0 tx=0x7f0ce402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddab8c76dc8d6a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0ccaffd640 1 -- 172.21.15.45:0/1480800531 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4032aa0 con 0x55acf3ebdb60 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0ccaffd640 1 -- 172.21.15.45:0/1480800531 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0ce4047030 con 0x55acf3ebdb60 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0cf4825740 1 -- 172.21.15.45:0/1480800531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cd40035f0 con 0x55acf3ebdb60 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0ccaffd640 1 -- 172.21.15.45:0/1480800531 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce4003970 con 0x55acf3ebdb60 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0cf4825740 10 client.16812 fetch_fsmap learned FSMap version 33 2024-01-31T05:40:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0cf4825740 1 -- 172.21.15.45:0/1480800531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55acf3dd8a50 con 0x55acf3ebdb60 2024-01-31T05:40:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0ccaffd640 1 -- 172.21.15.45:0/1480800531 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd40035f0 con 0x55acf3ebdb60 2024-01-31T05:40:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0ccaffd640 1 -- 172.21.15.45:0/1480800531 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ce403f460 con 0x55acf3ebdb60 2024-01-31T05:40:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0cf4825740 10 client.16812 fetch_fsmap finished waiting for FSMap version 33 2024-01-31T05:40:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0cf4825740 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-31T05:40:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0cf4825740 20 client.16812 populate_metadata read hostname 'smithi045' 2024-01-31T05:40:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0cf4825740 1 --2- 172.21.15.45:0/1480800531 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55acf3f25f80 0x55acf3f283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0cf4825740 4 client.16812 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-31T05:40:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.541+0000 7f0cf4825740 1 -- 172.21.15.45:0/1480800531 --> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55acf3b17600 con 0x55acf3f25f80 2024-01-31T05:40:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.542+0000 7f0ceaffd640 1 --2- 172.21.15.45:0/1480800531 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55acf3f25f80 0x55acf3f283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.542+0000 7f0ceaffd640 1 --2- 172.21.15.45:0/1480800531 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55acf3f25f80 0x55acf3f283a0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445ac070b2b67ebb server_cookie=613b4bb3f2e2685d in_seq=0 out_seq=0 2024-01-31T05:40:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:14.542+0000 7f0ccaffd640 10 client.16812 ms_handle_connect on v2:172.21.15.156:6838/3137398384 2024-01-31T05:40:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0ccaffd640 1 -- 172.21.15.45:0/1480800531 <== mds.0 v2:172.21.15.156:6838/3137398384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55acf3b17600 con 0x55acf3f25f80 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0ccaffd640 10 client.16812 handle_command_reply: tid=0 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0cf4825740 1 client.16812 shutdown 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0cf4825740 1 -- 172.21.15.45:0/1480800531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3ebdb60 msgr2=0x55acf3f1df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0cf4825740 1 --2- 172.21.15.45:0/1480800531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3ebdb60 0x55acf3f1df40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009cd0 tx=0x7f0ce402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0cf4825740 1 -- 172.21.15.45:0/1480800531 shutdown_connections 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0cf4825740 1 --2- 172.21.15.45:0/1480800531 >> [v2:172.21.15.156:6838/3137398384,v1:172.21.15.156:6839/3137398384] conn(0x55acf3f25f80 0x55acf3f283a0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0cf4825740 1 --2- 172.21.15.45:0/1480800531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55acf3ebdb60 0x55acf3f1df40 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0cf4825740 1 --2- 172.21.15.45:0/1480800531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf3eb04a0 0x55acf3ec4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0ccaffd640 0 client.16812 ms_handle_reset on v2:172.21.15.156:6838/3137398384 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0cf4825740 1 --2- 172.21.15.45:0/1480800531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55acf3eafb80 0x55acf3ec3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0cf4825740 1 -- 172.21.15.45:0/1480800531 >> 172.21.15.45:0/1480800531 conn(0x55acf3eaa980 msgr2=0x55acf3ea9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0cf4825740 1 -- 172.21.15.45:0/1480800531 shutdown_connections 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0cf4825740 1 -- 172.21.15.45:0/1480800531 wait complete. 2024-01-31T05:40:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.119+0000 7f0cf4825740 20 client.16812 trim_cache size 0 max 16384 2024-01-31T05:40:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-01-31T05:40:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-01-31T05:40:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-01-31T05:40:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-01-31T05:40:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.120+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/379709038 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0cd80a14f0 msgr2=0x7f0cd80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.120+0000 7f0cf1a5e640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0cd80a14f0 0x7f0cd80a39b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4005cc0 tx=0x7f0ce4032340 comp rx=0 tx=0).stop 2024-01-31T05:40:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.120+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/379709038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec105fe0 msgr2=0x7f0cec1976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.120+0000 7f0cf1a5e640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec105fe0 0x7f0cec1976e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4005fd0 tx=0x7f0cd400dde0 comp rx=0 tx=0).stop 2024-01-31T05:40:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.120+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/379709038 shutdown_connections 2024-01-31T05:40:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.120+0000 7f0cf1a5e640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f0cd80a14f0 0x7f0cd80a39b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.120+0000 7f0cf1a5e640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec105fe0 0x7f0cec1976e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.120+0000 7f0cf1a5e640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cec19ba60 0x7f0cec19bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.120+0000 7f0cf1a5e640 1 --2- 172.21.15.45:0/379709038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cec100f10 0x7f0cec1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.120+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/379709038 >> 172.21.15.45:0/379709038 conn(0x7f0cec0fc7b0 msgr2=0x7f0cec0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.120+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/379709038 shutdown_connections 2024-01-31T05:40:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-01-31T05:40:15.120+0000 7f0cf1a5e640 1 -- 172.21.15.45:0/379709038 wait complete. 2024-01-31T05:40:15.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:14 smithi045 ceph-mon[31585]: pgmap v559: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 445 op/s 2024-01-31T05:40:15.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:14 smithi156 ceph-mon[35720]: pgmap v559: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 445 op/s 2024-01-31T05:40:15.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:14 smithi186 ceph-mon[35659]: pgmap v559: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 445 op/s 2024-01-31T05:40:16.991 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-01-31T05:40:16.992 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-01-31T05:40:17.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:16 smithi045 ceph-mon[31585]: pgmap v560: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 8.4 MiB/s wr, 362 op/s 2024-01-31T05:40:17.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:16 smithi156 ceph-mon[35720]: pgmap v560: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 8.4 MiB/s wr, 362 op/s 2024-01-31T05:40:17.277 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi045 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=661946e7044723130c03627323f6a879a9848cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-01-31T05:40:17.516 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=2d6a28534a334789bad205b1b525d50a Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi045 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=661946e7044723130c03627323f6a879a9848cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-01-31T05:40:17.519 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-01-31T05:40:17.529 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi045.front.sepia.ceph.com 2024-01-31T05:40:17.543 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:16 smithi186 ceph-mon[35659]: pgmap v560: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 8.4 MiB/s wr, 362 op/s 2024-01-31T05:40:17.544 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-01-31T05:40:17.571 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-01-31T05:40:17.571 INFO:teuthology.task.exec:Executing custom commands... 2024-01-31T05:40:17.571 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-01-31T05:40:17.572 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-01-31T05:40:17.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-01-31T05:40:17.685 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-01-31T05:40:17.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.830+0000 7f68ef3a7640 1 -- 172.21.15.45:0/1535324886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8068d10 msgr2=0x7f68e810db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:17.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.830+0000 7f68ef3a7640 1 --2- 172.21.15.45:0/1535324886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8068d10 0x7f68e810db80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009ca0 tx=0x7f68e002c180 comp rx=0 tx=0).stop 2024-01-31T05:40:17.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.830+0000 7f68ef3a7640 1 -- 172.21.15.45:0/1535324886 shutdown_connections 2024-01-31T05:40:17.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.830+0000 7f68ef3a7640 1 --2- 172.21.15.45:0/1535324886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8068d10 0x7f68e810db80 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:17.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.830+0000 7f68ef3a7640 1 --2- 172.21.15.45:0/1535324886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8068370 0x7f68e80687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:17.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.830+0000 7f68ef3a7640 1 --2- 172.21.15.45:0/1535324886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8106de0 0x7f68e81071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:17.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.830+0000 7f68ef3a7640 1 -- 172.21.15.45:0/1535324886 >> 172.21.15.45:0/1535324886 conn(0x7f68e8076a40 msgr2=0x7f68e8076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:17.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.830+0000 7f68ef3a7640 1 -- 172.21.15.45:0/1535324886 shutdown_connections 2024-01-31T05:40:17.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.830+0000 7f68ef3a7640 1 -- 172.21.15.45:0/1535324886 wait complete. 2024-01-31T05:40:17.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.830+0000 7f68ef3a7640 1 Processor -- start 2024-01-31T05:40:17.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.830+0000 7f68ef3a7640 1 -- start start 2024-01-31T05:40:17.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ef3a7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8068370 0x7f68e80fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:17.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ef3a7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8068d10 0x7f68e8100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:17.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ef3a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8106de0 0x7f68e8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:17.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ef3a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e81a5490 con 0x7f68e8106de0 2024-01-31T05:40:17.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ef3a7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e81a5600 con 0x7f68e8068d10 2024-01-31T05:40:17.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ef3a7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e81a57c0 con 0x7f68e8068370 2024-01-31T05:40:17.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ed143640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8068370 0x7f68e80fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:17.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ed944640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8106de0 0x7f68e8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:17.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ed944640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8106de0 0x7f68e8100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52032/0 (socket says 172.21.15.45:52032) 2024-01-31T05:40:17.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ed143640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8068370 0x7f68e80fff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.45:54516/0 (socket says 172.21.15.45:54516) 2024-01-31T05:40:17.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ed944640 1 -- 172.21.15.45:0/3492776341 learned_addr learned my addr 172.21.15.45:0/3492776341 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:17.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ec942640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8068d10 0x7f68e8100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:17.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ed944640 1 -- 172.21.15.45:0/3492776341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8068370 msgr2=0x7f68e80fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:17.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ed944640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8068370 0x7f68e80fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:17.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ed944640 1 -- 172.21.15.45:0/3492776341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8068d10 msgr2=0x7f68e8100450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:17.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ed944640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8068d10 0x7f68e8100450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:17.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ed944640 1 -- 172.21.15.45:0/3492776341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e0009900 con 0x7f68e8106de0 2024-01-31T05:40:17.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ed143640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8068370 0x7f68e80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:40:17.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ec942640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8068d10 0x7f68e8100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:40:17.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.831+0000 7f68ed944640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8106de0 0x7f68e8100990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f68e002c790 tx=0x7f68e0034740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:17.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.832+0000 7f68de7fc640 1 -- 172.21.15.45:0/3492776341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0003d80 con 0x7f68e8106de0 2024-01-31T05:40:17.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.832+0000 7f68de7fc640 1 -- 172.21.15.45:0/3492776341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68e0032b70 con 0x7f68e8106de0 2024-01-31T05:40:17.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.832+0000 7f68ef3a7640 1 -- 172.21.15.45:0/3492776341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e81a5a40 con 0x7f68e8106de0 2024-01-31T05:40:17.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.832+0000 7f68de7fc640 1 -- 172.21.15.45:0/3492776341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00046a0 con 0x7f68e8106de0 2024-01-31T05:40:17.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.832+0000 7f68ef3a7640 1 -- 172.21.15.45:0/3492776341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e81a5ef0 con 0x7f68e8106de0 2024-01-31T05:40:17.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.833+0000 7f68ef3a7640 1 -- 172.21.15.45:0/3492776341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e804ea50 con 0x7f68e8106de0 2024-01-31T05:40:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.834+0000 7f68de7fc640 1 -- 172.21.15.45:0/3492776341 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f68e000a040 con 0x7f68e8106de0 2024-01-31T05:40:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.834+0000 7f68de7fc640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f68d40a1520 0x7f68d40a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.834+0000 7f68ed143640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f68d40a1520 0x7f68d40a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.834+0000 7f68de7fc640 1 -- 172.21.15.45:0/3492776341 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f68e00ef840 con 0x7f68e8106de0 2024-01-31T05:40:17.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.835+0000 7f68ed143640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f68d40a1520 0x7f68d40a39e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009f40 tx=0x7f68e4005f90 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:17.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:17.839+0000 7f68de7fc640 1 -- 172.21.15.45:0/3492776341 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f68e00b8b40 con 0x7f68e8106de0 2024-01-31T05:40:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.040+0000 7f68ef3a7640 1 -- 172.21.15.45:0/3492776341 --> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f68e8109240 con 0x7f68d40a1520 2024-01-31T05:40:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.051+0000 7f68de7fc640 1 -- 172.21.15.45:0/3492776341 <== mgr.15315 v2:172.21.15.45:6800/1984996061 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f68e8109240 con 0x7f68d40a1520 2024-01-31T05:40:18.051 INFO:teuthology.orchestra.run.smithi045.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-01-31T00:00:00", "created": "2024-01-31T05:22:42", "first": "2024-01-31T05:23:00", "last": "2024-01-31T05:40:00", "last_pruned": "2024-01-31T05:40:00", "created_count": 18, "pruned_count": 12, "active": true} 2024-01-31T05:40:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.053+0000 7f68ef3a7640 1 -- 172.21.15.45:0/3492776341 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f68d40a1520 msgr2=0x7f68d40a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.053+0000 7f68ef3a7640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f68d40a1520 0x7f68d40a39e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009f40 tx=0x7f68e4005f90 comp rx=0 tx=0).stop 2024-01-31T05:40:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.053+0000 7f68ef3a7640 1 -- 172.21.15.45:0/3492776341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8106de0 msgr2=0x7f68e8100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.053+0000 7f68ef3a7640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8106de0 0x7f68e8100990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f68e002c790 tx=0x7f68e0034740 comp rx=0 tx=0).stop 2024-01-31T05:40:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.053+0000 7f68ef3a7640 1 -- 172.21.15.45:0/3492776341 shutdown_connections 2024-01-31T05:40:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.053+0000 7f68ef3a7640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f68d40a1520 0x7f68d40a39e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.053+0000 7f68ef3a7640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8106de0 0x7f68e8100990 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.053+0000 7f68ef3a7640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8068d10 0x7f68e8100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.053+0000 7f68ef3a7640 1 --2- 172.21.15.45:0/3492776341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8068370 0x7f68e80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.053+0000 7f68ef3a7640 1 -- 172.21.15.45:0/3492776341 >> 172.21.15.45:0/3492776341 conn(0x7f68e8076a40 msgr2=0x7f68e80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.053+0000 7f68ef3a7640 1 -- 172.21.15.45:0/3492776341 shutdown_connections 2024-01-31T05:40:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.053+0000 7f68ef3a7640 1 -- 172.21.15.45:0/3492776341 wait complete. 2024-01-31T05:40:18.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-01-31T05:40:18.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.168+0000 7f67aa4df640 1 -- 172.21.15.45:0/2009116789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67a41060b0 msgr2=0x7f67a410dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.168+0000 7f67aa4df640 1 --2- 172.21.15.45:0/2009116789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67a41060b0 0x7f67a410dbf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f679c009cd0 tx=0x7f679c02c0f0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.168+0000 7f67aa4df640 1 -- 172.21.15.45:0/2009116789 shutdown_connections 2024-01-31T05:40:18.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.168+0000 7f67aa4df640 1 --2- 172.21.15.45:0/2009116789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67a41060b0 0x7f67a410dbf0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.168+0000 7f67aa4df640 1 --2- 172.21.15.45:0/2009116789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67a406a110 0x7f67a4105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.168+0000 7f67aa4df640 1 --2- 172.21.15.45:0/2009116789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67a4069760 0x7f67a4069b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.168+0000 7f67aa4df640 1 -- 172.21.15.45:0/2009116789 >> 172.21.15.45:0/2009116789 conn(0x7f67a40769c0 msgr2=0x7f67a4076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.168+0000 7f67aa4df640 1 -- 172.21.15.45:0/2009116789 shutdown_connections 2024-01-31T05:40:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.168+0000 7f67aa4df640 1 -- 172.21.15.45:0/2009116789 wait complete. 2024-01-31T05:40:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.169+0000 7f67aa4df640 1 Processor -- start 2024-01-31T05:40:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.169+0000 7f67aa4df640 1 -- start start 2024-01-31T05:40:18.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.169+0000 7f67aa4df640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67a4069760 0x7f67a41971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.169+0000 7f67aa4df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67a41060b0 0x7f67a4197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.169+0000 7f67aa4df640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67a419ba80 0x7f67a419bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.169+0000 7f67aa4df640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a419c470 con 0x7f67a4069760 2024-01-31T05:40:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.169+0000 7f67aa4df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a419c5e0 con 0x7f67a41060b0 2024-01-31T05:40:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.169+0000 7f67aa4df640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a419c750 con 0x7f67a419ba80 2024-01-31T05:40:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.169+0000 7f67a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67a4069760 0x7f67a41971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.169+0000 7f67a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67a4069760 0x7f67a41971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52056/0 (socket says 172.21.15.45:52056) 2024-01-31T05:40:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.169+0000 7f67a3fff640 1 -- 172.21.15.45:0/217585372 learned_addr learned my addr 172.21.15.45:0/217585372 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.169+0000 7f67a37fe640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67a41060b0 0x7f67a4197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.170+0000 7f67a8a7c640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67a419ba80 0x7f67a419bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.170+0000 7f67a3fff640 1 -- 172.21.15.45:0/217585372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67a419ba80 msgr2=0x7f67a419bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.170+0000 7f67a3fff640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67a419ba80 0x7f67a419bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.170+0000 7f67a3fff640 1 -- 172.21.15.45:0/217585372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67a41060b0 msgr2=0x7f67a4197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.170+0000 7f67a3fff640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67a41060b0 0x7f67a4197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.170+0000 7f67a3fff640 1 -- 172.21.15.45:0/217585372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c009900 con 0x7f67a4069760 2024-01-31T05:40:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.170+0000 7f67a3fff640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67a4069760 0x7f67a41971c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f679400d770 tx=0x7f679400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.170+0000 7f67a17fa640 1 -- 172.21.15.45:0/217585372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794009c30 con 0x7f67a4069760 2024-01-31T05:40:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.170+0000 7f67a17fa640 1 -- 172.21.15.45:0/217585372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f679400de80 con 0x7f67a4069760 2024-01-31T05:40:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.170+0000 7f67a17fa640 1 -- 172.21.15.45:0/217585372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794004de0 con 0x7f67a4069760 2024-01-31T05:40:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.170+0000 7f67aa4df640 1 -- 172.21.15.45:0/217585372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a41a3d30 con 0x7f67a4069760 2024-01-31T05:40:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.170+0000 7f67aa4df640 1 -- 172.21.15.45:0/217585372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a41a4190 con 0x7f67a4069760 2024-01-31T05:40:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.171+0000 7f67aa4df640 1 -- 172.21.15.45:0/217585372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a41023f0 con 0x7f67a4069760 2024-01-31T05:40:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.172+0000 7f67a17fa640 1 -- 172.21.15.45:0/217585372 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f6794004f40 con 0x7f67a4069760 2024-01-31T05:40:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.172+0000 7f67a17fa640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f67900a14f0 0x7f67900a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.173+0000 7f67a37fe640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f67900a14f0 0x7f67900a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.173+0000 7f67a17fa640 1 -- 172.21.15.45:0/217585372 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f67940cc9f0 con 0x7f67a4069760 2024-01-31T05:40:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.173+0000 7f67a37fe640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f67900a14f0 0x7f67900a39b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f67a40ff8e0 tx=0x7f678c00b390 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.177+0000 7f67a17fa640 1 -- 172.21.15.45:0/217585372 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f6794095b30 con 0x7f67a4069760 2024-01-31T05:40:18.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:17 smithi156 ceph-mon[35720]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-01-31T05:40:18.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:17 smithi156 ceph-mon[35720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-01-31T05:40:18.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:17 smithi186 ceph-mon[35659]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-01-31T05:40:18.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:17 smithi186 ceph-mon[35659]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-01-31T05:40:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.378+0000 7f67aa4df640 1 -- 172.21.15.45:0/217585372 --> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f67a406a110 con 0x7f67900a14f0 2024-01-31T05:40:18.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.388+0000 7f67a17fa640 1 -- 172.21.15.45:0/217585372 <== mgr.15315 v2:172.21.15.45:6800/1984996061 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f67a406a110 con 0x7f67900a14f0 2024-01-31T05:40:18.388 INFO:teuthology.orchestra.run.smithi045.stdout:/ 1m 6m3h 2024-01-31T05:40:18.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.390+0000 7f67aa4df640 1 -- 172.21.15.45:0/217585372 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f67900a14f0 msgr2=0x7f67900a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.390+0000 7f67aa4df640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f67900a14f0 0x7f67900a39b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f67a40ff8e0 tx=0x7f678c00b390 comp rx=0 tx=0).stop 2024-01-31T05:40:18.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.390+0000 7f67aa4df640 1 -- 172.21.15.45:0/217585372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67a4069760 msgr2=0x7f67a41971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.390+0000 7f67aa4df640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67a4069760 0x7f67a41971c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f679400d770 tx=0x7f679400dc30 comp rx=0 tx=0).stop 2024-01-31T05:40:18.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.390+0000 7f67aa4df640 1 -- 172.21.15.45:0/217585372 shutdown_connections 2024-01-31T05:40:18.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.390+0000 7f67aa4df640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f67900a14f0 0x7f67900a39b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.390+0000 7f67aa4df640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67a419ba80 0x7f67a419bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.390+0000 7f67aa4df640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67a41060b0 0x7f67a4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.390+0000 7f67aa4df640 1 --2- 172.21.15.45:0/217585372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67a4069760 0x7f67a41971c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.390+0000 7f67aa4df640 1 -- 172.21.15.45:0/217585372 >> 172.21.15.45:0/217585372 conn(0x7f67a40769c0 msgr2=0x7f67a40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.390+0000 7f67aa4df640 1 -- 172.21.15.45:0/217585372 shutdown_connections 2024-01-31T05:40:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.390+0000 7f67aa4df640 1 -- 172.21.15.45:0/217585372 wait complete. 2024-01-31T05:40:18.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:17 smithi045 ceph-mon[31585]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-01-31T05:40:18.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:17 smithi045 ceph-mon[31585]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-01-31T05:40:18.401 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-01-31T05:40:18.411 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:40:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.514+0000 7fecb6b9a640 1 -- 172.21.15.45:0/4051693250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb0069340 msgr2=0x7fecb0069780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.514+0000 7fecb6b9a640 1 --2- 172.21.15.45:0/4051693250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb0069340 0x7fecb0069780 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fec98009900 tx=0x7fec9802bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.514+0000 7fecb6b9a640 1 -- 172.21.15.45:0/4051693250 shutdown_connections 2024-01-31T05:40:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.514+0000 7fecb6b9a640 1 --2- 172.21.15.45:0/4051693250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb0069340 0x7fecb0069780 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.514+0000 7fecb6b9a640 1 --2- 172.21.15.45:0/4051693250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fecb0069cc0 0x7fecb010db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.514+0000 7fecb6b9a640 1 --2- 172.21.15.45:0/4051693250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb0106d10 0x7fecb01070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.514+0000 7fecb6b9a640 1 -- 172.21.15.45:0/4051693250 >> 172.21.15.45:0/4051693250 conn(0x7fecb00759f0 msgr2=0x7fecb0075e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.514+0000 7fecb6b9a640 1 -- 172.21.15.45:0/4051693250 shutdown_connections 2024-01-31T05:40:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.514+0000 7fecb6b9a640 1 -- 172.21.15.45:0/4051693250 wait complete. 2024-01-31T05:40:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.514+0000 7fecb6b9a640 1 Processor -- start 2024-01-31T05:40:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb6b9a640 1 -- start start 2024-01-31T05:40:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb6b9a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb0069340 0x7fecb01972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb6b9a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fecb0069cc0 0x7fecb01977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb6b9a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb0106d10 0x7fecb019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb6b9a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb019c160 con 0x7fecb0106d10 2024-01-31T05:40:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb6b9a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb019c2d0 con 0x7fecb0069cc0 2024-01-31T05:40:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb6b9a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb019c490 con 0x7fecb0069340 2024-01-31T05:40:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb5137640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb0106d10 0x7fecb019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb5137640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb0106d10 0x7fecb019bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52074/0 (socket says 172.21.15.45:52074) 2024-01-31T05:40:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb5137640 1 -- 172.21.15.45:0/622840625 learned_addr learned my addr 172.21.15.45:0/622840625 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb4936640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb0069340 0x7fecb01972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecaffff640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fecb0069cc0 0x7fecb01977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb5137640 1 -- 172.21.15.45:0/622840625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb0069340 msgr2=0x7fecb01972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb5137640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb0069340 0x7fecb01972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb5137640 1 -- 172.21.15.45:0/622840625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fecb0069cc0 msgr2=0x7fecb01977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb5137640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fecb0069cc0 0x7fecb01977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb5137640 1 -- 172.21.15.45:0/622840625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec98009560 con 0x7fecb0106d10 2024-01-31T05:40:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecb5137640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb0106d10 0x7fecb019bb60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7feca000d750 tx=0x7feca000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.515+0000 7fecadffb640 1 -- 172.21.15.45:0/622840625 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca0009bd0 con 0x7fecb0106d10 2024-01-31T05:40:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.516+0000 7fecadffb640 1 -- 172.21.15.45:0/622840625 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feca000de60 con 0x7fecb0106d10 2024-01-31T05:40:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.516+0000 7fecb6b9a640 1 -- 172.21.15.45:0/622840625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb019c770 con 0x7fecb0106d10 2024-01-31T05:40:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.516+0000 7fecadffb640 1 -- 172.21.15.45:0/622840625 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca0004e00 con 0x7fecb0106d10 2024-01-31T05:40:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.516+0000 7fecb6b9a640 1 -- 172.21.15.45:0/622840625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb01a3ff0 con 0x7fecb0106d10 2024-01-31T05:40:18.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.517+0000 7fecb6b9a640 1 -- 172.21.15.45:0/622840625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb01023b0 con 0x7fecb0106d10 2024-01-31T05:40:18.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.518+0000 7fecadffb640 1 -- 172.21.15.45:0/622840625 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7feca0004f60 con 0x7fecb0106d10 2024-01-31T05:40:18.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.518+0000 7fecadffb640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fec9c0a14d0 0x7fec9c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.518+0000 7fecb4936640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fec9c0a14d0 0x7fec9c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.519+0000 7fecadffb640 1 -- 172.21.15.45:0/622840625 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7feca00cc820 con 0x7fecb0106d10 2024-01-31T05:40:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.519+0000 7fecb4936640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fec9c0a14d0 0x7fec9c0a3990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feca8005fd0 tx=0x7feca8009680 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.523+0000 7fecadffb640 1 -- 172.21.15.45:0/622840625 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7feca00994a0 con 0x7fecb0106d10 2024-01-31T05:40:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.704+0000 7fecb6b9a640 1 -- 172.21.15.45:0/622840625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecb01985f0 con 0x7fecb0106d10 2024-01-31T05:40:18.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.706+0000 7fecadffb640 1 -- 172.21.15.45:0/622840625 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+23052 (secure 0 0 0) 0x7feca0020090 con 0x7fecb0106d10 2024-01-31T05:40:18.706 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:40:18.706 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":119,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:40:12.712633+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":118,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/1441130788":"2024-01-31T06:27:35.007948+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/782292734":"2024-02-01T05:22:41.447737+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2742663751":"2024-02-01T05:22:41.447737+0000","172.21.15.45:0/1829415949":"2024-02-01T05:22:41.447737+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2788245525":"2024-02-01T05:22:41.447737+0000","172.21.15.45:0/948304328":"2024-02-01T05:22:41.447737+0000","172.21.15.45:6801/2788245525":"2024-02-01T05:22:41.447737+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:40:18.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.708+0000 7fecb6b9a640 1 -- 172.21.15.45:0/622840625 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fec9c0a14d0 msgr2=0x7fec9c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.708+0000 7fecb6b9a640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fec9c0a14d0 0x7fec9c0a3990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feca8005fd0 tx=0x7feca8009680 comp rx=0 tx=0).stop 2024-01-31T05:40:18.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.708+0000 7fecb6b9a640 1 -- 172.21.15.45:0/622840625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb0106d10 msgr2=0x7fecb019bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.708+0000 7fecb6b9a640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb0106d10 0x7fecb019bb60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7feca000d750 tx=0x7feca000dc10 comp rx=0 tx=0).stop 2024-01-31T05:40:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.708+0000 7fecb6b9a640 1 -- 172.21.15.45:0/622840625 shutdown_connections 2024-01-31T05:40:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.708+0000 7fecb6b9a640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fec9c0a14d0 0x7fec9c0a3990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.708+0000 7fecb6b9a640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb0106d10 0x7fecb019bb60 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.708+0000 7fecb6b9a640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fecb0069cc0 0x7fecb01977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.708+0000 7fecb6b9a640 1 --2- 172.21.15.45:0/622840625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb0069340 0x7fecb01972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.708+0000 7fecb6b9a640 1 -- 172.21.15.45:0/622840625 >> 172.21.15.45:0/622840625 conn(0x7fecb00759f0 msgr2=0x7fecb00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.708+0000 7fecb6b9a640 1 -- 172.21.15.45:0/622840625 shutdown_connections 2024-01-31T05:40:18.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.708+0000 7fecb6b9a640 1 -- 172.21.15.45:0/622840625 wait complete. 2024-01-31T05:40:18.719 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:16:10.856581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:20:38.347599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:20:39.270522+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 118, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:40:18.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:40:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.876+0000 7f95b97c4640 1 -- 172.21.15.45:0/703367202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b4100780 msgr2=0x7f95b41047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.876+0000 7f95b97c4640 1 --2- 172.21.15.45:0/703367202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b4100780 0x7f95b41047c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f959c009930 tx=0x7f959c02bc20 comp rx=0 tx=0).stop 2024-01-31T05:40:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.876+0000 7f95b97c4640 1 -- 172.21.15.45:0/703367202 shutdown_connections 2024-01-31T05:40:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.876+0000 7f95b97c4640 1 --2- 172.21.15.45:0/703367202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b4100780 0x7f95b41047c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.876+0000 7f95b97c4640 1 --2- 172.21.15.45:0/703367202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95b4104ea0 0x7f95b410ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.876+0000 7f95b97c4640 1 --2- 172.21.15.45:0/703367202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95b40ffdd0 0x7f95b41001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.876+0000 7f95b97c4640 1 -- 172.21.15.45:0/703367202 >> 172.21.15.45:0/703367202 conn(0x7f95b4076cf0 msgr2=0x7f95b4077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.876+0000 7f95b97c4640 1 -- 172.21.15.45:0/703367202 shutdown_connections 2024-01-31T05:40:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.876+0000 7f95b97c4640 1 -- 172.21.15.45:0/703367202 wait complete. 2024-01-31T05:40:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95b97c4640 1 Processor -- start 2024-01-31T05:40:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95b97c4640 1 -- start start 2024-01-31T05:40:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95b97c4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95b40ffdd0 0x7f95b4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95b97c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b4104ea0 0x7f95b41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95b97c4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95b4198b80 0x7f95b41a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95b97c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b41a37d0 con 0x7f95b4104ea0 2024-01-31T05:40:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95b97c4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b41a3940 con 0x7f95b40ffdd0 2024-01-31T05:40:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95b97c4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b41a3b00 con 0x7f95b4198b80 2024-01-31T05:40:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95abfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b4104ea0 0x7f95b41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95abfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b4104ea0 0x7f95b41985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52084/0 (socket says 172.21.15.45:52084) 2024-01-31T05:40:18.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95b37fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95b4198b80 0x7f95b41a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95abfff640 1 -- 172.21.15.45:0/1048360977 learned_addr learned my addr 172.21.15.45:0/1048360977 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.877+0000 7f95b2ffd640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95b40ffdd0 0x7f95b4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.878+0000 7f95abfff640 1 -- 172.21.15.45:0/1048360977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95b4198b80 msgr2=0x7f95b41a30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.878+0000 7f95abfff640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95b4198b80 0x7f95b41a30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.878+0000 7f95abfff640 1 -- 172.21.15.45:0/1048360977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95b40ffdd0 msgr2=0x7f95b4198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.878+0000 7f95abfff640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95b40ffdd0 0x7f95b4198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.878+0000 7f95abfff640 1 -- 172.21.15.45:0/1048360977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c009560 con 0x7f95b4104ea0 2024-01-31T05:40:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.878+0000 7f95abfff640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b4104ea0 0x7f95b41985a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f959c009230 tx=0x7f959c02cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.878+0000 7f95b0ff9640 1 -- 172.21.15.45:0/1048360977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c004310 con 0x7f95b4104ea0 2024-01-31T05:40:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.878+0000 7f95b0ff9640 1 -- 172.21.15.45:0/1048360977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f959c004470 con 0x7f95b4104ea0 2024-01-31T05:40:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.878+0000 7f95b97c4640 1 -- 172.21.15.45:0/1048360977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b41a3d80 con 0x7f95b4104ea0 2024-01-31T05:40:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.878+0000 7f95b0ff9640 1 -- 172.21.15.45:0/1048360977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c003d80 con 0x7f95b4104ea0 2024-01-31T05:40:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.878+0000 7f95b97c4640 1 -- 172.21.15.45:0/1048360977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b41a4230 con 0x7f95b4104ea0 2024-01-31T05:40:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.879+0000 7f95b97c4640 1 -- 172.21.15.45:0/1048360977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b4067ae0 con 0x7f95b4104ea0 2024-01-31T05:40:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.880+0000 7f95b0ff9640 1 -- 172.21.15.45:0/1048360977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f959c005000 con 0x7f95b4104ea0 2024-01-31T05:40:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.881+0000 7f95b0ff9640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f95a00a1450 0x7f95a00a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.881+0000 7f95b2ffd640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f95a00a1450 0x7f95a00a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.881+0000 7f95b0ff9640 1 -- 172.21.15.45:0/1048360977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f959c02de40 con 0x7f95b4104ea0 2024-01-31T05:40:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.881+0000 7f95b2ffd640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f95a00a1450 0x7f95a00a3910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f95a40058c0 tx=0x7f95a400a2d0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:18.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:18.885+0000 7f95b0ff9640 1 -- 172.21.15.45:0/1048360977 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f959c0b8850 con 0x7f95b4104ea0 2024-01-31T05:40:19.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.069+0000 7f95b97c4640 1 -- 172.21.15.45:0/1048360977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95b41081d0 con 0x7f95b4104ea0 2024-01-31T05:40:19.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.069+0000 7f95b0ff9640 1 -- 172.21.15.45:0/1048360977 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f959c0bbe70 con 0x7f95b4104ea0 2024-01-31T05:40:19.070 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-01-31T05:40:19.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.071+0000 7f95b97c4640 1 -- 172.21.15.45:0/1048360977 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f95a00a1450 msgr2=0x7f95a00a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.071+0000 7f95b97c4640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f95a00a1450 0x7f95a00a3910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f95a40058c0 tx=0x7f95a400a2d0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.071+0000 7f95b97c4640 1 -- 172.21.15.45:0/1048360977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b4104ea0 msgr2=0x7f95b41985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.071+0000 7f95b97c4640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b4104ea0 0x7f95b41985a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f959c009230 tx=0x7f959c02cd10 comp rx=0 tx=0).stop 2024-01-31T05:40:19.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.071+0000 7f95b97c4640 1 -- 172.21.15.45:0/1048360977 shutdown_connections 2024-01-31T05:40:19.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.071+0000 7f95b97c4640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f95a00a1450 0x7f95a00a3910 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.071+0000 7f95b97c4640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b4104ea0 0x7f95b41985a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.071+0000 7f95b97c4640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95b4198b80 0x7f95b41a30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.071+0000 7f95b97c4640 1 --2- 172.21.15.45:0/1048360977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95b40ffdd0 0x7f95b4198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.071+0000 7f95b97c4640 1 -- 172.21.15.45:0/1048360977 >> 172.21.15.45:0/1048360977 conn(0x7f95b4076cf0 msgr2=0x7f95b40fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:19.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.071+0000 7f95b97c4640 1 -- 172.21.15.45:0/1048360977 shutdown_connections 2024-01-31T05:40:19.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.071+0000 7f95b97c4640 1 -- 172.21.15.45:0/1048360977 wait complete. 2024-01-31T05:40:19.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:18 smithi045 ceph-mon[31585]: pgmap v561: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 244 op/s 2024-01-31T05:40:19.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:18 smithi045 ceph-mon[31585]: from='client.16818 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:40:19.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:18 smithi045 ceph-mon[31585]: from='client.16824 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:40:19.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:18 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/622840625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:40:19.083 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:40:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.188+0000 7f725aa99640 1 -- 172.21.15.45:0/501113424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7254069fa0 msgr2=0x7f72541058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.188+0000 7f725aa99640 1 --2- 172.21.15.45:0/501113424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7254069fa0 0x7f72541058e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7240009930 tx=0x7f724002bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.188+0000 7f725aa99640 1 -- 172.21.15.45:0/501113424 shutdown_connections 2024-01-31T05:40:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.188+0000 7f725aa99640 1 --2- 172.21.15.45:0/501113424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7254069fa0 0x7f72541058e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.188+0000 7f725aa99640 1 --2- 172.21.15.45:0/501113424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7254105fc0 0x7f725410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.188+0000 7f725aa99640 1 --2- 172.21.15.45:0/501113424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72540695f0 0x7f72540699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.188+0000 7f725aa99640 1 -- 172.21.15.45:0/501113424 >> 172.21.15.45:0/501113424 conn(0x7f7254076cc0 msgr2=0x7f72540770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.188+0000 7f725aa99640 1 -- 172.21.15.45:0/501113424 shutdown_connections 2024-01-31T05:40:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.188+0000 7f725aa99640 1 -- 172.21.15.45:0/501113424 wait complete. 2024-01-31T05:40:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f725aa99640 1 Processor -- start 2024-01-31T05:40:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f725aa99640 1 -- start start 2024-01-31T05:40:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f725aa99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72540695f0 0x7f7254197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f725aa99640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7254105fc0 0x7f72541977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f725aa99640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f725419bb20 0x7f725419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f725aa99640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725419c510 con 0x7f72540695f0 2024-01-31T05:40:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f725aa99640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725419c680 con 0x7f7254105fc0 2024-01-31T05:40:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f725aa99640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725419c7f0 con 0x7f725419bb20 2024-01-31T05:40:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f7258835640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72540695f0 0x7f7254197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f7258835640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72540695f0 0x7f7254197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52108/0 (socket says 172.21.15.45:52108) 2024-01-31T05:40:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f7258835640 1 -- 172.21.15.45:0/1426239174 learned_addr learned my addr 172.21.15.45:0/1426239174 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f7253fff640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7254105fc0 0x7f72541977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:19.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f7259036640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f725419bb20 0x7f725419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f7258835640 1 -- 172.21.15.45:0/1426239174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f725419bb20 msgr2=0x7f725419bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f7258835640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f725419bb20 0x7f725419bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f7258835640 1 -- 172.21.15.45:0/1426239174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7254105fc0 msgr2=0x7f72541977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f7258835640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7254105fc0 0x7f72541977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.189+0000 7f7258835640 1 -- 172.21.15.45:0/1426239174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7240009560 con 0x7f72540695f0 2024-01-31T05:40:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.190+0000 7f7258835640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72540695f0 0x7f7254197260 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f724800e9f0 tx=0x7f724800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.190+0000 7f7251ffb640 1 -- 172.21.15.45:0/1426239174 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248009700 con 0x7f72540695f0 2024-01-31T05:40:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.190+0000 7f7251ffb640 1 -- 172.21.15.45:0/1426239174 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f724800cd80 con 0x7f72540695f0 2024-01-31T05:40:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.190+0000 7f725aa99640 1 -- 172.21.15.45:0/1426239174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72541a3dd0 con 0x7f72540695f0 2024-01-31T05:40:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.190+0000 7f7251ffb640 1 -- 172.21.15.45:0/1426239174 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248018860 con 0x7f72540695f0 2024-01-31T05:40:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.190+0000 7f725aa99640 1 -- 172.21.15.45:0/1426239174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72541a41f0 con 0x7f72540695f0 2024-01-31T05:40:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.191+0000 7f725aa99640 1 -- 172.21.15.45:0/1426239174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72541023c0 con 0x7f72540695f0 2024-01-31T05:40:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.192+0000 7f7251ffb640 1 -- 172.21.15.45:0/1426239174 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f72480189c0 con 0x7f72540695f0 2024-01-31T05:40:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.193+0000 7f7251ffb640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f72440a1450 0x7f72440a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.193+0000 7f7253fff640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f72440a1450 0x7f72440a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.193+0000 7f7251ffb640 1 -- 172.21.15.45:0/1426239174 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7248014070 con 0x7f72540695f0 2024-01-31T05:40:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.193+0000 7f7253fff640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f72440a1450 0x7f72440a3910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7240000c00 tx=0x7f7240005fd0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:19.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.197+0000 7f7251ffb640 1 -- 172.21.15.45:0/1426239174 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7248052600 con 0x7f72540695f0 2024-01-31T05:40:19.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:18 smithi156 ceph-mon[35720]: pgmap v561: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 244 op/s 2024-01-31T05:40:19.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:18 smithi156 ceph-mon[35720]: from='client.16818 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:40:19.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:18 smithi156 ceph-mon[35720]: from='client.16824 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:40:19.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:18 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/622840625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:40:19.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:18 smithi186 ceph-mon[35659]: pgmap v561: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 244 op/s 2024-01-31T05:40:19.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:18 smithi186 ceph-mon[35659]: from='client.16818 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:40:19.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:18 smithi186 ceph-mon[35659]: from='client.16824 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-31T05:40:19.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:18 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/622840625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:40:19.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.377+0000 7f725aa99640 1 -- 172.21.15.45:0/1426239174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72541985b0 con 0x7f72540695f0 2024-01-31T05:40:19.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.377+0000 7f7251ffb640 1 -- 172.21.15.45:0/1426239174 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7248096420 con 0x7f72540695f0 2024-01-31T05:40:19.377 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-01-31T05:40:19.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.379+0000 7f725aa99640 1 -- 172.21.15.45:0/1426239174 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f72440a1450 msgr2=0x7f72440a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.379+0000 7f725aa99640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f72440a1450 0x7f72440a3910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7240000c00 tx=0x7f7240005fd0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.379+0000 7f725aa99640 1 -- 172.21.15.45:0/1426239174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72540695f0 msgr2=0x7f7254197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.379+0000 7f725aa99640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72540695f0 0x7f7254197260 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f724800e9f0 tx=0x7f724800eeb0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.379+0000 7f725aa99640 1 -- 172.21.15.45:0/1426239174 shutdown_connections 2024-01-31T05:40:19.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.379+0000 7f725aa99640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f72440a1450 0x7f72440a3910 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.379+0000 7f725aa99640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f725419bb20 0x7f725419bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.379+0000 7f725aa99640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7254105fc0 0x7f72541977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.379+0000 7f725aa99640 1 --2- 172.21.15.45:0/1426239174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72540695f0 0x7f7254197260 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.379+0000 7f725aa99640 1 -- 172.21.15.45:0/1426239174 >> 172.21.15.45:0/1426239174 conn(0x7f7254076cc0 msgr2=0x7f72540fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:19.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.379+0000 7f725aa99640 1 -- 172.21.15.45:0/1426239174 shutdown_connections 2024-01-31T05:40:19.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.379+0000 7f725aa99640 1 -- 172.21.15.45:0/1426239174 wait complete. 2024-01-31T05:40:19.390 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:40:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.495+0000 7feb83775640 1 -- 172.21.15.45:0/1536124660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb7c1018a0 msgr2=0x7feb7c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.495+0000 7feb83775640 1 --2- 172.21.15.45:0/1536124660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb7c1018a0 0x7feb7c1058e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009930 tx=0x7feb6c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.495+0000 7feb83775640 1 -- 172.21.15.45:0/1536124660 shutdown_connections 2024-01-31T05:40:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.495+0000 7feb83775640 1 --2- 172.21.15.45:0/1536124660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb7c1018a0 0x7feb7c1058e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.495+0000 7feb83775640 1 --2- 172.21.15.45:0/1536124660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb7c105fc0 0x7feb7c10db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.495+0000 7feb83775640 1 --2- 172.21.15.45:0/1536124660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb7c100ef0 0x7feb7c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.495+0000 7feb83775640 1 -- 172.21.15.45:0/1536124660 >> 172.21.15.45:0/1536124660 conn(0x7feb7c0fc790 msgr2=0x7feb7c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb83775640 1 -- 172.21.15.45:0/1536124660 shutdown_connections 2024-01-31T05:40:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb83775640 1 -- 172.21.15.45:0/1536124660 wait complete. 2024-01-31T05:40:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb83775640 1 Processor -- start 2024-01-31T05:40:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb83775640 1 -- start start 2024-01-31T05:40:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb83775640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb7c100ef0 0x7feb7c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb83775640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb7c105fc0 0x7feb7c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb83775640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb7c19bb70 0x7feb7c19c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb83775640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c19c560 con 0x7feb7c105fc0 2024-01-31T05:40:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb83775640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c19c6d0 con 0x7feb7c100ef0 2024-01-31T05:40:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb83775640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c19c840 con 0x7feb7c19bb70 2024-01-31T05:40:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb80d10640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb7c105fc0 0x7feb7c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb80d10640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb7c105fc0 0x7feb7c197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52124/0 (socket says 172.21.15.45:52124) 2024-01-31T05:40:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb80d10640 1 -- 172.21.15.45:0/4022256762 learned_addr learned my addr 172.21.15.45:0/4022256762 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb81511640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb7c100ef0 0x7feb7c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.496+0000 7feb81d12640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb7c19bb70 0x7feb7c19c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.497+0000 7feb80d10640 1 -- 172.21.15.45:0/4022256762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb7c19bb70 msgr2=0x7feb7c19c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.497+0000 7feb80d10640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb7c19bb70 0x7feb7c19c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.497+0000 7feb80d10640 1 -- 172.21.15.45:0/4022256762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb7c100ef0 msgr2=0x7feb7c197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.497+0000 7feb80d10640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb7c100ef0 0x7feb7c197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.497+0000 7feb80d10640 1 -- 172.21.15.45:0/4022256762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6c009560 con 0x7feb7c105fc0 2024-01-31T05:40:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.497+0000 7feb80d10640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb7c105fc0 0x7feb7c197840 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009230 tx=0x7feb6c02cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.497+0000 7feb727fc640 1 -- 172.21.15.45:0/4022256762 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c004390 con 0x7feb7c105fc0 2024-01-31T05:40:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.497+0000 7feb727fc640 1 -- 172.21.15.45:0/4022256762 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb6c0044f0 con 0x7feb7c105fc0 2024-01-31T05:40:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.497+0000 7feb83775640 1 -- 172.21.15.45:0/4022256762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7c1a3e20 con 0x7feb7c105fc0 2024-01-31T05:40:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.497+0000 7feb727fc640 1 -- 172.21.15.45:0/4022256762 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c003db0 con 0x7feb7c105fc0 2024-01-31T05:40:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.497+0000 7feb83775640 1 -- 172.21.15.45:0/4022256762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c1a4270 con 0x7feb7c105fc0 2024-01-31T05:40:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.498+0000 7feb83775640 1 -- 172.21.15.45:0/4022256762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7c1023c0 con 0x7feb7c105fc0 2024-01-31T05:40:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.499+0000 7feb727fc640 1 -- 172.21.15.45:0/4022256762 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7feb6c005000 con 0x7feb7c105fc0 2024-01-31T05:40:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.500+0000 7feb727fc640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7feb680a14d0 0x7feb680a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.500+0000 7feb81511640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7feb680a14d0 0x7feb680a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.500+0000 7feb727fc640 1 -- 172.21.15.45:0/4022256762 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7feb6c0efa60 con 0x7feb7c105fc0 2024-01-31T05:40:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.500+0000 7feb81511640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7feb680a14d0 0x7feb680a3990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feb74005fd0 tx=0x7feb74009760 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.503+0000 7feb727fc640 1 -- 172.21.15.45:0/4022256762 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7feb6c0b8cb0 con 0x7feb7c105fc0 2024-01-31T05:40:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.683+0000 7feb83775640 1 -- 172.21.15.45:0/4022256762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feb7c198600 con 0x7feb7c105fc0 2024-01-31T05:40:19.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.683+0000 7feb727fc640 1 -- 172.21.15.45:0/4022256762 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+11 (secure 0 0 0) 0x7feb6c0bc2d0 con 0x7feb7c105fc0 2024-01-31T05:40:19.683 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-01-31T05:40:19.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.685+0000 7feb83775640 1 -- 172.21.15.45:0/4022256762 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7feb680a14d0 msgr2=0x7feb680a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.685+0000 7feb83775640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7feb680a14d0 0x7feb680a3990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feb74005fd0 tx=0x7feb74009760 comp rx=0 tx=0).stop 2024-01-31T05:40:19.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.685+0000 7feb83775640 1 -- 172.21.15.45:0/4022256762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb7c105fc0 msgr2=0x7feb7c197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.685+0000 7feb83775640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb7c105fc0 0x7feb7c197840 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009230 tx=0x7feb6c02cd00 comp rx=0 tx=0).stop 2024-01-31T05:40:19.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.685+0000 7feb83775640 1 -- 172.21.15.45:0/4022256762 shutdown_connections 2024-01-31T05:40:19.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.685+0000 7feb83775640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7feb680a14d0 0x7feb680a3990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.685+0000 7feb83775640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb7c105fc0 0x7feb7c197840 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.685+0000 7feb83775640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb7c19bb70 0x7feb7c19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.685+0000 7feb83775640 1 --2- 172.21.15.45:0/4022256762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb7c100ef0 0x7feb7c197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.685+0000 7feb83775640 1 -- 172.21.15.45:0/4022256762 >> 172.21.15.45:0/4022256762 conn(0x7feb7c0fc790 msgr2=0x7feb7c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:19.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.685+0000 7feb83775640 1 -- 172.21.15.45:0/4022256762 shutdown_connections 2024-01-31T05:40:19.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.685+0000 7feb83775640 1 -- 172.21.15.45:0/4022256762 wait complete. 2024-01-31T05:40:19.696 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-01-31T05:40:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.799+0000 7fd001fd0640 1 -- 172.21.15.45:0/773476007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcffc0ffe60 msgr2=0x7fcffc100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.799+0000 7fd001fd0640 1 --2- 172.21.15.45:0/773476007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcffc0ffe60 0x7fcffc100240 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009930 tx=0x7fcfec02bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.799+0000 7fd001fd0640 1 -- 172.21.15.45:0/773476007 shutdown_connections 2024-01-31T05:40:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.799+0000 7fd001fd0640 1 --2- 172.21.15.45:0/773476007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcffc104f30 0x7fcffc10cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.799+0000 7fd001fd0640 1 --2- 172.21.15.45:0/773476007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcffc100810 0x7fcffc104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.799+0000 7fd001fd0640 1 --2- 172.21.15.45:0/773476007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcffc0ffe60 0x7fcffc100240 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.799+0000 7fd001fd0640 1 -- 172.21.15.45:0/773476007 >> 172.21.15.45:0/773476007 conn(0x7fcffc076cf0 msgr2=0x7fcffc077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.799+0000 7fd001fd0640 1 -- 172.21.15.45:0/773476007 shutdown_connections 2024-01-31T05:40:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.799+0000 7fd001fd0640 1 -- 172.21.15.45:0/773476007 wait complete. 2024-01-31T05:40:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.800+0000 7fd001fd0640 1 Processor -- start 2024-01-31T05:40:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.800+0000 7fd001fd0640 1 -- start start 2024-01-31T05:40:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.800+0000 7fd001fd0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcffc0ffe60 0x7fcffc198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.800+0000 7fd001fd0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcffc104f30 0x7fcffc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.800+0000 7fd001fd0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcffc198c20 0x7fcffc1a31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.800+0000 7fd001fd0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc1a3870 con 0x7fcffc104f30 2024-01-31T05:40:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.800+0000 7fd001fd0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc1a39e0 con 0x7fcffc198c20 2024-01-31T05:40:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.800+0000 7fd001fd0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc1a3ba0 con 0x7fcffc0ffe60 2024-01-31T05:40:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.800+0000 7fcff2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcffc104f30 0x7fcffc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcff2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcffc104f30 0x7fcffc198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52144/0 (socket says 172.21.15.45:52144) 2024-01-31T05:40:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcff2ffd640 1 -- 172.21.15.45:0/2555775095 learned_addr learned my addr 172.21.15.45:0/2555775095 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcffb7fe640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcffc0ffe60 0x7fcffc198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcffbfff640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcffc198c20 0x7fcffc1a31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcff2ffd640 1 -- 172.21.15.45:0/2555775095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcffc0ffe60 msgr2=0x7fcffc198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcff2ffd640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcffc0ffe60 0x7fcffc198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcff2ffd640 1 -- 172.21.15.45:0/2555775095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcffc198c20 msgr2=0x7fcffc1a31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcff2ffd640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcffc198c20 0x7fcffc1a31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcff2ffd640 1 -- 172.21.15.45:0/2555775095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec009560 con 0x7fcffc104f30 2024-01-31T05:40:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcffbfff640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcffc198c20 0x7fcffc1a31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:40:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcff2ffd640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcffc104f30 0x7fcffc198690 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400d740 tx=0x7fcfe400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcff97fa640 1 -- 172.21.15.45:0/2555775095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4009c30 con 0x7fcffc104f30 2024-01-31T05:40:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcff97fa640 1 -- 172.21.15.45:0/2555775095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfe400de50 con 0x7fcffc104f30 2024-01-31T05:40:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fd001fd0640 1 -- 172.21.15.45:0/2555775095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc1a3e80 con 0x7fcffc104f30 2024-01-31T05:40:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fcff97fa640 1 -- 172.21.15.45:0/2555775095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4004ea0 con 0x7fcffc104f30 2024-01-31T05:40:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.801+0000 7fd001fd0640 1 -- 172.21.15.45:0/2555775095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc1a42b0 con 0x7fcffc104f30 2024-01-31T05:40:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.802+0000 7fd001fd0640 1 -- 172.21.15.45:0/2555775095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcffc067ae0 con 0x7fcffc104f30 2024-01-31T05:40:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.803+0000 7fcff97fa640 1 -- 172.21.15.45:0/2555775095 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fcfe4005000 con 0x7fcffc104f30 2024-01-31T05:40:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.804+0000 7fcff97fa640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fcfe80a1450 0x7fcfe80a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.804+0000 7fcff97fa640 1 -- 172.21.15.45:0/2555775095 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fcfe40ccd80 con 0x7fcffc104f30 2024-01-31T05:40:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.804+0000 7fcffb7fe640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fcfe80a1450 0x7fcfe80a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.804+0000 7fcffb7fe640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fcfe80a1450 0x7fcfe80a3910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006010 tx=0x7fcfec004e90 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:19.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:19.808+0000 7fcff97fa640 1 -- 172.21.15.45:0/2555775095 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fcfe4095e60 con 0x7fcffc104f30 2024-01-31T05:40:20.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.044+0000 7fd001fd0640 1 -- 172.21.15.45:0/2555775095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fcffc108190 con 0x7fcffc104f30 2024-01-31T05:40:20.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.052+0000 7fcff97fa640 1 -- 172.21.15.45:0/2555775095 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+732996 (secure 0 0 0) 0x7fcfe4020090 con 0x7fcffc104f30 2024-01-31T05:40:20.063 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:40:20.063 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-01-31T05:40:20.063 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 26, 2024-01-31T05:40:20.063 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.064 INFO:teuthology.orchestra.run.smithi045.stdout: "active_gid": 15315, 2024-01-31T05:40:20.064 INFO:teuthology.orchestra.run.smithi045.stdout: "active_name": "x", 2024-01-31T05:40:20.064 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addrs": { 2024-01-31T05:40:20.064 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-01-31T05:40:20.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.064 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-01-31T05:40:20.064 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2024-01-31T05:40:20.064 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1984996061 2024-01-31T05:40:20.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-01-31T05:40:20.065 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2024-01-31T05:40:20.065 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1984996061 2024-01-31T05:40:20.065 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.065 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-01-31T05:40:20.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.065 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addr": "172.21.15.45:6801/1984996061", 2024-01-31T05:40:20.065 INFO:teuthology.orchestra.run.smithi045.stdout: "active_change": "2024-01-31T05:22:41.447947+0000", 2024-01-31T05:40:20.066 INFO:teuthology.orchestra.run.smithi045.stdout: "active_mgr_features": 4540701272860131327, 2024-01-31T05:40:20.066 INFO:teuthology.orchestra.run.smithi045.stdout: "available": true, 2024-01-31T05:40:20.066 INFO:teuthology.orchestra.run.smithi045.stdout: "standbys": [ 2024-01-31T05:40:20.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.066 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 24863, 2024-01-31T05:40:20.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "y", 2024-01-31T05:40:20.066 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540701272860131327, 2024-01-31T05:40:20.066 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-01-31T05:40:20.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-01-31T05:40:20.067 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.067 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.067 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.067 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:20.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:20.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.068 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:20.068 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.068 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.068 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.068 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-01-31T05:40:20.068 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.068 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.068 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.069 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.069 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.069 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.069 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.069 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.069 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.070 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.070 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.070 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.070 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.070 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.070 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.070 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.070 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.070 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.071 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.071 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.071 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.071 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.072 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.072 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.073 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.073 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.073 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.073 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.073 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.073 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.074 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.074 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.074 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.074 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.074 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.074 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.074 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.074 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.074 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.075 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.075 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.075 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.075 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.075 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.075 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.076 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.076 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.076 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.076 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.076 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.076 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.076 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.077 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-01-31T05:40:20.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-01-31T05:40:20.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.077 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.077 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.077 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.077 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.077 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.077 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.078 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-01-31T05:40:20.078 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.078 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.078 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.078 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-01-31T05:40:20.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-01-31T05:40:20.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.079 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-01-31T05:40:20.079 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.079 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-01-31T05:40:20.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.080 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.080 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-01-31T05:40:20.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-01-31T05:40:20.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.080 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.080 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.080 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.081 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.081 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-01-31T05:40:20.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.081 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-01-31T05:40:20.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-01-31T05:40:20.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.082 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-01-31T05:40:20.083 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.083 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.083 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.083 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-01-31T05:40:20.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-01-31T05:40:20.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.083 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.083 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.084 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-01-31T05:40:20.084 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.084 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.084 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.084 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-01-31T05:40:20.084 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.084 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.085 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-01-31T05:40:20.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-01-31T05:40:20.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.085 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.085 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.085 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.085 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.085 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.085 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.086 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-01-31T05:40:20.086 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.086 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.086 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.086 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-01-31T05:40:20.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-01-31T05:40:20.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.086 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.087 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.087 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.087 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.087 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.087 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.087 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-01-31T05:40:20.087 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.088 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-01-31T05:40:20.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-01-31T05:40:20.088 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.088 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.088 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.088 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.088 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.089 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.089 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.089 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-01-31T05:40:20.089 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.089 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.089 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.089 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.089 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-01-31T05:40:20.090 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.090 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.090 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.090 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-01-31T05:40:20.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-01-31T05:40:20.090 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.090 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.091 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.091 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.091 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.091 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.091 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.091 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-01-31T05:40:20.091 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.091 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.091 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.092 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-01-31T05:40:20.092 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-01-31T05:40:20.092 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.092 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.092 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.092 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-01-31T05:40:20.093 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.093 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.093 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.093 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-01-31T05:40:20.093 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-31T05:40:20.093 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.093 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.093 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-01-31T05:40:20.094 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-01-31T05:40:20.094 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:20.094 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.094 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.094 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:20.094 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-01-31T05:40:20.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-01-31T05:40:20.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.094 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-01-31T05:40:20.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-31T05:40:20.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.095 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-01-31T05:40:20.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-01-31T05:40:20.095 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:20.095 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.096 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-01-31T05:40:20.096 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-01-31T05:40:20.096 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-01-31T05:40:20.096 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.096 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-01-31T05:40:20.096 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.096 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.096 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.097 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-01-31T05:40:20.097 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-01-31T05:40:20.097 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.097 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.097 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.097 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-01-31T05:40:20.097 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.097 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.098 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-01-31T05:40:20.098 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-01-31T05:40:20.098 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.098 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.098 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-01-31T05:40:20.098 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-01-31T05:40:20.098 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:20.098 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.099 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.099 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-01-31T05:40:20.099 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-01-31T05:40:20.099 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-01-31T05:40:20.099 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.099 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-01-31T05:40:20.099 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-01-31T05:40:20.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.099 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.100 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-01-31T05:40:20.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-01-31T05:40:20.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.100 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.100 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-01-31T05:40:20.100 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.101 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.101 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-01-31T05:40:20.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-31T05:40:20.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.101 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.101 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-01-31T05:40:20.101 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-01-31T05:40:20.101 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:20.102 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.102 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:20.102 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-01-31T05:40:20.102 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-01-31T05:40:20.102 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.102 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-01-31T05:40:20.102 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-31T05:40:20.102 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.103 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.103 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.103 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.103 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.103 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.103 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.103 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.104 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.104 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.104 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.104 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.104 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.104 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.104 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.105 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.105 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.105 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.105 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.105 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.105 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.106 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.106 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.106 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.106 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.106 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.107 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.107 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.107 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.107 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.107 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.108 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.108 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.108 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.108 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.108 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.108 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.108 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.108 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.109 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.109 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.109 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.109 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.109 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.109 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.109 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.109 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.110 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.110 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.110 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.110 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.110 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.110 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.110 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.111 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.111 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.111 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.111 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-01-31T05:40:20.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-01-31T05:40:20.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:20.111 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.112 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.112 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:20.112 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.112 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.112 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.112 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-01-31T05:40:20.112 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.113 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.113 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.113 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-01-31T05:40:20.113 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-01-31T05:40:20.113 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.113 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.113 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-01-31T05:40:20.114 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.114 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.114 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.114 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-01-31T05:40:20.114 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-01-31T05:40:20.114 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap" 2024-01-31T05:40:20.114 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.114 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-01-31T05:40:20.114 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.115 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.115 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-01-31T05:40:20.115 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-01-31T05:40:20.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.115 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.116 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.116 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.116 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.116 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.116 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-01-31T05:40:20.116 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.116 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.116 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.117 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:20.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:20.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.117 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.117 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.117 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:20.117 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.117 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.118 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.118 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-01-31T05:40:20.118 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.118 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.118 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.118 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-01-31T05:40:20.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-01-31T05:40:20.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.118 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.119 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-01-31T05:40:20.119 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-01-31T05:40:20.119 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.119 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.119 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-01-31T05:40:20.119 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-01-31T05:40:20.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.120 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-01-31T05:40:20.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-01-31T05:40:20.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:20.120 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.120 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.121 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.121 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.121 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-01-31T05:40:20.121 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.121 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.121 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.121 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.121 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-01-31T05:40:20.122 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.122 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.122 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.122 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-01-31T05:40:20.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-01-31T05:40:20.122 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:20.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.123 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-01-31T05:40:20.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.123 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.123 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.123 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-01-31T05:40:20.123 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.124 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.124 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-01-31T05:40:20.124 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-01-31T05:40:20.124 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.124 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.124 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.124 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-01-31T05:40:20.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-01-31T05:40:20.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.125 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.125 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-01-31T05:40:20.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-01-31T05:40:20.126 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.126 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.126 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.126 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-01-31T05:40:20.126 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.126 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.126 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.126 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-01-31T05:40:20.127 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.127 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.127 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-01-31T05:40:20.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-01-31T05:40:20.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.127 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.128 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.128 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.128 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.128 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.128 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-01-31T05:40:20.128 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-01-31T05:40:20.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.129 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.129 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-01-31T05:40:20.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-01-31T05:40:20.129 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.129 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.129 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:20.130 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.130 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.130 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-01-31T05:40:20.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.130 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-01-31T05:40:20.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-01-31T05:40:20.131 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:20.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-01-31T05:40:20.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.131 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.131 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.131 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-01-31T05:40:20.131 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.132 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.132 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-01-31T05:40:20.132 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-01-31T05:40:20.132 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.132 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.132 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.133 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.133 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.133 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.133 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.133 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-01-31T05:40:20.133 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-01-31T05:40:20.133 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-01-31T05:40:20.133 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.134 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-01-31T05:40:20.134 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.134 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.134 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.134 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-01-31T05:40:20.134 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-01-31T05:40:20.134 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.134 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.135 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.135 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.135 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.135 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.135 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-01-31T05:40:20.135 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.135 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.135 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.136 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-01-31T05:40:20.136 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-01-31T05:40:20.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.136 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.136 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.137 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.137 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.137 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.137 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-01-31T05:40:20.137 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.137 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.137 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.138 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-01-31T05:40:20.138 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-01-31T05:40:20.138 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.138 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.138 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.138 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.138 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.138 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.138 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.139 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-01-31T05:40:20.139 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.139 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.139 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.139 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-01-31T05:40:20.139 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-01-31T05:40:20.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.140 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.140 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.140 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-01-31T05:40:20.140 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.140 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.140 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-01-31T05:40:20.140 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.141 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.141 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.141 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-01-31T05:40:20.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-01-31T05:40:20.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.141 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.141 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.141 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-01-31T05:40:20.142 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.142 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.142 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.142 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-01-31T05:40:20.142 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.142 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.142 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-01-31T05:40:20.143 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-01-31T05:40:20.143 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.143 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.143 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.143 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-01-31T05:40:20.143 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.143 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.144 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-01-31T05:40:20.144 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.144 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.144 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.144 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-01-31T05:40:20.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-01-31T05:40:20.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-01-31T05:40:20.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.145 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.145 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-01-31T05:40:20.145 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.146 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.146 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.146 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-01-31T05:40:20.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-01-31T05:40:20.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.146 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.146 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.146 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-01-31T05:40:20.147 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.147 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.147 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.147 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-01-31T05:40:20.147 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.147 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.147 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.148 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-01-31T05:40:20.148 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-01-31T05:40:20.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.148 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.148 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.148 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-01-31T05:40:20.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.149 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.149 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-01-31T05:40:20.149 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.149 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.149 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-01-31T05:40:20.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-01-31T05:40:20.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.150 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.150 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.150 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-01-31T05:40:20.150 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.150 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.150 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-01-31T05:40:20.151 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.151 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.151 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-01-31T05:40:20.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-01-31T05:40:20.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.152 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.152 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.152 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-01-31T05:40:20.152 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.152 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.152 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-01-31T05:40:20.153 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.153 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.153 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-01-31T05:40:20.153 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-01-31T05:40:20.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.153 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.154 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.154 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-01-31T05:40:20.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.154 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.154 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.154 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-01-31T05:40:20.154 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.154 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.155 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-01-31T05:40:20.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-01-31T05:40:20.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.155 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.155 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-01-31T05:40:20.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.156 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-01-31T05:40:20.156 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.156 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.156 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.156 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-01-31T05:40:20.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-01-31T05:40:20.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.157 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.157 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.157 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-01-31T05:40:20.157 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.157 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.157 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.158 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-01-31T05:40:20.158 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.158 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.158 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.158 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-01-31T05:40:20.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-01-31T05:40:20.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.158 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.159 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.159 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-01-31T05:40:20.159 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.159 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.159 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.159 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-01-31T05:40:20.159 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.159 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.160 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.160 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-01-31T05:40:20.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-01-31T05:40:20.160 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.160 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.160 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.160 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-01-31T05:40:20.161 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.161 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.161 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.161 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-01-31T05:40:20.161 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.161 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.161 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-01-31T05:40:20.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-01-31T05:40:20.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.162 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.162 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.162 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-01-31T05:40:20.162 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.162 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.162 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.163 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-01-31T05:40:20.163 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.163 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.163 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.163 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-01-31T05:40:20.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-01-31T05:40:20.163 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.164 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.164 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.164 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-01-31T05:40:20.164 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.164 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.164 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.164 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-01-31T05:40:20.164 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.164 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.165 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.165 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-01-31T05:40:20.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-01-31T05:40:20.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.165 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.165 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.165 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.166 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.166 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.166 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.166 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-01-31T05:40:20.166 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.166 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.166 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.167 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-01-31T05:40:20.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-01-31T05:40:20.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.167 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.167 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.167 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:20.167 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.167 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.168 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.168 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-01-31T05:40:20.168 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.168 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.168 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.168 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-01-31T05:40:20.168 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-01-31T05:40:20.169 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.169 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.169 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.169 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-01-31T05:40:20.169 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.169 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.169 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.169 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-01-31T05:40:20.170 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.170 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.170 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.170 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-01-31T05:40:20.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-01-31T05:40:20.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.170 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.170 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.171 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-01-31T05:40:20.171 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.171 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.171 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.171 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-01-31T05:40:20.171 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.171 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.171 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.172 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-01-31T05:40:20.172 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-01-31T05:40:20.172 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.172 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.172 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.172 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-01-31T05:40:20.173 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.173 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.173 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.173 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-01-31T05:40:20.173 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.173 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.173 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.173 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-01-31T05:40:20.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-01-31T05:40:20.174 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.174 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.174 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.174 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.174 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.174 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.174 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.174 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-01-31T05:40:20.174 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.175 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.175 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.175 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-01-31T05:40:20.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-01-31T05:40:20.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.175 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.175 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.175 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:20.176 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.176 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.176 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.176 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-01-31T05:40:20.176 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.176 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.176 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.177 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-01-31T05:40:20.177 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-01-31T05:40:20.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.177 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.177 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:20.177 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.177 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.177 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.178 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-01-31T05:40:20.178 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.178 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.178 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.178 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-01-31T05:40:20.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-01-31T05:40:20.178 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.179 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.179 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.179 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.179 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.179 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.179 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.179 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-01-31T05:40:20.179 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.180 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.181 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-01-31T05:40:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-01-31T05:40:20.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-01-31T05:40:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.183 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.184 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-01-31T05:40:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.185 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.186 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.187 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-01-31T05:40:20.189 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-01-31T05:40:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-01-31T05:40:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.190 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-01-31T05:40:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-01-31T05:40:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-01-31T05:40:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-01-31T05:40:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-01-31T05:40:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-01-31T05:40:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.193 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-01-31T05:40:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-01-31T05:40:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-01-31T05:40:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.194 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-01-31T05:40:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.195 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-01-31T05:40:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-01-31T05:40:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.196 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-01-31T05:40:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-01-31T05:40:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-01-31T05:40:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.197 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-01-31T05:40:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-01-31T05:40:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-01-31T05:40:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.198 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-01-31T05:40:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-01-31T05:40:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-01-31T05:40:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-01-31T05:40:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-01-31T05:40:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.200 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-01-31T05:40:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-01-31T05:40:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.201 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-01-31T05:40:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-01-31T05:40:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-01-31T05:40:20.202 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.203 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.203 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.203 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.203 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.203 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.203 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-01-31T05:40:20.203 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.203 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.204 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.204 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-01-31T05:40:20.204 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-01-31T05:40:20.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.204 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.205 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.205 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.205 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.205 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.205 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-01-31T05:40:20.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.205 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.206 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-01-31T05:40:20.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-01-31T05:40:20.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.206 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.206 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.206 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.206 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.206 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.206 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.206 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-01-31T05:40:20.207 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.207 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.207 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.207 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-01-31T05:40:20.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-01-31T05:40:20.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.207 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.208 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.208 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.208 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.208 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.208 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-01-31T05:40:20.208 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.208 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.209 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-01-31T05:40:20.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-01-31T05:40:20.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.209 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.209 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.209 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-01-31T05:40:20.209 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.210 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.210 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-01-31T05:40:20.210 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.210 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.210 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.210 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-01-31T05:40:20.210 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-01-31T05:40:20.210 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.211 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.211 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.211 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.211 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.211 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.211 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.211 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-01-31T05:40:20.211 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.212 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.212 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-01-31T05:40:20.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-01-31T05:40:20.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.212 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.212 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.213 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-01-31T05:40:20.213 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.213 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.213 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-01-31T05:40:20.213 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-01-31T05:40:20.214 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.214 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.214 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.214 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.214 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.214 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.214 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.214 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-01-31T05:40:20.214 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.215 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.215 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-01-31T05:40:20.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-01-31T05:40:20.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.215 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.215 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.216 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.216 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.216 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.216 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-01-31T05:40:20.216 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.216 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.216 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.217 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-01-31T05:40:20.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-01-31T05:40:20.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.217 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.217 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.217 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.217 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.217 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.218 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-01-31T05:40:20.218 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.218 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.218 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-01-31T05:40:20.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-01-31T05:40:20.218 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.219 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.219 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.219 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.219 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.219 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-01-31T05:40:20.219 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.219 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.220 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.220 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.220 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.220 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-01-31T05:40:20.220 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.220 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.221 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.221 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.221 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.221 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.221 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.221 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.221 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.222 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.222 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.222 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.222 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.222 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.222 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.222 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.222 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.222 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.223 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.223 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.223 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.223 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.224 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.224 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.224 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.224 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.224 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.224 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.225 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.225 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.225 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.225 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.226 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.226 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.226 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.226 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.226 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.226 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.226 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.226 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.226 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.227 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.227 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.227 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.227 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.227 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.227 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.227 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.228 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.228 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.228 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.228 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.228 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.228 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.228 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.229 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-01-31T05:40:20.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-01-31T05:40:20.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.229 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.229 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-01-31T05:40:20.229 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.229 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.230 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.230 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-01-31T05:40:20.230 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.230 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.230 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.230 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-01-31T05:40:20.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-01-31T05:40:20.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.231 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.231 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.231 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-01-31T05:40:20.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.231 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-01-31T05:40:20.231 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.231 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.232 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-01-31T05:40:20.232 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.232 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.232 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.233 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-01-31T05:40:20.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-01-31T05:40:20.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.233 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.233 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.233 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.233 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.234 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.234 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.234 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.234 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-01-31T05:40:20.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-01-31T05:40:20.234 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.235 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.235 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.235 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.235 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.235 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.235 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.236 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-01-31T05:40:20.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-01-31T05:40:20.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.236 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.236 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.236 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.237 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.237 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.237 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.237 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.237 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.237 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.237 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-01-31T05:40:20.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-01-31T05:40:20.238 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.238 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.238 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.238 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.238 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.238 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.238 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.238 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.239 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.239 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.239 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.239 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-01-31T05:40:20.239 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-01-31T05:40:20.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.239 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.239 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.240 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.240 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.240 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.240 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.240 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.240 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.240 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.240 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.241 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-01-31T05:40:20.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-01-31T05:40:20.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.241 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.241 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.241 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.241 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.241 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.242 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.242 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.242 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.242 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.242 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.242 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-01-31T05:40:20.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-01-31T05:40:20.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.243 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.243 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.243 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.243 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.243 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.243 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.243 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.243 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.244 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.244 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-01-31T05:40:20.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-01-31T05:40:20.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.244 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.245 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.245 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.245 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.245 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.245 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.245 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-01-31T05:40:20.245 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-01-31T05:40:20.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.246 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.246 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.246 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.246 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.246 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.246 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.246 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.246 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.247 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-01-31T05:40:20.247 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-01-31T05:40:20.247 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.247 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.247 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.247 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.248 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.248 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.248 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.248 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.248 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.248 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.248 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.249 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-01-31T05:40:20.249 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-01-31T05:40:20.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.249 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.249 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.249 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.249 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.250 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-01-31T05:40:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-01-31T05:40:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.251 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.252 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.252 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-01-31T05:40:20.252 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-01-31T05:40:20.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.253 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.253 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.253 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.254 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.254 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.254 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-01-31T05:40:20.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-01-31T05:40:20.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.254 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.254 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.254 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.255 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.255 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.255 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.255 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.255 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.255 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.256 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-01-31T05:40:20.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-01-31T05:40:20.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-01-31T05:40:20.256 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.257 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.257 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.257 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.257 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.257 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.257 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-01-31T05:40:20.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-01-31T05:40:20.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.258 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.258 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.258 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.258 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.258 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.258 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.259 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.259 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.259 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.259 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-01-31T05:40:20.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-01-31T05:40:20.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.260 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.260 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.260 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.260 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.260 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.260 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.261 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.261 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-01-31T05:40:20.261 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-01-31T05:40:20.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.261 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-01-31T05:40:20.262 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.262 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.262 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.262 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.262 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.263 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-01-31T05:40:20.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-01-31T05:40:20.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.263 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.263 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.263 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.263 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.264 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.264 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.264 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.264 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.264 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-01-31T05:40:20.264 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-01-31T05:40:20.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.265 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.265 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.265 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.265 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.265 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.265 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.265 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.266 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.266 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.266 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-01-31T05:40:20.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-01-31T05:40:20.266 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.267 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.267 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.267 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.267 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.268 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-01-31T05:40:20.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-01-31T05:40:20.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.268 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.268 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.269 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.269 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.269 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-01-31T05:40:20.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-01-31T05:40:20.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.270 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.270 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.270 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.270 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.270 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.271 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.271 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.271 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.271 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-01-31T05:40:20.271 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-01-31T05:40:20.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.272 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.272 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.272 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.272 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.273 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.273 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-01-31T05:40:20.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-01-31T05:40:20.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.273 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.273 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.273 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.273 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.274 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.274 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.274 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.274 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.274 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.274 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-01-31T05:40:20.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-01-31T05:40:20.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.275 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.275 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.275 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.276 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.276 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.276 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-01-31T05:40:20.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-01-31T05:40:20.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.276 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.276 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.277 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.277 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.277 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.277 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.277 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.277 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.278 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-01-31T05:40:20.278 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-01-31T05:40:20.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.278 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.278 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.278 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.278 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.279 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.279 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.279 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.279 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-01-31T05:40:20.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-01-31T05:40:20.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.280 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.280 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.280 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.280 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.280 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.280 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.280 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.281 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.281 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-01-31T05:40:20.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-01-31T05:40:20.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.281 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.281 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.282 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.282 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.282 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.282 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.282 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.282 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.282 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.282 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-01-31T05:40:20.283 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-01-31T05:40:20.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.283 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.283 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.284 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.284 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-01-31T05:40:20.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-01-31T05:40:20.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.284 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.285 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.285 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.285 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.285 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.285 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.285 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.285 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.286 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-01-31T05:40:20.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-01-31T05:40:20.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.286 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.286 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.286 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-01-31T05:40:20.286 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.286 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.286 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.287 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.287 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.287 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.287 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.287 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-01-31T05:40:20.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-01-31T05:40:20.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.287 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.288 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.288 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.288 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.288 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.288 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.288 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.288 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.289 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.289 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-01-31T05:40:20.289 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-01-31T05:40:20.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.289 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.289 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.289 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-01-31T05:40:20.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.290 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.290 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.290 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.290 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.290 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.290 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-01-31T05:40:20.291 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-01-31T05:40:20.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.291 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.291 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.291 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-01-31T05:40:20.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.291 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.291 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.292 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-01-31T05:40:20.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-01-31T05:40:20.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.292 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.293 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.293 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.293 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.293 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.293 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.294 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-01-31T05:40:20.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-01-31T05:40:20.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.294 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.294 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-01-31T05:40:20.294 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.294 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.294 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.295 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.295 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-01-31T05:40:20.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-01-31T05:40:20.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.295 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.296 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.296 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.296 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.296 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.296 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.296 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.296 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.297 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-01-31T05:40:20.297 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-01-31T05:40:20.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.297 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.298 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.298 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.298 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.298 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.298 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-01-31T05:40:20.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-01-31T05:40:20.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.299 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.299 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.299 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:20.299 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.299 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.299 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.299 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.300 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.300 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.300 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-01-31T05:40:20.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-01-31T05:40:20.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.300 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.301 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.301 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.301 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.301 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.301 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.301 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.301 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.302 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-01-31T05:40:20.302 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-01-31T05:40:20.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.302 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.302 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-01-31T05:40:20.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.303 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.303 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.303 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-01-31T05:40:20.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-01-31T05:40:20.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.303 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.304 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.304 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.305 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.305 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-01-31T05:40:20.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-01-31T05:40:20.305 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.305 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.305 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.305 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.305 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.306 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.306 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.306 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.306 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-01-31T05:40:20.306 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-01-31T05:40:20.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.307 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.307 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.307 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.307 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.307 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.307 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-01-31T05:40:20.307 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.308 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.308 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-01-31T05:40:20.308 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-01-31T05:40:20.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.308 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-01-31T05:40:20.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.309 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.309 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.309 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.309 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.309 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.309 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-01-31T05:40:20.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-01-31T05:40:20.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.310 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.310 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.310 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.310 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.310 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.311 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.311 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.311 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.311 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.311 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.311 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.311 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.312 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.312 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.312 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.312 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.312 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.312 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.312 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.312 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.312 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.313 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.313 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.313 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.313 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.313 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.313 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.314 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.314 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.314 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.314 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.314 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.315 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.315 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.316 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.316 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.316 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.316 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.316 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.316 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.316 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.316 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.317 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.317 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.317 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.317 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.317 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.317 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.317 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.317 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.318 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.318 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.318 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.318 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.319 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.319 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.319 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.319 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-01-31T05:40:20.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-01-31T05:40:20.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.319 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.320 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.320 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.320 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.320 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-01-31T05:40:20.320 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.320 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.320 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.321 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-01-31T05:40:20.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-01-31T05:40:20.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.321 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.321 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.321 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.321 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.322 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.322 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.322 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.322 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.322 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.322 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-01-31T05:40:20.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-01-31T05:40:20.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.323 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.323 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.323 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-01-31T05:40:20.323 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.323 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.323 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.323 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.323 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.324 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.324 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-01-31T05:40:20.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-01-31T05:40:20.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.324 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.324 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-01-31T05:40:20.325 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.325 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.325 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.325 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.325 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.325 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.325 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-01-31T05:40:20.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-01-31T05:40:20.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.326 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.327 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.327 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-01-31T05:40:20.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-01-31T05:40:20.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.328 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-01-31T05:40:20.328 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.328 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.328 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-01-31T05:40:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-01-31T05:40:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-01-31T05:40:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-01-31T05:40:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-01-31T05:40:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-01-31T05:40:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-01-31T05:40:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-01-31T05:40:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-01-31T05:40:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-01-31T05:40:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-01-31T05:40:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-01-31T05:40:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-01-31T05:40:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-01-31T05:40:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-01-31T05:40:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-01-31T05:40:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-01-31T05:40:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-01-31T05:40:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-01-31T05:40:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-01-31T05:40:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-01-31T05:40:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-01-31T05:40:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-01-31T05:40:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-01-31T05:40:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-01-31T05:40:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-01-31T05:40:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-01-31T05:40:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-01-31T05:40:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-01-31T05:40:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-01-31T05:40:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-01-31T05:40:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-01-31T05:40:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-01-31T05:40:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-01-31T05:40:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-01-31T05:40:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-01-31T05:40:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-01-31T05:40:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-01-31T05:40:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-01-31T05:40:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-01-31T05:40:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-01-31T05:40:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-01-31T05:40:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-01-31T05:40:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-01-31T05:40:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-01-31T05:40:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-01-31T05:40:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-01-31T05:40:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-01-31T05:40:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-01-31T05:40:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-01-31T05:40:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-01-31T05:40:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-01-31T05:40:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-01-31T05:40:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-01-31T05:40:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-01-31T05:40:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-01-31T05:40:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-01-31T05:40:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-01-31T05:40:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-01-31T05:40:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-01-31T05:40:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-01-31T05:40:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-01-31T05:40:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-01-31T05:40:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-01-31T05:40:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-01-31T05:40:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-01-31T05:40:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-01-31T05:40:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-01-31T05:40:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-01-31T05:40:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-01-31T05:40:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-01-31T05:40:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-01-31T05:40:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-01-31T05:40:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-01-31T05:40:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-01-31T05:40:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-01-31T05:40:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-01-31T05:40:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-01-31T05:40:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-01-31T05:40:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-01-31T05:40:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-01-31T05:40:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-01-31T05:40:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-01-31T05:40:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-01-31T05:40:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-01-31T05:40:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-01-31T05:40:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-01-31T05:40:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-01-31T05:40:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.398 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.399 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-01-31T05:40:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-01-31T05:40:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-01-31T05:40:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-01-31T05:40:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-01-31T05:40:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-01-31T05:40:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-01-31T05:40:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-01-31T05:40:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-01-31T05:40:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-01-31T05:40:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.423 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.423 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.423 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.423 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-01-31T05:40:20.423 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.423 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.424 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.424 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-01-31T05:40:20.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-01-31T05:40:20.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.424 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.424 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.424 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-01-31T05:40:20.424 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.424 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.425 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-01-31T05:40:20.425 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-01-31T05:40:20.425 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.425 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.425 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.426 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.426 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.426 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.426 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.426 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.426 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.427 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.427 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.427 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.427 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.427 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.428 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.428 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.428 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.429 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.429 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.429 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.429 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.429 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.429 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.430 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.430 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.430 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.430 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.430 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.430 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.431 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.431 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.431 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.431 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.431 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.431 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.431 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.432 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.432 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.432 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.432 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.432 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.433 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.433 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.433 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.433 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-01-31T05:40:20.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-01-31T05:40:20.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.434 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.434 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.434 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.434 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-01-31T05:40:20.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-01-31T05:40:20.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.435 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.435 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-01-31T05:40:20.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-01-31T05:40:20.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-01-31T05:40:20.435 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.436 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.436 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.436 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-01-31T05:40:20.436 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.436 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.436 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-01-31T05:40:20.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-01-31T05:40:20.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-01-31T05:40:20.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.437 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-01-31T05:40:20.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.438 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-01-31T05:40:20.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-01-31T05:40:20.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.438 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.438 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.438 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.439 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.439 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-01-31T05:40:20.439 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.439 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.440 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-01-31T05:40:20.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-01-31T05:40:20.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.440 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.440 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.440 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-01-31T05:40:20.440 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.440 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.440 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-01-31T05:40:20.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-01-31T05:40:20.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.441 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.441 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-01-31T05:40:20.442 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.442 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.442 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.442 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.443 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.443 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.443 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.443 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.443 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.443 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.443 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.443 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.444 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.444 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.444 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.444 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.444 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.444 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.445 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.445 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.445 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.445 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.445 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.446 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.446 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.446 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.446 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.446 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.447 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.447 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.447 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.447 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.447 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.447 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.447 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.447 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.447 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.448 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.448 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.448 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.448 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.448 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.448 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.449 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.449 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.449 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.450 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-01-31T05:40:20.450 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.451 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.451 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.451 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.451 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.452 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.452 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.452 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.452 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.452 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.452 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.452 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.452 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.453 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.453 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.453 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.453 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.453 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.453 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.453 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.454 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.454 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.454 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.454 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.454 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.454 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.454 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.455 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.455 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.456 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.456 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.456 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.456 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.456 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.456 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.457 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.457 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.457 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.457 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.457 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.457 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.457 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.458 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.458 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.458 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.459 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.459 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-01-31T05:40:20.459 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.460 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.460 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.460 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.460 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.460 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.461 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.461 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.466 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.466 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.466 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.466 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.466 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.466 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.467 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.467 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.467 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.467 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.467 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.467 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.467 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.467 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.467 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.468 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.468 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.468 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.468 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.469 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.469 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.469 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.469 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.469 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.469 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.470 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-01-31T05:40:20.470 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.470 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.470 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.470 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-01-31T05:40:20.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-01-31T05:40:20.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.471 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.471 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.471 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.471 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.471 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.471 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-01-31T05:40:20.472 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.472 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.472 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.472 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.472 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.472 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.473 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.473 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.473 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.473 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.473 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.473 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.473 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.474 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.474 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.474 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.474 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.474 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.474 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.475 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.475 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.475 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.475 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.475 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.475 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.475 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.476 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.476 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.476 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.476 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.477 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.477 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.477 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.477 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.477 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.477 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.477 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.478 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.478 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.478 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.478 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.478 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.478 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.479 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.479 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.479 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-01-31T05:40:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-01-31T05:40:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-01-31T05:40:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-01-31T05:40:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-01-31T05:40:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-01-31T05:40:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.483 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-01-31T05:40:20.483 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.483 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.483 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.483 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.484 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.484 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.484 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.484 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.484 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.484 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.484 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.485 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.485 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.485 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.485 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.485 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.485 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.485 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.485 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.486 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.486 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.486 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.486 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.486 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.487 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.487 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.487 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.487 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.487 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.487 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.488 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.488 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.488 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.488 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.488 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.488 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.488 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.489 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.489 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.489 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.489 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.489 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.489 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.489 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.489 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.490 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.490 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.490 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.490 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.490 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.491 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.491 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.491 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.491 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.491 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.491 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.492 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.492 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-01-31T05:40:20.492 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.492 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.493 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.493 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.493 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.493 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.493 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.493 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.493 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.494 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.494 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.494 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.494 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.494 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.494 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.494 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.495 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.495 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.495 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.495 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.495 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.496 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.496 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.496 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.496 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.496 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.496 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.496 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.497 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.497 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.497 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.497 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.497 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.497 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.498 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.498 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.498 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.498 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.498 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.498 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.498 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.498 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.499 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.499 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.499 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.499 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.499 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.499 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.500 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.500 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.500 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.500 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.500 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.500 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.501 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.501 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.501 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.501 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.501 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.502 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-01-31T05:40:20.502 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.502 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.502 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.502 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.502 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.503 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.503 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.503 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.503 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.503 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.503 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.503 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.503 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.503 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.504 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.504 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.504 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.504 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.504 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.504 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.504 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.505 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.505 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.505 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.505 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.505 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.505 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.505 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.506 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.506 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.506 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.507 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.507 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.507 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.507 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.507 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.507 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.507 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.507 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.508 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.508 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.508 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.508 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.508 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.508 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.508 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.508 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.509 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.509 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.509 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.510 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.510 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.510 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.510 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.510 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.510 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.510 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:20.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:20.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.511 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:20.511 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.511 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.511 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.511 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.511 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.512 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.512 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.512 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-01-31T05:40:20.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-01-31T05:40:20.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:20.512 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.513 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-01-31T05:40:20.513 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-01-31T05:40:20.513 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.513 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.513 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-01-31T05:40:20.513 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-01-31T05:40:20.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.514 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-01-31T05:40:20.514 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.514 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.514 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.514 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-01-31T05:40:20.515 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-01-31T05:40:20.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.515 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.515 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.515 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.515 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.515 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.515 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.515 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-01-31T05:40:20.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.516 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-01-31T05:40:20.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-01-31T05:40:20.516 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.516 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.516 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.517 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.517 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.517 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.517 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.517 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.517 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.517 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.517 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.518 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.518 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.518 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.519 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.519 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.519 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.519 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.519 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.519 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.519 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.519 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.519 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.520 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.520 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.520 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.520 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.520 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.521 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.521 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.521 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.521 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.521 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.522 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.522 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.522 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.522 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.522 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.522 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.523 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.523 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.523 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.523 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.523 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.523 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.523 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.523 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.523 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.523 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.524 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.524 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.524 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.524 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.524 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.525 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.525 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.525 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.525 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.525 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.525 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.526 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.526 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-01-31T05:40:20.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-01-31T05:40:20.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.526 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.526 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.526 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-01-31T05:40:20.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.527 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-01-31T05:40:20.527 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.527 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.527 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.527 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:20.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:20.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.528 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.528 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.528 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-01-31T05:40:20.528 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.528 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.528 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.528 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-01-31T05:40:20.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.529 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.529 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.529 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-01-31T05:40:20.530 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.530 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.530 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.530 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-01-31T05:40:20.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-01-31T05:40:20.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.530 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.530 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.531 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.531 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.531 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.531 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.531 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.531 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.532 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-01-31T05:40:20.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-01-31T05:40:20.532 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.532 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.532 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.532 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.532 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.532 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.532 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.533 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-01-31T05:40:20.533 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-01-31T05:40:20.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.533 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.533 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.534 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.534 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.534 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.534 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.534 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.534 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.534 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.534 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.534 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.535 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.535 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.535 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.535 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.535 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.535 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.535 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.535 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.536 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.536 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.536 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.536 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.536 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.536 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.536 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.536 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.537 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.537 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.537 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.537 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.537 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.537 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.538 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.538 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.538 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.538 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.538 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.538 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.538 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.538 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.539 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.539 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.539 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.539 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.539 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.539 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.539 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.540 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.540 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.540 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.540 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.541 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.541 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.541 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.541 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.541 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.541 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.541 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-01-31T05:40:20.542 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-01-31T05:40:20.542 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.542 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.542 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.542 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.542 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.542 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.543 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.543 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-01-31T05:40:20.543 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-01-31T05:40:20.543 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.543 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.543 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.544 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-01-31T05:40:20.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.544 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.544 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.544 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.544 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.544 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.545 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-01-31T05:40:20.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-01-31T05:40:20.545 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:20.545 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.545 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.545 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-01-31T05:40:20.545 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.545 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.546 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.546 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.546 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.546 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.546 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-01-31T05:40:20.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-01-31T05:40:20.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.547 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.547 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-01-31T05:40:20.547 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.547 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.547 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.547 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-01-31T05:40:20.547 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.548 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.548 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.548 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-01-31T05:40:20.548 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-01-31T05:40:20.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.548 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.548 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.549 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-01-31T05:40:20.549 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.549 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.549 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-01-31T05:40:20.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.549 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.550 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-01-31T05:40:20.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-01-31T05:40:20.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.550 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-01-31T05:40:20.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.550 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.550 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.551 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.551 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.551 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-01-31T05:40:20.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-01-31T05:40:20.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.552 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.552 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-01-31T05:40:20.552 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.552 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.552 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.552 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-01-31T05:40:20.552 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-01-31T05:40:20.553 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.553 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.553 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.553 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.553 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.553 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-01-31T05:40:20.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-01-31T05:40:20.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.553 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.554 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.554 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-01-31T05:40:20.554 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-01-31T05:40:20.554 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-01-31T05:40:20.554 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.554 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.554 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.554 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-01-31T05:40:20.555 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.555 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.555 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.555 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.556 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.556 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.556 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.556 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.557 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.557 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.557 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.557 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.557 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.557 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.557 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.557 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.558 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.558 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.558 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.558 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.558 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.558 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.559 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.559 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.559 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.559 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.559 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.559 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.559 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.559 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.560 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.560 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.560 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.560 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.560 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.560 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.560 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.561 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.561 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.561 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.561 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.561 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.561 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.561 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.561 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.562 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.562 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.562 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.562 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.562 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.562 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.563 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.563 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.563 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.563 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.563 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.564 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-01-31T05:40:20.564 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-01-31T05:40:20.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.564 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.564 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.564 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.564 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.565 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.565 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.565 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.565 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.565 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.565 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.565 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-01-31T05:40:20.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-01-31T05:40:20.566 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.566 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.566 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.566 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.566 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.566 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.566 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.567 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.567 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.567 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.567 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-01-31T05:40:20.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-01-31T05:40:20.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.567 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.567 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.568 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.568 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.568 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.568 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.568 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.568 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.568 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.569 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.569 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-01-31T05:40:20.569 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.569 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.569 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.569 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-01-31T05:40:20.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-01-31T05:40:20.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.570 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.570 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.570 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.570 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.570 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.570 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.570 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.571 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.571 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.571 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.571 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-01-31T05:40:20.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-01-31T05:40:20.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.572 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.572 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.572 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.572 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.573 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.574 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.574 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.574 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.574 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.574 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.574 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.574 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.574 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.574 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.575 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.575 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.575 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.575 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.575 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.575 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.575 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.575 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.576 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.576 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.576 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.576 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.576 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.576 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.577 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.577 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.577 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.577 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.577 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.577 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.577 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.578 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.578 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.578 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.578 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.578 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.578 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.578 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.578 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.578 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.579 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.579 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.579 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.579 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.580 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.580 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.580 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.580 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.580 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.581 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.581 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-01-31T05:40:20.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-01-31T05:40:20.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.581 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.581 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.582 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.582 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-01-31T05:40:20.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-01-31T05:40:20.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.583 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.583 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.583 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.583 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.583 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.583 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.583 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.584 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.584 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.584 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.584 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.584 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-01-31T05:40:20.585 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.585 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.585 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.585 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.585 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.585 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.585 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.586 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.586 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.586 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.586 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.586 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.586 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.586 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.586 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.586 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.587 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.587 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.588 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.588 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.588 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.588 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.588 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.588 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.588 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.588 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.589 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.589 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.589 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.589 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.590 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.590 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.590 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.590 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.590 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.590 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.590 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.590 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.590 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.590 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.591 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.591 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.591 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.591 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.591 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.592 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.592 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.592 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.592 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.592 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.592 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.592 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.593 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-01-31T05:40:20.593 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.593 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.593 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.593 INFO:teuthology.orchestra.run.smithi045.stdout: "drive_group_interval": { 2024-01-31T05:40:20.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "drive_group_interval", 2024-01-31T05:40:20.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:20.594 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.594 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300.0", 2024-01-31T05:40:20.594 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.594 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.594 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.594 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-01-31T05:40:20.594 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.595 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.595 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.595 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.595 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.596 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.596 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.596 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.596 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.596 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.596 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.596 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.597 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.597 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.597 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.597 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.597 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.597 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.597 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.598 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.598 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.598 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.598 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.598 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.598 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.599 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.599 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.599 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.599 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.599 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.600 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.600 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.600 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.600 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.600 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.600 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.600 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.601 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.601 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.601 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.601 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.601 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.601 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.601 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.601 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.602 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.602 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.602 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.602 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.603 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.603 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.603 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.603 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.604 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-01-31T05:40:20.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-01-31T05:40:20.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.604 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.604 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-01-31T05:40:20.604 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.605 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.605 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-01-31T05:40:20.605 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.605 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.605 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.605 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.605 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-01-31T05:40:20.606 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.606 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.606 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.606 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.607 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.607 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.607 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.607 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.607 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.608 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.608 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.608 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.608 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.608 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.608 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.608 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.609 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.609 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.609 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.609 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.609 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.609 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.610 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.610 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.610 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.610 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.610 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.610 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.611 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.611 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.611 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.611 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.611 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.611 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.611 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.612 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.612 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.612 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.612 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.612 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.612 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.612 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.612 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.613 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.613 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.613 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.613 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.613 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.613 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.613 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.614 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.614 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.614 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.614 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.614 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.614 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-01-31T05:40:20.614 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-01-31T05:40:20.614 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.615 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.615 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.615 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.615 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.615 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.615 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.616 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-01-31T05:40:20.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-01-31T05:40:20.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.616 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-01-31T05:40:20.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.617 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.617 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.617 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.617 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.617 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.617 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-01-31T05:40:20.617 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-01-31T05:40:20.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.618 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.618 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.618 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.618 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.618 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.618 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.618 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.619 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.619 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-01-31T05:40:20.619 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-01-31T05:40:20.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.619 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.619 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.619 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.620 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.620 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.620 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.620 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.620 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.620 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.620 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.621 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-01-31T05:40:20.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-01-31T05:40:20.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:20.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.621 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.621 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.621 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.621 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.622 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.622 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.622 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.622 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-01-31T05:40:20.622 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-01-31T05:40:20.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.622 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.622 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.623 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.623 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.623 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.623 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.623 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.623 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.623 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.623 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.624 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-01-31T05:40:20.624 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-01-31T05:40:20.624 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.624 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.624 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.624 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.625 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.625 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.625 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-01-31T05:40:20.625 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-01-31T05:40:20.625 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.625 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.626 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.626 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.626 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.626 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.626 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.626 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.626 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.626 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.627 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-01-31T05:40:20.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-01-31T05:40:20.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.627 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.627 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.627 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-01-31T05:40:20.627 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-01-31T05:40:20.628 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.628 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.628 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.628 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.628 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-01-31T05:40:20.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-01-31T05:40:20.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.629 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.629 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.629 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.629 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.629 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.629 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.629 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.630 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.630 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.630 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-01-31T05:40:20.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-01-31T05:40:20.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.630 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.630 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.631 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.631 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.631 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.631 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.631 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.632 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-01-31T05:40:20.632 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-01-31T05:40:20.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.632 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.633 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.633 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.633 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-01-31T05:40:20.634 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.634 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.634 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.634 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-01-31T05:40:20.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-01-31T05:40:20.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.634 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.634 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.634 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.635 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.635 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.635 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.635 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-01-31T05:40:20.635 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.635 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.635 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-01-31T05:40:20.636 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-01-31T05:40:20.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.636 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.636 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.636 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.636 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.637 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-01-31T05:40:20.637 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.637 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.637 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.637 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.637 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.637 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.638 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.638 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.638 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.638 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.638 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.638 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.638 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.638 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.639 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.639 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.639 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.639 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.639 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.639 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.640 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.640 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.640 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.640 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.640 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.641 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.641 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.641 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.641 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.641 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.642 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.642 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.642 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.642 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.642 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.642 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.643 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.643 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.643 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.643 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.644 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.644 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.644 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.644 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.644 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.645 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.645 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.645 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.645 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.645 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-01-31T05:40:20.645 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.646 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.646 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.646 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.646 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.646 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.646 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.646 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.646 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.647 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.647 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.647 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.647 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.647 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.647 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.647 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.647 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.647 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.648 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.648 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.648 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.648 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.649 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.649 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.649 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.649 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.649 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.649 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.649 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.650 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.650 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.650 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.650 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.650 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.650 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.651 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.651 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.651 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.651 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.651 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.651 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.651 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.651 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.653 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-01-31T05:40:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.654 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.655 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.656 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-01-31T05:40:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-01-31T05:40:20.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-01-31T05:40:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-01-31T05:40:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.664 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.665 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.665 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.665 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:20.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:20.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.665 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.665 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-01-31T05:40:20.666 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.666 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.666 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.666 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.666 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.666 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.666 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.666 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.667 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.667 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.667 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.667 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.667 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.667 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.668 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.668 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.668 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.668 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.668 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.668 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.669 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.669 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.670 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.670 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.670 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.670 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.670 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.670 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.670 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.671 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.671 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.671 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.671 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.671 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.671 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.671 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.671 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.672 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.672 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.672 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.672 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.672 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.672 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.672 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.672 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.673 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.673 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.673 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.673 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.673 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.673 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.673 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.673 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.674 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.674 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.674 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.674 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.674 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.674 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.674 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.674 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-01-31T05:40:20.675 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.675 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.675 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.675 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-01-31T05:40:20.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-01-31T05:40:20.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.676 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.676 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.676 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.676 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.676 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.676 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.676 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-01-31T05:40:20.676 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.676 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.677 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.677 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-01-31T05:40:20.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-01-31T05:40:20.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.677 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.677 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.677 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.678 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.678 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.678 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-01-31T05:40:20.678 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.678 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.678 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.678 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-01-31T05:40:20.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-01-31T05:40:20.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.679 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.679 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.679 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.679 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.679 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.679 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-01-31T05:40:20.679 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.680 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.680 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.680 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-01-31T05:40:20.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-01-31T05:40:20.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.680 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.680 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.681 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.681 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.681 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.681 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-01-31T05:40:20.681 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.681 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.681 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.682 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-01-31T05:40:20.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-01-31T05:40:20.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.682 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.682 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.682 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.682 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.682 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.682 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.683 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-01-31T05:40:20.683 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.683 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.683 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.683 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-01-31T05:40:20.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-01-31T05:40:20.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.683 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.684 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.684 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.684 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.684 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.684 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.684 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.684 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.684 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.684 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.685 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-01-31T05:40:20.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-01-31T05:40:20.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.685 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.685 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.685 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.685 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.685 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.686 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.686 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.686 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.686 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.686 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-01-31T05:40:20.686 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-01-31T05:40:20.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.687 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.687 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.687 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-01-31T05:40:20.687 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.687 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.687 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.687 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.687 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.688 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.688 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-01-31T05:40:20.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-01-31T05:40:20.688 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.688 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.688 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.688 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.689 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.689 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.689 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.689 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.689 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.689 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.689 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.689 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:20.690 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:20.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.690 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.690 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.690 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-01-31T05:40:20.690 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-01-31T05:40:20.690 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.690 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.690 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.691 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.691 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.691 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.691 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-01-31T05:40:20.691 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-01-31T05:40:20.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.691 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.692 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.692 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-01-31T05:40:20.692 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.692 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.692 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.692 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.692 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.692 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.692 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.693 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-01-31T05:40:20.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-01-31T05:40:20.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.693 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.693 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.693 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.693 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.693 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.694 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.694 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.694 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.694 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.694 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-01-31T05:40:20.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-01-31T05:40:20.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.695 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.695 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.695 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.695 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.695 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.695 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.695 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.695 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.695 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.696 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.696 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.696 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.696 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.696 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.696 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.696 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.696 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.697 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.697 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.697 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.697 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.697 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.697 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.697 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.697 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.697 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.698 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.698 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.698 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.698 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.698 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.698 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.698 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.699 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.699 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.699 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.699 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.699 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.699 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.699 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.699 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.700 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.700 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.700 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.700 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.700 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.700 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.700 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.701 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.701 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.701 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.701 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.701 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.701 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.701 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.701 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.701 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.702 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.702 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.702 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.702 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.702 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.702 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.703 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.703 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.703 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.703 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.703 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.703 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.703 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.703 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.704 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-01-31T05:40:20.704 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-01-31T05:40:20.704 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.704 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.704 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.704 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.704 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.705 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.705 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.705 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.705 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.705 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.705 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.705 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-01-31T05:40:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-01-31T05:40:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.707 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.707 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.707 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-01-31T05:40:20.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-01-31T05:40:20.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.707 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.707 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.708 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-01-31T05:40:20.708 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.708 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.708 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.708 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.708 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.708 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.708 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.708 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.709 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-01-31T05:40:20.709 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.709 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.709 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.709 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.709 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.710 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.710 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.710 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.710 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.710 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.710 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.710 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.711 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.711 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.711 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.711 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.711 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.711 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.711 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.711 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.712 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.712 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.712 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.712 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.712 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.713 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.713 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.713 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.713 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.713 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.713 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.713 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.713 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.714 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.714 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.714 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.714 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.714 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.714 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.714 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.714 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-01-31T05:40:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.727 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.727 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-01-31T05:40:20.727 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-01-31T05:40:20.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.727 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.727 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-01-31T05:40:20.727 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.728 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.728 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.728 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-01-31T05:40:20.728 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.728 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.728 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.728 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-01-31T05:40:20.728 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-01-31T05:40:20.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.729 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.729 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.729 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.729 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.729 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.729 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.729 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-01-31T05:40:20.730 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.730 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.730 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-01-31T05:40:20.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-01-31T05:40:20.730 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.730 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:20.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.731 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.731 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.731 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-01-31T05:40:20.731 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.731 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-01-31T05:40:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-01-31T05:40:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-01-31T05:40:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-01-31T05:40:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.734 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.734 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.734 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.734 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.734 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-01-31T05:40:20.734 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-01-31T05:40:20.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.735 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.735 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.735 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.735 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.735 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.735 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.736 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.736 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:20.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:20.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.736 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.736 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:20.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.737 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.737 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.737 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.737 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.737 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.738 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.738 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.738 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.738 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.738 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.738 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.738 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.739 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.739 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.739 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.739 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.739 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.739 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.739 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.740 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.740 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.740 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.740 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.740 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.740 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.741 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.741 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.741 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.741 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.741 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.741 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.741 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.741 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.742 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.742 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.742 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.742 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.742 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.742 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.742 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.743 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.743 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.743 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.743 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.743 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.743 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.743 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.744 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.744 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.744 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.744 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.744 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.744 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.744 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.745 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.745 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.745 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.745 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.745 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.746 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.746 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.746 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.746 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.746 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.746 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-01-31T05:40:20.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-01-31T05:40:20.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.747 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.747 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.747 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.747 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.748 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.748 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.748 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-01-31T05:40:20.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-01-31T05:40:20.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.748 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.748 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-01-31T05:40:20.749 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.749 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.749 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.749 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.749 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.749 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.750 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-01-31T05:40:20.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-01-31T05:40:20.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.750 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.750 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.750 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-01-31T05:40:20.750 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.750 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.751 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.751 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.751 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.751 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.751 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.751 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.751 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.751 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-01-31T05:40:20.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.752 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 24874, 2024-01-31T05:40:20.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "z", 2024-01-31T05:40:20.752 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540701272860131327, 2024-01-31T05:40:20.752 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-01-31T05:40:20.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-01-31T05:40:20.753 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.753 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.753 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.753 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:20.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:20.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.753 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.753 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.753 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:20.754 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.754 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.754 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.754 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-01-31T05:40:20.754 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.754 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.754 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.754 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.755 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.755 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.755 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.755 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.755 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.755 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.755 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.755 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.756 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.756 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.756 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.756 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.756 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.756 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.756 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.756 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.757 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.757 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.757 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.757 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.758 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.758 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.758 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.758 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.758 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.758 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.758 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.758 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.759 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.759 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.759 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.759 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.759 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.760 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.760 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.760 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.760 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.760 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.760 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.760 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.760 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.761 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.761 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.761 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.761 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.761 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.761 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.762 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.762 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.762 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.762 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.762 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.762 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.762 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.762 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.763 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-01-31T05:40:20.763 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-01-31T05:40:20.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.763 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.763 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.763 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.763 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.764 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.764 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-01-31T05:40:20.764 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.764 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.764 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-01-31T05:40:20.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-01-31T05:40:20.765 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.765 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.765 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.765 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-01-31T05:40:20.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.765 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.765 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-01-31T05:40:20.765 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.766 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-01-31T05:40:20.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-01-31T05:40:20.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.766 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.766 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.766 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.767 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.767 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.767 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-01-31T05:40:20.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.767 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-01-31T05:40:20.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-01-31T05:40:20.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.768 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.768 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.768 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.768 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.768 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.769 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-01-31T05:40:20.769 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.769 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.769 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-01-31T05:40:20.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-01-31T05:40:20.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.770 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.770 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-01-31T05:40:20.770 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.770 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.770 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.770 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-01-31T05:40:20.770 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.770 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.771 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.771 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-01-31T05:40:20.771 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-01-31T05:40:20.771 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.771 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.771 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.771 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.771 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.772 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.772 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-01-31T05:40:20.772 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.772 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.772 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.772 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-01-31T05:40:20.772 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-01-31T05:40:20.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.773 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.773 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.773 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.773 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.773 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-01-31T05:40:20.774 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.774 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.774 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.774 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-01-31T05:40:20.774 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-01-31T05:40:20.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.774 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.775 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.775 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.775 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.775 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.775 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-01-31T05:40:20.775 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.775 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.775 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.775 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.776 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-01-31T05:40:20.776 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.776 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.776 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.776 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-01-31T05:40:20.777 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-01-31T05:40:20.777 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.777 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.777 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.777 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.777 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.777 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.777 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-01-31T05:40:20.777 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.778 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.778 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-01-31T05:40:20.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-01-31T05:40:20.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.778 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.779 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-01-31T05:40:20.779 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.779 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.779 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-01-31T05:40:20.779 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-31T05:40:20.779 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.779 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.780 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-01-31T05:40:20.780 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-01-31T05:40:20.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:20.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:20.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-01-31T05:40:20.780 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-01-31T05:40:20.781 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.781 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-01-31T05:40:20.781 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-31T05:40:20.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.781 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.781 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-01-31T05:40:20.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-01-31T05:40:20.781 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:20.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.782 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-01-31T05:40:20.782 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-01-31T05:40:20.782 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-01-31T05:40:20.782 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.782 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-01-31T05:40:20.782 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.782 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.783 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-01-31T05:40:20.783 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-01-31T05:40:20.783 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.783 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.783 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.783 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-01-31T05:40:20.784 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.784 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.784 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.784 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-01-31T05:40:20.784 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-01-31T05:40:20.784 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.784 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.784 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-01-31T05:40:20.785 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-01-31T05:40:20.785 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:20.785 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.785 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.785 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-01-31T05:40:20.785 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-01-31T05:40:20.785 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-01-31T05:40:20.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.785 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-01-31T05:40:20.785 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-01-31T05:40:20.786 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.786 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.786 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-01-31T05:40:20.786 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-01-31T05:40:20.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.786 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.786 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.786 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-01-31T05:40:20.787 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.787 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.787 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.787 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-01-31T05:40:20.787 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-31T05:40:20.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.787 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.787 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-01-31T05:40:20.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-01-31T05:40:20.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:20.788 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:20.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-01-31T05:40:20.788 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-01-31T05:40:20.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.788 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-01-31T05:40:20.789 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-31T05:40:20.789 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.789 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.789 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.789 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.789 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.790 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.790 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.790 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.790 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.790 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.790 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.790 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.790 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.790 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.791 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.791 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.791 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.791 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.791 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.791 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.791 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.792 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.792 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.792 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.792 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.792 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.793 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.793 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.793 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.793 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.793 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.794 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.794 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.794 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.794 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.794 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.794 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.794 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.794 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.795 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.795 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.795 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.795 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.796 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.796 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.796 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.797 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.797 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.797 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-01-31T05:40:20.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-01-31T05:40:20.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:20.797 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.797 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.798 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:20.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.798 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.798 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.798 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-01-31T05:40:20.798 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.798 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.799 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-01-31T05:40:20.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-01-31T05:40:20.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.799 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-01-31T05:40:20.799 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.799 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.800 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.800 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-01-31T05:40:20.800 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-01-31T05:40:20.800 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap" 2024-01-31T05:40:20.800 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-01-31T05:40:20.800 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.800 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.801 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-01-31T05:40:20.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-01-31T05:40:20.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.801 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.802 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.802 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.802 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-01-31T05:40:20.802 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.802 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.802 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.802 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:20.802 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:20.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.803 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.803 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:20.803 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.803 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-01-31T05:40:20.803 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.804 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.804 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-01-31T05:40:20.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-01-31T05:40:20.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.804 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-01-31T05:40:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-01-31T05:40:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-01-31T05:40:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-01-31T05:40:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-01-31T05:40:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-01-31T05:40:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-01-31T05:40:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-01-31T05:40:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-01-31T05:40:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-01-31T05:40:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-01-31T05:40:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.809 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.809 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-01-31T05:40:20.809 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.809 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-01-31T05:40:20.810 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-01-31T05:40:20.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-01-31T05:40:20.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.810 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.810 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.810 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-01-31T05:40:20.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.811 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-01-31T05:40:20.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-01-31T05:40:20.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.811 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.811 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.812 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-01-31T05:40:20.812 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.812 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.812 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.812 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-01-31T05:40:20.812 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.812 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.813 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-01-31T05:40:20.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-01-31T05:40:20.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.813 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-01-31T05:40:20.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-01-31T05:40:20.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.814 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-01-31T05:40:20.814 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-01-31T05:40:20.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:20.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.815 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.815 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.815 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-01-31T05:40:20.815 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.816 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.816 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-01-31T05:40:20.816 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-01-31T05:40:20.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:20.816 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.817 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-01-31T05:40:20.817 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.817 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.817 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-01-31T05:40:20.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.818 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-01-31T05:40:20.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-01-31T05:40:20.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.818 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.818 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.819 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-01-31T05:40:20.819 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-01-31T05:40:20.819 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-01-31T05:40:20.819 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-01-31T05:40:20.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.820 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.820 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-01-31T05:40:20.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-01-31T05:40:20.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.821 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.821 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.821 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-01-31T05:40:20.821 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.821 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.821 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-01-31T05:40:20.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-01-31T05:40:20.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.822 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.823 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-01-31T05:40:20.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.823 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-01-31T05:40:20.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-01-31T05:40:20.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.823 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-01-31T05:40:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-01-31T05:40:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-01-31T05:40:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-01-31T05:40:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-01-31T05:40:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-01-31T05:40:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-01-31T05:40:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-01-31T05:40:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-01-31T05:40:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-01-31T05:40:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-01-31T05:40:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-01-31T05:40:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-01-31T05:40:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-01-31T05:40:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-01-31T05:40:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-01-31T05:40:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-01-31T05:40:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-01-31T05:40:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-01-31T05:40:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-01-31T05:40:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-01-31T05:40:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-01-31T05:40:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-01-31T05:40:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-01-31T05:40:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.835 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-01-31T05:40:20.835 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.835 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.835 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-01-31T05:40:20.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-01-31T05:40:20.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.835 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.836 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.836 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-01-31T05:40:20.836 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.836 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-01-31T05:40:20.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.837 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.837 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-01-31T05:40:20.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-01-31T05:40:20.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-01-31T05:40:20.838 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.838 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-01-31T05:40:20.838 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.838 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.838 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-01-31T05:40:20.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-01-31T05:40:20.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.839 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.839 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.839 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-01-31T05:40:20.839 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.839 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.839 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.840 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-01-31T05:40:20.840 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.840 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.840 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.840 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-01-31T05:40:20.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-01-31T05:40:20.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.841 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.841 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.841 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-01-31T05:40:20.841 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.841 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.841 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.841 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-01-31T05:40:20.842 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.842 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-01-31T05:40:20.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-01-31T05:40:20.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.842 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-01-31T05:40:20.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.843 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.843 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.843 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-01-31T05:40:20.843 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.843 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.844 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.844 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-01-31T05:40:20.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-01-31T05:40:20.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.844 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.844 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-01-31T05:40:20.845 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.845 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-01-31T05:40:20.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.845 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.845 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-01-31T05:40:20.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-01-31T05:40:20.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-01-31T05:40:20.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.846 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.846 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-01-31T05:40:20.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.847 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.847 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.847 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-01-31T05:40:20.847 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-01-31T05:40:20.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.847 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.848 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-01-31T05:40:20.848 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.848 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.848 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.848 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-01-31T05:40:20.848 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.848 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.849 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-01-31T05:40:20.849 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-01-31T05:40:20.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.849 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.849 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-01-31T05:40:20.849 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.850 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.850 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.850 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-01-31T05:40:20.850 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.850 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.850 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-01-31T05:40:20.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-01-31T05:40:20.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.851 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.851 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.851 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.851 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-01-31T05:40:20.851 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.852 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.852 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-01-31T05:40:20.852 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-01-31T05:40:20.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.852 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.852 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:20.853 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.853 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.853 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.853 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-01-31T05:40:20.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.853 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.854 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-01-31T05:40:20.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-01-31T05:40:20.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.854 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.854 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-01-31T05:40:20.854 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.854 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.854 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-01-31T05:40:20.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.855 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-01-31T05:40:20.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-01-31T05:40:20.855 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.855 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.856 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-01-31T05:40:20.856 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.856 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.856 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.856 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-01-31T05:40:20.856 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.857 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-01-31T05:40:20.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-01-31T05:40:20.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.857 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.857 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.857 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-01-31T05:40:20.857 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.857 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.858 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.858 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-01-31T05:40:20.858 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.858 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.858 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.858 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-01-31T05:40:20.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-01-31T05:40:20.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.859 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.859 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.859 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.859 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.859 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.859 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-01-31T05:40:20.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.860 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.860 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-01-31T05:40:20.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-01-31T05:40:20.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.860 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.861 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:20.861 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.861 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.861 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.861 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-01-31T05:40:20.861 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.862 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.862 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-01-31T05:40:20.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-01-31T05:40:20.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.862 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:20.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.863 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.863 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.863 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-01-31T05:40:20.863 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.863 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-01-31T05:40:20.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-01-31T05:40:20.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.864 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.864 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.864 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.864 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.864 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.864 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-01-31T05:40:20.865 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.865 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.865 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.865 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.865 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.866 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.866 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.866 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.866 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.866 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.866 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.866 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.867 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.867 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.867 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.867 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.867 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.867 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-01-31T05:40:20.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-01-31T05:40:20.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.868 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.868 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.868 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.868 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.871 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-01-31T05:40:20.871 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.871 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.871 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.871 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.872 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.872 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.872 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.872 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.872 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.872 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.872 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.872 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-01-31T05:40:20.873 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.873 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.873 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.873 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.874 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.874 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.874 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.874 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.874 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.874 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.874 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.874 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.874 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.875 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.875 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.875 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.875 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.875 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.875 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.876 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.876 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.876 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.876 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.876 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.876 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.877 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.877 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-01-31T05:40:20.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-01-31T05:40:20.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.877 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.877 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-01-31T05:40:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-01-31T05:40:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-01-31T05:40:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-01-31T05:40:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-01-31T05:40:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-01-31T05:40:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-01-31T05:40:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-01-31T05:40:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-01-31T05:40:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-01-31T05:40:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-01-31T05:40:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-01-31T05:40:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-01-31T05:40:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-01-31T05:40:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-01-31T05:40:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-01-31T05:40:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-01-31T05:40:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-01-31T05:40:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-01-31T05:40:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-01-31T05:40:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-01-31T05:40:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-01-31T05:40:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-01-31T05:40:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-01-31T05:40:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-01-31T05:40:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-01-31T05:40:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-01-31T05:40:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-01-31T05:40:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-01-31T05:40:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-01-31T05:40:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.892 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-01-31T05:40:20.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-01-31T05:40:20.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.892 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.892 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.892 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.893 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-01-31T05:40:20.893 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.893 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.893 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-01-31T05:40:20.893 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-01-31T05:40:20.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-01-31T05:40:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.895 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.895 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-01-31T05:40:20.895 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-01-31T05:40:20.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.896 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-01-31T05:40:20.896 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.896 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.896 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-01-31T05:40:20.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-01-31T05:40:20.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.897 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.897 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.897 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-01-31T05:40:20.897 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.897 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-01-31T05:40:20.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.898 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.898 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.898 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-01-31T05:40:20.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-01-31T05:40:20.898 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.898 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.898 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.898 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.899 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.899 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-01-31T05:40:20.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.899 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-01-31T05:40:20.900 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-01-31T05:40:20.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.900 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.900 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.900 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.900 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-01-31T05:40:20.901 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.901 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.901 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.901 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-01-31T05:40:20.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-01-31T05:40:20.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.901 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.902 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.902 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.902 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.902 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.902 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-01-31T05:40:20.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.903 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-01-31T05:40:20.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-01-31T05:40:20.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.903 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.904 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-01-31T05:40:20.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.904 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.904 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-01-31T05:40:20.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-01-31T05:40:20.904 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.905 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.905 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.905 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.905 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.905 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-01-31T05:40:20.905 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.905 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.906 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.906 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-01-31T05:40:20.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-01-31T05:40:20.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.906 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.906 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.906 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.906 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.907 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.907 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.907 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-01-31T05:40:20.907 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.907 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.907 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.907 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.907 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-01-31T05:40:20.908 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.908 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.908 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.908 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:20.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:20.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.909 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.909 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.909 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.909 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.909 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.909 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.909 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.909 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.909 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.910 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.910 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.910 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.910 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.910 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.911 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:20.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:20.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.911 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.911 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.911 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.911 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.912 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.912 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.912 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:20.912 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:20.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.912 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:20.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.913 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.913 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:20.913 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:20.913 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:20.913 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:20.913 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:20.914 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:20.914 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:20.914 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:20.914 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.914 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.914 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.914 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.914 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:20.915 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:20.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.915 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.915 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.915 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.915 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.915 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.915 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.915 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.916 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.916 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.916 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-01-31T05:40:20.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-01-31T05:40:20.916 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-01-31T05:40:20.917 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.917 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.917 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.917 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-01-31T05:40:20.917 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.917 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.918 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.918 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-01-31T05:40:20.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-01-31T05:40:20.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:20.918 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.918 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:20.918 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-01-31T05:40:20.918 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.919 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.919 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-01-31T05:40:20.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:20.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:20.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-01-31T05:40:20.920 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:20.920 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:20.920 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:20.920 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-01-31T05:40:20.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-01-31T05:40:20.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.921 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.921 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.921 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.921 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.921 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.921 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.922 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-01-31T05:40:20.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-01-31T05:40:20.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.922 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.922 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.923 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.923 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.923 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.923 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.923 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-01-31T05:40:20.923 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-01-31T05:40:20.923 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.924 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.924 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.924 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.924 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.924 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.925 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.925 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-01-31T05:40:20.925 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-01-31T05:40:20.925 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.925 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.925 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.926 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.926 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.926 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.926 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.926 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.926 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.926 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.926 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.927 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-01-31T05:40:20.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-01-31T05:40:20.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.927 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.927 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.927 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.928 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.928 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.928 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-01-31T05:40:20.928 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-01-31T05:40:20.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.929 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.929 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.929 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.929 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.929 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.929 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.929 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.930 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.930 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.930 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.930 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-01-31T05:40:20.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-01-31T05:40:20.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.930 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.930 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.931 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.931 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.931 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.931 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.931 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.931 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.932 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-01-31T05:40:20.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-01-31T05:40:20.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.932 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.932 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.932 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.932 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.932 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-01-31T05:40:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-01-31T05:40:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-01-31T05:40:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-01-31T05:40:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-01-31T05:40:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-01-31T05:40:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-01-31T05:40:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-01-31T05:40:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-01-31T05:40:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-01-31T05:40:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-01-31T05:40:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-01-31T05:40:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-01-31T05:40:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-01-31T05:40:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-01-31T05:40:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-01-31T05:40:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-01-31T05:40:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.946 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.946 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.946 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.946 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-01-31T05:40:20.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-01-31T05:40:20.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.947 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.947 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.947 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.947 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.947 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.948 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.948 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.948 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.948 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-01-31T05:40:20.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-01-31T05:40:20.948 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.949 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-01-31T05:40:20.949 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.949 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.949 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.949 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.949 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.950 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.950 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-01-31T05:40:20.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-01-31T05:40:20.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.950 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.951 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.952 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-01-31T05:40:20.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-01-31T05:40:20.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.952 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.952 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.952 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.952 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.953 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.953 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-01-31T05:40:20.953 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-01-31T05:40:20.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.954 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.954 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.954 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.954 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.954 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.954 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.955 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.955 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-01-31T05:40:20.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-01-31T05:40:20.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.955 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.956 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.956 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.956 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.956 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.957 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-01-31T05:40:20.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-01-31T05:40:20.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.957 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.957 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.957 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.957 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.958 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.958 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.958 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.958 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.958 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-01-31T05:40:20.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-01-31T05:40:20.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.958 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.959 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.959 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.959 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.959 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.959 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.959 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.959 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.959 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.959 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.960 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-01-31T05:40:20.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-01-31T05:40:20.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.960 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.961 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.961 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.961 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.961 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-01-31T05:40:20.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-01-31T05:40:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.963 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.963 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-01-31T05:40:20.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-01-31T05:40:20.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.963 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.963 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.963 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.964 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.964 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.964 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.964 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.964 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.965 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-01-31T05:40:20.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-01-31T05:40:20.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.965 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.965 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.965 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.965 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.966 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.966 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.966 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.966 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.966 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-01-31T05:40:20.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-01-31T05:40:20.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.967 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.967 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.967 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.967 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.967 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.967 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.968 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.968 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.968 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-01-31T05:40:20.968 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-01-31T05:40:20.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.968 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.969 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.969 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.969 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.969 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.969 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.969 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.969 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.970 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-01-31T05:40:20.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-01-31T05:40:20.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.970 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.970 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.970 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.970 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.970 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.971 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.971 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.971 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.971 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.971 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.971 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-01-31T05:40:20.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-01-31T05:40:20.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.972 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.972 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.972 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.972 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.972 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.973 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.973 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.973 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.973 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-01-31T05:40:20.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-01-31T05:40:20.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.974 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-01-31T05:40:20.974 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.974 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.974 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.974 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.974 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.975 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.975 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-01-31T05:40:20.975 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-01-31T05:40:20.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.976 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.976 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.976 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.976 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.976 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.976 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.977 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-01-31T05:40:20.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-01-31T05:40:20.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.977 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.977 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.977 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-01-31T05:40:20.977 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.977 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.977 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.978 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.978 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.978 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.978 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-01-31T05:40:20.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-01-31T05:40:20.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.979 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.979 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-01-31T05:40:20.979 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.979 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.979 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.979 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.979 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.980 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.980 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.980 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-01-31T05:40:20.980 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-01-31T05:40:20.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.980 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.981 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.981 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.981 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.981 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.981 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.981 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.981 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.981 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.981 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.982 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-01-31T05:40:20.982 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-01-31T05:40:20.982 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.982 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.982 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.982 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-01-31T05:40:20.982 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.982 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.983 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.983 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.983 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.983 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.983 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.983 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-01-31T05:40:20.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-01-31T05:40:20.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.984 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.984 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.984 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.984 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.984 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.985 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.985 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.985 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.985 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-01-31T05:40:20.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-01-31T05:40:20.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.985 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.986 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.986 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:20.986 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.986 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.986 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.986 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.986 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.986 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.987 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.987 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-01-31T05:40:20.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-01-31T05:40:20.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.987 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.987 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:20.988 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.988 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.988 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.988 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.988 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.988 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.989 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-01-31T05:40:20.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-01-31T05:40:20.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.989 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.989 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:20.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.989 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.990 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.990 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.990 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.990 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.990 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.990 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-01-31T05:40:20.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-01-31T05:40:20.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.991 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.991 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.991 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-01-31T05:40:20.991 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.991 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.992 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.992 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-01-31T05:40:20.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-01-31T05:40:20.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.992 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.992 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.992 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.993 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.993 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.993 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.993 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.993 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-01-31T05:40:20.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-01-31T05:40:20.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.994 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.994 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.994 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.994 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.994 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.995 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.995 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.995 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.995 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-01-31T05:40:20.995 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-01-31T05:40:20.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:20.995 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.995 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.996 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:20.996 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.996 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.996 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-01-31T05:40:20.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.996 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.997 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-01-31T05:40:20.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-01-31T05:40:20.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:20.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-01-31T05:40:20.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.998 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.998 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:20.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:20.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:20.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:20.998 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-01-31T05:40:20.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-01-31T05:40:20.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:20.999 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:20.999 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:20.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:20.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:20.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:20.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:20.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:20.999 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.000 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.000 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.000 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.000 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.001 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.001 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.001 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.001 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.001 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.001 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.001 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.001 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.002 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.002 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.002 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.002 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.002 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.002 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.003 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.003 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.003 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.003 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.003 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.003 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.004 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.004 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.004 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.004 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.005 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.005 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.005 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.005 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.005 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.005 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.005 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.005 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.006 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.006 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.006 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.006 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.006 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.006 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.006 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.006 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.007 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.007 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.007 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.007 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.007 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.007 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.007 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.008 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.008 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.008 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.008 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-01-31T05:40:21.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-01-31T05:40:21.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.008 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-01-31T05:40:21.009 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.009 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.010 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-01-31T05:40:21.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-01-31T05:40:21.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.010 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.010 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.010 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.010 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.010 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.010 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.011 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.011 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.011 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.011 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.011 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-01-31T05:40:21.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-01-31T05:40:21.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.012 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.012 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-01-31T05:40:21.012 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.012 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.012 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.012 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.012 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.013 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.013 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-01-31T05:40:21.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-01-31T05:40:21.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.013 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.013 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.013 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-01-31T05:40:21.013 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.013 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.014 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.014 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.014 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.014 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.014 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.014 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-01-31T05:40:21.014 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-01-31T05:40:21.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.015 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.015 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.015 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.015 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.015 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.015 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.015 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.015 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.016 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.016 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-01-31T05:40:21.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-01-31T05:40:21.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.016 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.016 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-01-31T05:40:21.017 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.018 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-01-31T05:40:21.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-01-31T05:40:21.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-01-31T05:40:21.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.018 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.019 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.019 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.019 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-01-31T05:40:21.019 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.019 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.019 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.020 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-01-31T05:40:21.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-01-31T05:40:21.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.020 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-01-31T05:40:21.020 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-01-31T05:40:21.020 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-01-31T05:40:21.021 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.021 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.021 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.021 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-01-31T05:40:21.021 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-01-31T05:40:21.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.022 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.022 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.022 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.022 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.022 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.022 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.022 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.023 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.023 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-01-31T05:40:21.023 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.023 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.024 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.024 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-01-31T05:40:21.024 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-01-31T05:40:21.024 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.024 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.024 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.024 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.024 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.025 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.025 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.025 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-01-31T05:40:21.025 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.025 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.025 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.025 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.026 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.026 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.026 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.026 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.026 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.026 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.027 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.027 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.027 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.027 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.027 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.027 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.027 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.028 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.028 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.028 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.028 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.028 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.028 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.029 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.029 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.029 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.029 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.029 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.029 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.029 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.030 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.030 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.030 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.030 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.030 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.030 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.031 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.031 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.031 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.031 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.031 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.031 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.031 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.031 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.032 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.032 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.032 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.032 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.032 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.032 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.033 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.033 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.033 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.033 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.033 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.033 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.033 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.034 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.034 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-01-31T05:40:21.034 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-01-31T05:40:21.034 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.034 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.034 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.034 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-01-31T05:40:21.035 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.035 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.035 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-01-31T05:40:21.035 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.035 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.035 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-01-31T05:40:21.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-01-31T05:40:21.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.036 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.036 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.036 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-01-31T05:40:21.036 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.036 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.036 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.037 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-01-31T05:40:21.037 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.037 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.037 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.037 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-01-31T05:40:21.037 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-01-31T05:40:21.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.038 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.038 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.038 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-01-31T05:40:21.038 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.038 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.038 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-01-31T05:40:21.039 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.039 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.039 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.039 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-01-31T05:40:21.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-01-31T05:40:21.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.039 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.040 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-01-31T05:40:21.040 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.040 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.040 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.040 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-01-31T05:40:21.040 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.040 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.041 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.041 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-01-31T05:40:21.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-01-31T05:40:21.041 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.041 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.041 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.041 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.041 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.042 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.042 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.042 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-01-31T05:40:21.042 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.042 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.042 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.042 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:21.042 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:21.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.043 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.043 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.043 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:21.043 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.043 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.043 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.043 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-01-31T05:40:21.043 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.044 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.044 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.044 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-01-31T05:40:21.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-01-31T05:40:21.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.044 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.044 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.044 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-01-31T05:40:21.045 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.045 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.045 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.045 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-01-31T05:40:21.045 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.045 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.045 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.045 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.045 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.046 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-01-31T05:40:21.046 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.046 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.046 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.046 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.046 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.047 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.047 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.047 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.047 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.047 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.047 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.047 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.047 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.048 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.048 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.048 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.048 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.048 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.048 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.048 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.048 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.049 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.049 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.049 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.049 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.049 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.049 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.050 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.050 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.050 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.050 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.050 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.050 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.050 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.050 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.051 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.051 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.051 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.051 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.051 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.051 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.051 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.052 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.052 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.052 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.052 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.052 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.052 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.052 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.052 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.052 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.053 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.053 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.053 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.053 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.053 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.054 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.054 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.054 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.054 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.054 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.054 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.054 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.054 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.055 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-01-31T05:40:21.055 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-01-31T05:40:21.055 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.055 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.055 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.055 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-01-31T05:40:21.055 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.055 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.055 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.056 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.056 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.056 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.056 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.056 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-01-31T05:40:21.056 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-01-31T05:40:21.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.056 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.057 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.057 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-01-31T05:40:21.057 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.057 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.057 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.057 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.057 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.057 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.058 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.058 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:21.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:21.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.058 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.058 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.058 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:21.058 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.059 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.059 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.059 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.059 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.059 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.059 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-01-31T05:40:21.060 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-01-31T05:40:21.060 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-01-31T05:40:21.060 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.060 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-01-31T05:40:21.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-01-31T05:40:21.061 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.061 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.061 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.061 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-01-31T05:40:21.061 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.061 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.061 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.061 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-01-31T05:40:21.062 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.062 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.062 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.062 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-01-31T05:40:21.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-01-31T05:40:21.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.062 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.063 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.063 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-01-31T05:40:21.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.063 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-01-31T05:40:21.063 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.063 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.063 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.064 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-01-31T05:40:21.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-01-31T05:40:21.064 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.064 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.064 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.064 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.064 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.064 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.065 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.065 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-01-31T05:40:21.065 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.065 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:21.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:21.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.066 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.066 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-01-31T05:40:21.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-01-31T05:40:21.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.066 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.066 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-01-31T05:40:21.066 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.066 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.067 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.067 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.067 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.067 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.068 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.068 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.068 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.068 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.068 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.068 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.068 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.068 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.069 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.069 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.069 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.070 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.070 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.070 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.070 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.070 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.070 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.070 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.070 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.071 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.071 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.071 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.072 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.072 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.072 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.072 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.072 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.072 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.073 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.073 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.073 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.073 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.073 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.073 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.074 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.074 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.074 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.074 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.074 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.074 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.074 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.074 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.075 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.075 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.075 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-01-31T05:40:21.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-01-31T05:40:21.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.075 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.075 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.075 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.075 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.076 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.076 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.076 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-01-31T05:40:21.076 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.076 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.076 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.076 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-01-31T05:40:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-01-31T05:40:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-01-31T05:40:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-01-31T05:40:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-01-31T05:40:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-01-31T05:40:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-01-31T05:40:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-01-31T05:40:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-01-31T05:40:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-01-31T05:40:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-01-31T05:40:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-01-31T05:40:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-01-31T05:40:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-01-31T05:40:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-01-31T05:40:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-01-31T05:40:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-01-31T05:40:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-01-31T05:40:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-01-31T05:40:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-01-31T05:40:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-01-31T05:40:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-01-31T05:40:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-01-31T05:40:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-01-31T05:40:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-01-31T05:40:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-01-31T05:40:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-01-31T05:40:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-01-31T05:40:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-01-31T05:40:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-01-31T05:40:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-01-31T05:40:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-01-31T05:40:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-01-31T05:40:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-01-31T05:40:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-01-31T05:40:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-01-31T05:40:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-01-31T05:40:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-01-31T05:40:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.122 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.122 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.122 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.122 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.122 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.122 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.122 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.124 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.124 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.124 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-01-31T05:40:21.124 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-01-31T05:40:21.124 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.124 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.124 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.125 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-01-31T05:40:21.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-01-31T05:40:21.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.125 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.126 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-01-31T05:40:21.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-01-31T05:40:21.126 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.126 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.126 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.126 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-01-31T05:40:21.126 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.126 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.127 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.127 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-01-31T05:40:21.127 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.127 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.127 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-01-31T05:40:21.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-01-31T05:40:21.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.127 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.128 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-01-31T05:40:21.128 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.128 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.128 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.128 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-01-31T05:40:21.128 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.128 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.129 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-01-31T05:40:21.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-01-31T05:40:21.129 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.129 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.129 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.130 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-01-31T05:40:21.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.130 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-01-31T05:40:21.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-01-31T05:40:21.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-01-31T05:40:21.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.131 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.131 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.131 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-01-31T05:40:21.131 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-01-31T05:40:21.131 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.131 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.132 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-01-31T05:40:21.132 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.132 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.133 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.133 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.133 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.133 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.133 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.133 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.133 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.134 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.134 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.134 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.134 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.134 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.134 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.134 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.134 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.134 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.135 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.135 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.135 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.135 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.135 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.135 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.136 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.136 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.136 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.136 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.136 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.136 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.137 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.137 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.137 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.137 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.137 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.137 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.137 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.137 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.138 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.138 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.138 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.138 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.138 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.138 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.138 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.138 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.138 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.139 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.139 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.139 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.139 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.139 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.139 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.140 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.140 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.140 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.140 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.140 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.140 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.140 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.140 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-01-31T05:40:21.141 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.141 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.141 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.141 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.142 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.142 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.142 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.142 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.142 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.142 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.142 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.142 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.143 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.143 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.143 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.143 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.143 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.144 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.144 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.144 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.144 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.144 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.145 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.145 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.145 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.146 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.146 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.146 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.146 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.146 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.146 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.147 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.147 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.147 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.147 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.147 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.147 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.147 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.147 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.147 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.148 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.148 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.148 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.148 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.148 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.148 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.148 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.149 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.149 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.149 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.149 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.149 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.149 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.149 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.150 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.150 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.150 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.150 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.150 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-01-31T05:40:21.150 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.151 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.151 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.151 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.151 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.151 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.151 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.151 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.152 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.152 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.152 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.152 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.152 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.152 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.152 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.152 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.153 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.153 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.153 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.153 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.153 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.153 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.154 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.154 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.154 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.154 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.154 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.154 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.154 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.155 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.155 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.155 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.155 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.156 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.156 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.156 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.156 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.156 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.156 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.157 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.157 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.157 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.157 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.157 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.158 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.158 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.158 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.158 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.158 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.158 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.158 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.158 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.158 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.159 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.159 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.159 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-01-31T05:40:21.159 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.159 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.159 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.160 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-01-31T05:40:21.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-01-31T05:40:21.160 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.160 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.160 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.160 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.160 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.160 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.160 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.161 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-01-31T05:40:21.161 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.161 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.161 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.161 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.162 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.162 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.162 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.162 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.162 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.162 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.162 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.162 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.163 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.163 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.163 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.163 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.163 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.163 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.163 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.163 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.164 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.164 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.164 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.164 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.164 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.164 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.164 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.164 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.165 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.165 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.165 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.165 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.165 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.165 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.166 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.166 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.166 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.166 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.166 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.166 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.166 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.167 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.167 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.167 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.167 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.167 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.167 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.167 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.167 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.168 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.168 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.168 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.168 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.168 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.168 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.169 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.169 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.169 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.169 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.169 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.169 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.169 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.170 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-01-31T05:40:21.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-01-31T05:40:21.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.170 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.170 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.170 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.170 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.170 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.170 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.171 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-01-31T05:40:21.171 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-01-31T05:40:21.171 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-01-31T05:40:21.171 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.171 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-01-31T05:40:21.171 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.171 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.171 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.172 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.172 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.172 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-01-31T05:40:21.172 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.172 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.172 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.173 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.173 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.173 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.173 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.173 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.173 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.174 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.174 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.174 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.174 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.174 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.174 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.174 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.174 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.179 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.179 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.179 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.179 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.179 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.179 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.179 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.180 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.180 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.180 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.180 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.180 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.180 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.180 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.181 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.181 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.181 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.181 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.181 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.181 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.181 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-01-31T05:40:21.182 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.182 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.182 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.182 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.182 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.182 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.183 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.183 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.183 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.183 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.183 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.183 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.183 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.184 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.184 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.184 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.184 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.184 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.184 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.184 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.185 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.185 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.185 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.185 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.185 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.185 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.185 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-01-31T05:40:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.198 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.198 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.198 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.198 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.199 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.199 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.199 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.199 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.199 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.199 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.199 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.200 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:21.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:21.200 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.200 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.200 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.200 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:21.200 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.200 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.201 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.201 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.201 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.201 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.201 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.201 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-01-31T05:40:21.201 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-01-31T05:40:21.201 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:21.202 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.202 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.202 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-01-31T05:40:21.202 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-01-31T05:40:21.202 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.202 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.202 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-01-31T05:40:21.202 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-01-31T05:40:21.203 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.203 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.203 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.203 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-01-31T05:40:21.203 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.203 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.204 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.204 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-01-31T05:40:21.204 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-01-31T05:40:21.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.204 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.204 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.204 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.205 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.205 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-01-31T05:40:21.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.205 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.205 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-01-31T05:40:21.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-01-31T05:40:21.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.206 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.206 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.206 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.206 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.206 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.206 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.206 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.206 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.206 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.207 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.207 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.207 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.207 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.207 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.207 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.207 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.208 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.208 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.208 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.208 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.208 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.208 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.208 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.208 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.209 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.209 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.209 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.209 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.209 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-01-31T05:40:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-01-31T05:40:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-01-31T05:40:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-01-31T05:40:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-01-31T05:40:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-01-31T05:40:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-01-31T05:40:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-01-31T05:40:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-01-31T05:40:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-01-31T05:40:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-01-31T05:40:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-01-31T05:40:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-01-31T05:40:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.223 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.223 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.223 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.223 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.223 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.223 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.223 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.223 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.223 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.223 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.224 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.224 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.224 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.225 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.225 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.225 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.225 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.225 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.225 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.225 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.226 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.226 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.226 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.226 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.226 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.226 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.226 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.226 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.227 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.227 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.227 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.227 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.227 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.227 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.227 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.227 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.227 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.228 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.228 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.228 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.228 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.228 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.228 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.229 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.229 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.229 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.229 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.229 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.229 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.229 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.229 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.230 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.230 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-01-31T05:40:21.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-01-31T05:40:21.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.230 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.230 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.231 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.231 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.231 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.231 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.231 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-01-31T05:40:21.231 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-01-31T05:40:21.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.232 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.232 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-01-31T05:40:21.232 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.232 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.232 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.232 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.232 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.233 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.233 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.233 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-01-31T05:40:21.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-01-31T05:40:21.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:21.233 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.234 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-01-31T05:40:21.234 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.234 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.234 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.234 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.234 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.234 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.235 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-01-31T05:40:21.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-01-31T05:40:21.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.235 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-01-31T05:40:21.235 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.235 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.235 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.236 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-01-31T05:40:21.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.236 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-01-31T05:40:21.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-01-31T05:40:21.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.237 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.237 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-01-31T05:40:21.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.237 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.237 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.237 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-01-31T05:40:21.237 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.238 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.238 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.238 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-01-31T05:40:21.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-01-31T05:40:21.238 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.238 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.238 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.238 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-01-31T05:40:21.239 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.239 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.239 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.239 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.239 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.239 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.239 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.239 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-01-31T05:40:21.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-01-31T05:40:21.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.240 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.240 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.240 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-01-31T05:40:21.240 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.240 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.240 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.241 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-01-31T05:40:21.241 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-01-31T05:40:21.241 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.241 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.241 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.241 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.241 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.241 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-01-31T05:40:21.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-01-31T05:40:21.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.242 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.242 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.242 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-01-31T05:40:21.242 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-01-31T05:40:21.242 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-01-31T05:40:21.242 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.242 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.243 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.243 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.243 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.243 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-01-31T05:40:21.243 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.244 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.244 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.244 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.244 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.245 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.245 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.245 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.245 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.245 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.245 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.245 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.245 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.246 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.246 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.246 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.246 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.246 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.246 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.246 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.247 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.247 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.247 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.247 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.247 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.247 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.248 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.248 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-01-31T05:40:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-01-31T05:40:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-01-31T05:40:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-01-31T05:40:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-01-31T05:40:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-01-31T05:40:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-01-31T05:40:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-01-31T05:40:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-01-31T05:40:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-01-31T05:40:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-01-31T05:40:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-01-31T05:40:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-01-31T05:40:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-01-31T05:40:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-01-31T05:40:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-01-31T05:40:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-01-31T05:40:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "drive_group_interval": { 2024-01-31T05:40:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "drive_group_interval", 2024-01-31T05:40:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300.0", 2024-01-31T05:40:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-01-31T05:40:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-01-31T05:40:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-01-31T05:40:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-01-31T05:40:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-01-31T05:40:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-01-31T05:40:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-01-31T05:40:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-01-31T05:40:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-01-31T05:40:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-01-31T05:40:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-01-31T05:40:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-01-31T05:40:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-01-31T05:40:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-01-31T05:40:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-01-31T05:40:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-01-31T05:40:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-01-31T05:40:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-01-31T05:40:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-01-31T05:40:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-01-31T05:40:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-01-31T05:40:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-01-31T05:40:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-01-31T05:40:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.318 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-01-31T05:40:21.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-01-31T05:40:21.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.318 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.319 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.319 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-01-31T05:40:21.319 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-01-31T05:40:21.319 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.319 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.319 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.319 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.319 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.320 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-01-31T05:40:21.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-01-31T05:40:21.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.320 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.320 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.320 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-01-31T05:40:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-01-31T05:40:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-01-31T05:40:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-01-31T05:40:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-01-31T05:40:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-01-31T05:40:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-01-31T05:40:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-01-31T05:40:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-01-31T05:40:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-01-31T05:40:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-01-31T05:40:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-01-31T05:40:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-01-31T05:40:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.352 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.352 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.352 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.352 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.352 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.352 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.352 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.353 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.353 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.353 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.353 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.353 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.354 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.354 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-01-31T05:40:21.355 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.355 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.355 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.355 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-01-31T05:40:21.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-01-31T05:40:21.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.355 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.355 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.356 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-01-31T05:40:21.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.356 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.356 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.356 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.356 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.356 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.357 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:21.357 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:21.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.357 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.357 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.357 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-01-31T05:40:21.357 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.357 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.358 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.358 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.358 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.359 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.359 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.359 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.359 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.359 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.359 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.359 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.359 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.360 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.360 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.360 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.360 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.361 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.361 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.362 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.362 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.362 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.362 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.363 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.363 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.363 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.363 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.363 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.363 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.364 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.364 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.364 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.364 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.364 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.364 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.365 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.365 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.365 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.365 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.365 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.366 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.366 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.366 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.366 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.366 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-01-31T05:40:21.367 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.367 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.367 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.367 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-01-31T05:40:21.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-01-31T05:40:21.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.367 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.367 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.368 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.368 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.368 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.368 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-01-31T05:40:21.368 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.368 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.368 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.369 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-01-31T05:40:21.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-01-31T05:40:21.369 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.369 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.369 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.369 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.369 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.369 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.370 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.370 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-01-31T05:40:21.370 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.370 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.370 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.370 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-01-31T05:40:21.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-01-31T05:40:21.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.371 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.371 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.371 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-01-31T05:40:21.371 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.372 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.372 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-01-31T05:40:21.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-01-31T05:40:21.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.372 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.373 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.373 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.373 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.373 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.373 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-01-31T05:40:21.373 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.374 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-01-31T05:40:21.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-01-31T05:40:21.374 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.374 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.374 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.374 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.375 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.375 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-01-31T05:40:21.375 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.375 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-01-31T05:40:21.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-01-31T05:40:21.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-01-31T05:40:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-01-31T05:40:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.378 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.378 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.378 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.378 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-01-31T05:40:21.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-01-31T05:40:21.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-01-31T05:40:21.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.379 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.379 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-01-31T05:40:21.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-01-31T05:40:21.380 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.380 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.381 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.381 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.381 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.381 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.381 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.381 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.381 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.382 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:21.382 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:21.382 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.382 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.382 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.382 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-01-31T05:40:21.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-01-31T05:40:21.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.383 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.383 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.383 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.383 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.383 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.383 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-01-31T05:40:21.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-01-31T05:40:21.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-01-31T05:40:21.384 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.384 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.384 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.384 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.384 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.385 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.385 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-01-31T05:40:21.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-01-31T05:40:21.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.385 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.385 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.385 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.386 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.386 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.386 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.386 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.386 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.386 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.386 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-01-31T05:40:21.387 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-01-31T05:40:21.387 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.387 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.387 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.388 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.388 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.388 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.388 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.389 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.389 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.389 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.389 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.389 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.389 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.389 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.389 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.390 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.390 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.390 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.390 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.390 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.391 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.391 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.391 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.391 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.391 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.391 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.392 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.392 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.392 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.392 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.392 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.393 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.393 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.393 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.393 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.393 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.394 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.394 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.394 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.394 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.395 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.395 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.396 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-01-31T05:40:21.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-01-31T05:40:21.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.396 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.396 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.397 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.397 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.397 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.397 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.398 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-01-31T05:40:21.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-01-31T05:40:21.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.398 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.398 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.399 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.399 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-01-31T05:40:21.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-01-31T05:40:21.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.399 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.400 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-01-31T05:40:21.400 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.400 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.400 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.400 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.400 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-01-31T05:40:21.401 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.401 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.401 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.402 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.402 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.402 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.402 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.402 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.403 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.403 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.403 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.403 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.403 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.403 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.403 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.403 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.403 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.404 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.404 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.404 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.404 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.405 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.405 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.405 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.405 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.405 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.405 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.406 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.406 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.406 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.406 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.406 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.407 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.407 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.407 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.407 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.407 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.407 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.407 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.407 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.407 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.407 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.408 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.408 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.409 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.409 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.409 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-01-31T05:40:21.410 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.410 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.410 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.410 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.411 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.411 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.411 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.411 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.411 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.412 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.412 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.412 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.412 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.412 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.412 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.412 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.413 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.413 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.413 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.413 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.413 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.414 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.414 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.414 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.414 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.414 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.414 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.415 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.415 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.416 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.416 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.416 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.416 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.416 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.416 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.416 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.416 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.417 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.417 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.417 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.417 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.418 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.418 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.418 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.418 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-01-31T05:40:21.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-01-31T05:40:21.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.419 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-01-31T05:40:21.419 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.419 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-01-31T05:40:21.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.420 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.420 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.420 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-01-31T05:40:21.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-01-31T05:40:21.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.420 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.421 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.421 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.421 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.421 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-01-31T05:40:21.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.422 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-01-31T05:40:21.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-01-31T05:40:21.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.422 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:21.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.422 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.422 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.423 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-01-31T05:40:21.423 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.423 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.423 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.423 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-01-31T05:40:21.424 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.424 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.424 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.424 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-01-31T05:40:21.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-01-31T05:40:21.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:21.424 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.425 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.425 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-01-31T05:40:21.425 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.425 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.425 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.425 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.425 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.426 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-01-31T05:40:21.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-01-31T05:40:21.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.426 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.426 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.426 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.426 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.427 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:21.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:21.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.427 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.428 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:21.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.428 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.428 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.428 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.428 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.428 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.429 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.429 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.429 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.429 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.430 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.430 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.430 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.430 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.430 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.430 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.430 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.430 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.431 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.431 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.432 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.432 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.432 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.432 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.432 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.432 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.433 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.433 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.434 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.434 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.434 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.434 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.435 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.435 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.435 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.435 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.435 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.436 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.436 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.436 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.438 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.438 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.438 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.439 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-01-31T05:40:21.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-01-31T05:40:21.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.440 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.440 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.440 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.440 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.440 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.440 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.440 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.441 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-01-31T05:40:21.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-01-31T05:40:21.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.441 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.441 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-01-31T05:40:21.441 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.442 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.442 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.442 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.442 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.442 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-01-31T05:40:21.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-01-31T05:40:21.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.443 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.443 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-01-31T05:40:21.443 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.443 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.443 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.443 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.444 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.444 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-01-31T05:40:21.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.444 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.444 INFO:teuthology.orchestra.run.smithi045.stdout: "modules": [ 2024-01-31T05:40:21.444 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-01-31T05:40:21.445 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard", 2024-01-31T05:40:21.445 INFO:teuthology.orchestra.run.smithi045.stdout: "iostat", 2024-01-31T05:40:21.445 INFO:teuthology.orchestra.run.smithi045.stdout: "nfs", 2024-01-31T05:40:21.445 INFO:teuthology.orchestra.run.smithi045.stdout: "restful", 2024-01-31T05:40:21.445 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_schedule" 2024-01-31T05:40:21.445 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.445 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-01-31T05:40:21.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-01-31T05:40:21.446 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.446 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.446 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.446 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:21.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:21.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.446 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.446 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:21.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.447 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.447 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.447 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-01-31T05:40:21.447 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.448 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.448 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.448 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.448 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.448 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.448 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.448 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.449 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.449 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.449 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.449 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.449 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.449 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.449 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.450 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.450 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.450 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.451 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.451 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.451 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.451 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.452 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.452 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.452 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.452 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.453 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.453 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.453 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.453 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.453 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.453 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.453 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.453 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.454 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.454 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.454 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.454 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.454 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.454 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.455 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.455 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.455 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.455 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.455 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.455 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.456 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.456 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-01-31T05:40:21.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-01-31T05:40:21.456 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.456 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.456 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.456 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.457 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.457 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.457 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-01-31T05:40:21.457 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.457 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.457 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.457 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-01-31T05:40:21.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-01-31T05:40:21.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.458 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.458 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-01-31T05:40:21.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.458 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-01-31T05:40:21.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.459 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.459 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.459 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-01-31T05:40:21.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-01-31T05:40:21.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.460 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.460 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.460 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.460 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-01-31T05:40:21.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.460 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.461 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-01-31T05:40:21.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-01-31T05:40:21.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.461 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.461 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.461 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.461 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.461 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.462 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-01-31T05:40:21.462 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.462 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.462 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.462 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-01-31T05:40:21.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-01-31T05:40:21.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.463 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.463 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.463 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-01-31T05:40:21.463 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.463 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.463 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.463 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-01-31T05:40:21.463 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.463 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.464 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.464 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-01-31T05:40:21.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-01-31T05:40:21.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.464 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.464 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.465 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.465 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-01-31T05:40:21.465 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.465 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.465 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-01-31T05:40:21.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-01-31T05:40:21.466 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.466 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.466 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.466 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.466 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.466 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-01-31T05:40:21.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.467 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.467 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.467 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-01-31T05:40:21.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-01-31T05:40:21.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.467 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.467 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.468 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.468 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.468 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-01-31T05:40:21.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.468 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.468 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-01-31T05:40:21.469 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.469 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.469 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.469 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-01-31T05:40:21.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-01-31T05:40:21.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.470 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.470 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.470 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.470 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.470 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.470 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-01-31T05:40:21.470 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.471 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-01-31T05:40:21.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-01-31T05:40:21.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.471 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.471 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.471 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-01-31T05:40:21.472 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.472 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.472 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.472 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-01-31T05:40:21.472 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-31T05:40:21.472 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.472 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.473 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-01-31T05:40:21.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-01-31T05:40:21.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:21.473 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.473 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.473 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:21.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-01-31T05:40:21.473 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-01-31T05:40:21.473 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.474 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-01-31T05:40:21.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-31T05:40:21.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.474 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.474 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-01-31T05:40:21.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-01-31T05:40:21.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:21.474 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.475 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.475 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-01-31T05:40:21.475 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-01-31T05:40:21.475 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-01-31T05:40:21.475 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.475 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-01-31T05:40:21.475 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.476 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-01-31T05:40:21.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-01-31T05:40:21.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.476 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.476 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-01-31T05:40:21.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.476 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.477 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.477 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-01-31T05:40:21.477 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-01-31T05:40:21.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.477 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.477 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-01-31T05:40:21.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-01-31T05:40:21.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:21.478 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-01-31T05:40:21.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-01-31T05:40:21.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-01-31T05:40:21.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-01-31T05:40:21.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-01-31T05:40:21.478 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.479 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.479 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-01-31T05:40:21.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-01-31T05:40:21.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.479 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.479 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-01-31T05:40:21.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-01-31T05:40:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-31T05:40:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-01-31T05:40:21.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-01-31T05:40:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-01-31T05:40:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-01-31T05:40:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-01-31T05:40:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-31T05:40:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.483 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.485 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.486 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.487 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.488 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.489 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-01-31T05:40:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-01-31T05:40:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:21.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-01-31T05:40:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-01-31T05:40:21.491 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-01-31T05:40:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-01-31T05:40:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.492 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-01-31T05:40:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-01-31T05:40:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap" 2024-01-31T05:40:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-01-31T05:40:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-01-31T05:40:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-01-31T05:40:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.494 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-01-31T05:40:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.495 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-01-31T05:40:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.496 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-01-31T05:40:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-01-31T05:40:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-01-31T05:40:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-01-31T05:40:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-01-31T05:40:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-01-31T05:40:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-01-31T05:40:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-01-31T05:40:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.498 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-01-31T05:40:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-01-31T05:40:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.500 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-01-31T05:40:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-01-31T05:40:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-01-31T05:40:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.501 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-01-31T05:40:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-01-31T05:40:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-01-31T05:40:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.502 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-01-31T05:40:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-01-31T05:40:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-01-31T05:40:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-01-31T05:40:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-01-31T05:40:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.505 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.505 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-01-31T05:40:21.505 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.505 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.505 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.505 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-01-31T05:40:21.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-01-31T05:40:21.505 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.505 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.506 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.506 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.506 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.506 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-01-31T05:40:21.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-01-31T05:40:21.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.506 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.507 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-01-31T05:40:21.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-01-31T05:40:21.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.507 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.507 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.507 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:21.507 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.507 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.508 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.508 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-01-31T05:40:21.508 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.508 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.508 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.508 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-01-31T05:40:21.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-01-31T05:40:21.508 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-01-31T05:40:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-01-31T05:40:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.509 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-01-31T05:40:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-01-31T05:40:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-01-31T05:40:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-01-31T05:40:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-01-31T05:40:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-01-31T05:40:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.511 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-01-31T05:40:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-01-31T05:40:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.512 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.513 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.513 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.513 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.513 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-01-31T05:40:21.513 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.513 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-01-31T05:40:21.513 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-01-31T05:40:21.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.514 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.514 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.514 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.514 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.514 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.514 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.514 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-01-31T05:40:21.515 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.515 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.515 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.515 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-01-31T05:40:21.515 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-01-31T05:40:21.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.515 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.515 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.516 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.516 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.516 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.516 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-01-31T05:40:21.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.516 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.517 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-01-31T05:40:21.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-01-31T05:40:21.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.517 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.517 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.517 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-01-31T05:40:21.517 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.517 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.517 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.518 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-01-31T05:40:21.518 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.518 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.518 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-01-31T05:40:21.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-01-31T05:40:21.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.519 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.519 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-01-31T05:40:21.519 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.519 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.519 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.519 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-01-31T05:40:21.519 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.519 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.519 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.520 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-01-31T05:40:21.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-01-31T05:40:21.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.520 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.520 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.520 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-01-31T05:40:21.520 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.521 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-01-31T05:40:21.521 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.521 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.521 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.521 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-01-31T05:40:21.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-01-31T05:40:21.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.522 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.522 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.522 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-01-31T05:40:21.522 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.522 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.522 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.523 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-01-31T05:40:21.523 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.523 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.523 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.523 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-01-31T05:40:21.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-01-31T05:40:21.523 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.523 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.524 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.524 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-01-31T05:40:21.524 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.524 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.524 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.524 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-01-31T05:40:21.524 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.524 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.525 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.525 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-01-31T05:40:21.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-01-31T05:40:21.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.525 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.525 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.525 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-01-31T05:40:21.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.526 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.526 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.526 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-01-31T05:40:21.526 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.526 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.526 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.526 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-01-31T05:40:21.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-01-31T05:40:21.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.527 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.527 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-01-31T05:40:21.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.527 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-01-31T05:40:21.528 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.528 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.528 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.528 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-01-31T05:40:21.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-01-31T05:40:21.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.529 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.529 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.529 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-01-31T05:40:21.529 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.529 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.529 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-01-31T05:40:21.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.530 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.530 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.530 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-01-31T05:40:21.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-01-31T05:40:21.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.530 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.530 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-01-31T05:40:21.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.531 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.531 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.531 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-01-31T05:40:21.531 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.531 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.531 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.531 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-01-31T05:40:21.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-01-31T05:40:21.532 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.532 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.532 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.532 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-01-31T05:40:21.532 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.532 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.532 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.533 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-01-31T05:40:21.533 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.533 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-01-31T05:40:21.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-01-31T05:40:21.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.534 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.534 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.534 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-01-31T05:40:21.534 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.534 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.534 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.534 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-01-31T05:40:21.534 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.534 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.535 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.535 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-01-31T05:40:21.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-01-31T05:40:21.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.535 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.535 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.535 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-01-31T05:40:21.536 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.536 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.536 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.536 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-01-31T05:40:21.536 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.536 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.536 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.537 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-01-31T05:40:21.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-01-31T05:40:21.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.537 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.537 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.537 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-01-31T05:40:21.537 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.538 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.538 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.538 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-01-31T05:40:21.538 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.538 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.538 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-01-31T05:40:21.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-01-31T05:40:21.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.539 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.539 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.539 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-01-31T05:40:21.539 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.539 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.539 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-01-31T05:40:21.540 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.540 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.540 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.540 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-01-31T05:40:21.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-01-31T05:40:21.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.541 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-01-31T05:40:21.541 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.541 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.541 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.541 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-01-31T05:40:21.541 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.541 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.542 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.542 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-01-31T05:40:21.542 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-01-31T05:40:21.542 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.542 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.542 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.543 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.543 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.543 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-01-31T05:40:21.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.543 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.543 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-01-31T05:40:21.544 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-01-31T05:40:21.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.544 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.544 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:21.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.544 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.544 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.545 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-01-31T05:40:21.545 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.545 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.545 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.545 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-01-31T05:40:21.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-01-31T05:40:21.545 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.546 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.546 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.546 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-01-31T05:40:21.546 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.546 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.546 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-01-31T05:40:21.546 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.547 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.547 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.547 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-01-31T05:40:21.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-01-31T05:40:21.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.547 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.547 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-01-31T05:40:21.547 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.548 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.548 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.548 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-01-31T05:40:21.548 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.548 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.548 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.548 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-01-31T05:40:21.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-01-31T05:40:21.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.549 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.549 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.549 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-01-31T05:40:21.549 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.549 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.549 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-01-31T05:40:21.550 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.550 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.550 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.550 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-01-31T05:40:21.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-01-31T05:40:21.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.551 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.551 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.551 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.551 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.551 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-01-31T05:40:21.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.551 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.552 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-01-31T05:40:21.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-01-31T05:40:21.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.552 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.552 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:21.552 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.552 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.552 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.553 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-01-31T05:40:21.553 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.553 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.553 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.553 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-01-31T05:40:21.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-01-31T05:40:21.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.553 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.554 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.554 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:21.554 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.554 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.554 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.554 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-01-31T05:40:21.554 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.554 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.555 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-01-31T05:40:21.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-01-31T05:40:21.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.555 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.555 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.555 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.555 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.555 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.556 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.556 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-01-31T05:40:21.556 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.556 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.556 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.556 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.557 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.557 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.557 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.557 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.557 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.557 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.557 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.557 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.557 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.558 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.558 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.558 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.558 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.558 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.558 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.558 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.559 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-01-31T05:40:21.559 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-01-31T05:40:21.559 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.559 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.559 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.559 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.559 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.559 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.560 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-01-31T05:40:21.560 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.560 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.560 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.560 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.560 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.561 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.561 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.561 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.561 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-01-31T05:40:21.561 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.561 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.561 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.562 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.562 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.562 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.562 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.562 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.562 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.562 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.563 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.563 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.563 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.563 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.563 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.563 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.563 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.564 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.564 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.564 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.564 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.564 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.564 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.565 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.565 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.565 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.565 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.565 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.565 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.565 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.565 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.566 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-01-31T05:40:21.566 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-01-31T05:40:21.566 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.566 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.566 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.566 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.567 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.567 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.567 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-01-31T05:40:21.567 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.567 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.567 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.567 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-01-31T05:40:21.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-01-31T05:40:21.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.568 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.568 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-01-31T05:40:21.568 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.568 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.568 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.568 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-01-31T05:40:21.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.569 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.569 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.569 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-01-31T05:40:21.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-01-31T05:40:21.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.569 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.570 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.570 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:21.570 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.570 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.570 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.570 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-01-31T05:40:21.570 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.570 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.571 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-01-31T05:40:21.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-01-31T05:40:21.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:21.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.572 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.572 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-01-31T05:40:21.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.572 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-01-31T05:40:21.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-01-31T05:40:21.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.574 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-01-31T05:40:21.574 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.574 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.574 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.574 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-01-31T05:40:21.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-01-31T05:40:21.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.574 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.575 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.575 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-01-31T05:40:21.575 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.575 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.575 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.575 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-01-31T05:40:21.575 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-01-31T05:40:21.575 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.575 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-01-31T05:40:21.576 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.576 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.576 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.576 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-01-31T05:40:21.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-01-31T05:40:21.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.577 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-01-31T05:40:21.577 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-01-31T05:40:21.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.578 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-01-31T05:40:21.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-01-31T05:40:21.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.578 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.578 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.578 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.578 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.578 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.579 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-01-31T05:40:21.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.579 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.579 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-01-31T05:40:21.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-01-31T05:40:21.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.580 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.580 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.580 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.580 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-01-31T05:40:21.580 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.581 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.581 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.581 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-01-31T05:40:21.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-01-31T05:40:21.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.581 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.582 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-01-31T05:40:21.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.582 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.582 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-01-31T05:40:21.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-01-31T05:40:21.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.583 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.583 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.583 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.583 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.583 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.583 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.583 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-01-31T05:40:21.583 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.584 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.584 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.584 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-01-31T05:40:21.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-01-31T05:40:21.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.584 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.585 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.585 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.585 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.585 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.585 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-01-31T05:40:21.585 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.585 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.586 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-01-31T05:40:21.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-01-31T05:40:21.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.586 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.586 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-01-31T05:40:21.586 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.586 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.587 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-01-31T05:40:21.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.587 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-01-31T05:40:21.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-01-31T05:40:21.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.587 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.588 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.588 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.588 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.588 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.588 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-01-31T05:40:21.588 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.588 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.589 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.589 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-01-31T05:40:21.589 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-01-31T05:40:21.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.589 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.589 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.590 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.590 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.590 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-01-31T05:40:21.590 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.590 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.590 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.590 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-01-31T05:40:21.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-01-31T05:40:21.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.591 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.591 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.591 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.591 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.591 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.591 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-01-31T05:40:21.591 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.591 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.592 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.592 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-01-31T05:40:21.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-01-31T05:40:21.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.592 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.593 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.593 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.593 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.593 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-01-31T05:40:21.593 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.593 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-01-31T05:40:21.594 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-01-31T05:40:21.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.594 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.594 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.594 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.594 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.594 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.594 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-01-31T05:40:21.595 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.595 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.595 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.595 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-01-31T05:40:21.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-01-31T05:40:21.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.595 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.595 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.596 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-01-31T05:40:21.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.596 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.596 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.597 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-01-31T05:40:21.597 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.597 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.597 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.597 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.598 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.598 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.598 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.598 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.598 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.598 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.598 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.599 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.599 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.599 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.599 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.599 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.599 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.599 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.599 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.599 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.600 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.600 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.600 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.600 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.600 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.600 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.600 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.601 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.601 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.601 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.601 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.601 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.601 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.601 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.601 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.602 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.602 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.602 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.602 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.602 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.602 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.603 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.603 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.603 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.603 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.603 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.603 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.604 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.604 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.604 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.604 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.605 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.605 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.605 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.605 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.605 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.605 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.605 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.605 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-01-31T05:40:21.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-01-31T05:40:21.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.606 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.606 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.606 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-01-31T05:40:21.606 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.606 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.606 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.606 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-01-31T05:40:21.607 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.607 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.607 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.607 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-01-31T05:40:21.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-01-31T05:40:21.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.608 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.608 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-01-31T05:40:21.608 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.608 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.608 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.608 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-01-31T05:40:21.608 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.609 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-01-31T05:40:21.609 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.609 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.610 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.610 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-01-31T05:40:21.610 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-01-31T05:40:21.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.610 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.610 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:21.610 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.611 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.611 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.611 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.611 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.611 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.611 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-01-31T05:40:21.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-01-31T05:40:21.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.612 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.612 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.612 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.612 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.612 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.612 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.613 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.613 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.613 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.613 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.613 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-01-31T05:40:21.613 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-01-31T05:40:21.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.613 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.614 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.614 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.614 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.614 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.614 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.614 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.614 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.615 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.615 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-01-31T05:40:21.615 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-01-31T05:40:21.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.615 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.615 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.615 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.616 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.616 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.616 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.616 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.616 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.616 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.616 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-01-31T05:40:21.617 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-01-31T05:40:21.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.617 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.617 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.617 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.617 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.617 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.617 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.617 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.618 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.618 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.618 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.618 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-01-31T05:40:21.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-01-31T05:40:21.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.619 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.619 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.619 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.619 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.619 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.619 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.619 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.619 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.620 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-01-31T05:40:21.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-01-31T05:40:21.620 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.620 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.620 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.620 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.620 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.621 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.621 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.621 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.621 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.621 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.621 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.621 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-01-31T05:40:21.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-01-31T05:40:21.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.622 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.622 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.622 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.622 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.622 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.622 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.623 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.623 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.623 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.623 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-01-31T05:40:21.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-01-31T05:40:21.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.623 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.624 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.624 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.624 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.624 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.624 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.625 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-01-31T05:40:21.625 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-01-31T05:40:21.625 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.625 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.625 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.625 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.626 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.626 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.626 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.626 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.626 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.626 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.626 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.626 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-01-31T05:40:21.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-01-31T05:40:21.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.627 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.627 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.627 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.627 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.627 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.628 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.628 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.628 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.628 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-01-31T05:40:21.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-01-31T05:40:21.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.629 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.629 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.629 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.629 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.629 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.629 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.629 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.630 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.630 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.630 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-01-31T05:40:21.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-01-31T05:40:21.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.630 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.630 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.631 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.631 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.631 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.631 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.631 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.632 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-01-31T05:40:21.632 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-01-31T05:40:21.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.632 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.633 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.633 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.633 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.633 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-01-31T05:40:21.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-01-31T05:40:21.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.634 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.634 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.634 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-01-31T05:40:21.634 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.634 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.634 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.634 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.634 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.635 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.635 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-01-31T05:40:21.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-01-31T05:40:21.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.635 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.635 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.636 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.636 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.636 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.636 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.636 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.636 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.637 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-01-31T05:40:21.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-01-31T05:40:21.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.637 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.637 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.637 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.637 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.637 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.637 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.638 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.638 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.638 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.638 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.638 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-01-31T05:40:21.638 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-01-31T05:40:21.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.639 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.639 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.639 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-01-31T05:40:21.639 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.639 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.639 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.639 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.640 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.640 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.640 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-01-31T05:40:21.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-01-31T05:40:21.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.640 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.641 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.641 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.641 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.641 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.641 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.641 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.641 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.642 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-01-31T05:40:21.642 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-01-31T05:40:21.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.642 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.643 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.643 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.643 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-01-31T05:40:21.643 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-01-31T05:40:21.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.644 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.644 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.644 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.644 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.644 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.645 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.645 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.645 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-01-31T05:40:21.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-01-31T05:40:21.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.646 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.646 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.646 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.646 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.646 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.646 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.646 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.646 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.647 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-01-31T05:40:21.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-01-31T05:40:21.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.647 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.647 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.647 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.648 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.648 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.648 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.648 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.648 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-01-31T05:40:21.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-01-31T05:40:21.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.649 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.649 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.649 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.649 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.649 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.650 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.650 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.650 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.650 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.650 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-01-31T05:40:21.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-01-31T05:40:21.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.650 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-01-31T05:40:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-01-31T05:40:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.652 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-01-31T05:40:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-01-31T05:40:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.654 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-01-31T05:40:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-01-31T05:40:21.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.656 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-01-31T05:40:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-01-31T05:40:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.657 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-01-31T05:40:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-01-31T05:40:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.659 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-01-31T05:40:21.660 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-01-31T05:40:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-01-31T05:40:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-01-31T05:40:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.663 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-01-31T05:40:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-01-31T05:40:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-01-31T05:40:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.664 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-01-31T05:40:21.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-01-31T05:40:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.666 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-01-31T05:40:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-01-31T05:40:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-01-31T05:40:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-01-31T05:40:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-01-31T05:40:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-01-31T05:40:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.669 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-01-31T05:40:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-01-31T05:40:21.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.671 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-01-31T05:40:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-01-31T05:40:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.672 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.673 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-01-31T05:40:21.673 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.673 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.673 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.673 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.673 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.673 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.673 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.674 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-01-31T05:40:21.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-01-31T05:40:21.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.674 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.674 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.674 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.674 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.675 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.675 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.675 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.675 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.675 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.675 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-01-31T05:40:21.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-01-31T05:40:21.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.676 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.676 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.676 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.676 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.676 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.676 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.676 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.676 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.677 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.677 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.677 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-01-31T05:40:21.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-01-31T05:40:21.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.677 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.678 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:21.678 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.678 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.678 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.678 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.678 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.678 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.678 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.679 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-01-31T05:40:21.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-01-31T05:40:21.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.679 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.679 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:21.679 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.680 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.680 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.680 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.680 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.680 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.680 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.680 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-01-31T05:40:21.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-01-31T05:40:21.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.681 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.681 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.681 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-01-31T05:40:21.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.681 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.681 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.682 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.682 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.682 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.682 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.682 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-01-31T05:40:21.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-01-31T05:40:21.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.682 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.683 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.683 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.683 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.683 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.683 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.683 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.683 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.683 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.683 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.684 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-01-31T05:40:21.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-01-31T05:40:21.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.684 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.684 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.684 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.684 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.684 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.685 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.685 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.685 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.685 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.685 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.685 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-01-31T05:40:21.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-01-31T05:40:21.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.686 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.686 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.686 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.686 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.686 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.686 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.686 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-01-31T05:40:21.686 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.687 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.687 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-01-31T05:40:21.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-01-31T05:40:21.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.687 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.687 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.687 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-01-31T05:40:21.687 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.688 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.688 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.688 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.688 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.688 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.688 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-01-31T05:40:21.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-01-31T05:40:21.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.689 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.689 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.689 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.689 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.689 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.689 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.690 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.690 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.690 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.690 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.690 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.690 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.690 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.691 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.691 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.691 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.691 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.691 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.691 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.691 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.691 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.692 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.692 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.692 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.692 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.692 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.692 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.692 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.692 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.693 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.693 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.693 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.693 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.693 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.693 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.694 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.694 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.694 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.694 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.694 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.695 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.695 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.695 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.695 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.695 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.695 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.695 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.695 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.695 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.696 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.696 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.696 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.696 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.696 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.696 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.696 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.696 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.697 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.697 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.697 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.697 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.697 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.697 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.698 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.698 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.698 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.698 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.698 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.698 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-01-31T05:40:21.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-01-31T05:40:21.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.699 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.699 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.699 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.699 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.699 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.699 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-01-31T05:40:21.699 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.700 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.700 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-01-31T05:40:21.700 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-01-31T05:40:21.700 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.700 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.700 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.700 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.701 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.701 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.701 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.701 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.701 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.701 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.701 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.702 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-01-31T05:40:21.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-01-31T05:40:21.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.702 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.702 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.702 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-01-31T05:40:21.702 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.702 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.702 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.703 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.703 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.703 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.703 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-01-31T05:40:21.703 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-01-31T05:40:21.703 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.703 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.704 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.704 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-01-31T05:40:21.704 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.704 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.704 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.704 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.704 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.704 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.705 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.705 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-01-31T05:40:21.705 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-01-31T05:40:21.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.705 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.705 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.705 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.705 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.706 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.706 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.706 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.706 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.706 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.706 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.706 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-01-31T05:40:21.706 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-01-31T05:40:21.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.707 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.707 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.707 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-01-31T05:40:21.707 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.707 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.707 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.707 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.707 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.708 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.708 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.708 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-01-31T05:40:21.708 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-01-31T05:40:21.708 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.708 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.708 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-01-31T05:40:21.709 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.709 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.709 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.709 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.709 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-01-31T05:40:21.709 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.709 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.709 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.710 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.710 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.710 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-01-31T05:40:21.710 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-01-31T05:40:21.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.710 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.710 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.710 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-01-31T05:40:21.711 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-01-31T05:40:21.711 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-01-31T05:40:21.711 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.711 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.711 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.711 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.711 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.711 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-01-31T05:40:21.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-01-31T05:40:21.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.712 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.712 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.712 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.712 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.712 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.712 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.713 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.713 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.713 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.713 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.713 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.713 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.714 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-01-31T05:40:21.714 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.714 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.714 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-01-31T05:40:21.714 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-01-31T05:40:21.714 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.714 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.714 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.715 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.715 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.715 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.715 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.715 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-01-31T05:40:21.715 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.715 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.715 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.716 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.716 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.716 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.716 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.716 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.716 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.716 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.717 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.717 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.717 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.717 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.717 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.717 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.717 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.717 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.717 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.718 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.718 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.718 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.718 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.718 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.718 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.718 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.719 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.719 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.719 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.719 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.719 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.719 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.719 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.719 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.719 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.720 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.720 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.720 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.720 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.720 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.720 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.720 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.721 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.721 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.721 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.721 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.721 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.721 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.721 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.721 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.721 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.722 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.722 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.722 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.722 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.722 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.723 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.723 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.723 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.723 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.723 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.723 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.723 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.723 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.724 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.724 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.724 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-01-31T05:40:21.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-01-31T05:40:21.724 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.724 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.724 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.725 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-01-31T05:40:21.725 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.725 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.725 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.725 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-01-31T05:40:21.725 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.725 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.725 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.726 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-01-31T05:40:21.726 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-01-31T05:40:21.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.726 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.726 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.726 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-01-31T05:40:21.726 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.726 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.727 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.727 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-01-31T05:40:21.727 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.727 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.727 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.727 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-01-31T05:40:21.727 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-01-31T05:40:21.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.728 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.728 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.728 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-01-31T05:40:21.728 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.728 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.728 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.728 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-01-31T05:40:21.728 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-01-31T05:40:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-01-31T05:40:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-01-31T05:40:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-01-31T05:40:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-01-31T05:40:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-01-31T05:40:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-01-31T05:40:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-01-31T05:40:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-01-31T05:40:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-01-31T05:40:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-01-31T05:40:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-01-31T05:40:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-01-31T05:40:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-01-31T05:40:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-01-31T05:40:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-01-31T05:40:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-01-31T05:40:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-01-31T05:40:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-01-31T05:40:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-01-31T05:40:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-01-31T05:40:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-01-31T05:40:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-01-31T05:40:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-01-31T05:40:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-01-31T05:40:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-01-31T05:40:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-01-31T05:40:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-01-31T05:40:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-01-31T05:40:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-01-31T05:40:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-01-31T05:40:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-01-31T05:40:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-01-31T05:40:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-01-31T05:40:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-01-31T05:40:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-01-31T05:40:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-01-31T05:40:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-01-31T05:40:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-01-31T05:40:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-01-31T05:40:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-01-31T05:40:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-01-31T05:40:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-01-31T05:40:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-01-31T05:40:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-01-31T05:40:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-01-31T05:40:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-01-31T05:40:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-01-31T05:40:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-01-31T05:40:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-01-31T05:40:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-01-31T05:40:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-01-31T05:40:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-01-31T05:40:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-01-31T05:40:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-01-31T05:40:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-01-31T05:40:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-01-31T05:40:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-01-31T05:40:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-01-31T05:40:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-01-31T05:40:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-01-31T05:40:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-01-31T05:40:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-01-31T05:40:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-01-31T05:40:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-01-31T05:40:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-01-31T05:40:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-01-31T05:40:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-01-31T05:40:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-01-31T05:40:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-01-31T05:40:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-01-31T05:40:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-01-31T05:40:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-01-31T05:40:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-01-31T05:40:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-01-31T05:40:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-01-31T05:40:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-01-31T05:40:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-01-31T05:40:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-01-31T05:40:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-01-31T05:40:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-01-31T05:40:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-01-31T05:40:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-01-31T05:40:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-01-31T05:40:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-01-31T05:40:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-01-31T05:40:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-01-31T05:40:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-01-31T05:40:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-01-31T05:40:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-01-31T05:40:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-01-31T05:40:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-01-31T05:40:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-01-31T05:40:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-01-31T05:40:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-01-31T05:40:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-01-31T05:40:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-01-31T05:40:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-01-31T05:40:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-01-31T05:40:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-01-31T05:40:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.833 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.834 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.838 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.839 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.840 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-01-31T05:40:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.841 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.843 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.844 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.845 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.846 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.848 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-01-31T05:40:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-01-31T05:40:21.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-01-31T05:40:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.851 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-01-31T05:40:21.852 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.852 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.852 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.852 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.853 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.853 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.853 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.853 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.853 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.853 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.853 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.853 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.854 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.854 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.854 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.854 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.854 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.854 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.854 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.855 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.855 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.855 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.855 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.855 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.856 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.856 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.856 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.856 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.856 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.857 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.857 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.857 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.857 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.857 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.857 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.857 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.858 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.858 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.858 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.858 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.858 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.858 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.858 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.858 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.859 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.859 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.860 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.861 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-01-31T05:40:21.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-01-31T05:40:21.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.861 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.861 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.861 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.862 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.862 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-01-31T05:40:21.862 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-01-31T05:40:21.862 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-01-31T05:40:21.862 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.862 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-01-31T05:40:21.862 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.862 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.863 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.863 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-01-31T05:40:21.863 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.863 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.863 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.864 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.864 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.864 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.864 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.864 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.865 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.865 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.865 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.865 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.865 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.865 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.865 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.865 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.866 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.866 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.866 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.866 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.866 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.867 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.867 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.867 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.867 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.867 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.867 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.867 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.867 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.868 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.868 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.868 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.869 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.869 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.869 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.869 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.869 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.869 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.869 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.869 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.870 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.870 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.870 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.870 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.870 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.870 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.871 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.871 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.871 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.871 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.871 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.871 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.871 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.872 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-01-31T05:40:21.872 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.873 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.873 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.873 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.873 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.874 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.874 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.874 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.874 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.874 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.874 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.874 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.874 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.874 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.875 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.875 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.875 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.875 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.876 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.876 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.876 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.876 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.876 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.877 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.877 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.877 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.877 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.878 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.878 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.878 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.878 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.878 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.878 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.878 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.878 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.878 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.879 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.879 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.879 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.879 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.879 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.879 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.879 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.880 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.880 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.880 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.880 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.880 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.880 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.881 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.881 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.881 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.881 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-01-31T05:40:21.882 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.882 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.882 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.882 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.882 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.883 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.883 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.883 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.883 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.883 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.883 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.883 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.884 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.884 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.884 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.884 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.884 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.885 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.885 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.885 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.885 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.885 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.886 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.886 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.886 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.886 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.886 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.886 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.887 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.887 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.887 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.888 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.888 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.888 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.888 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.888 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.888 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.889 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.889 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.889 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.890 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.890 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.890 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.890 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.890 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.890 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.890 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.891 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:21.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:21.891 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.891 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.891 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.891 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:21.891 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.891 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.891 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.892 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.892 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.892 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.892 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.892 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-01-31T05:40:21.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-01-31T05:40:21.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:21.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.893 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-01-31T05:40:21.893 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-01-31T05:40:21.893 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.893 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-01-31T05:40:21.893 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-01-31T05:40:21.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.894 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.894 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.894 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-01-31T05:40:21.895 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.895 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.895 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.895 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-01-31T05:40:21.895 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-01-31T05:40:21.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.896 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.896 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-01-31T05:40:21.896 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.896 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-01-31T05:40:21.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-01-31T05:40:21.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.897 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.897 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.897 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.897 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.898 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.898 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.898 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.898 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.898 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.898 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.899 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.899 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.899 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.899 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.899 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.899 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.900 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.900 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.900 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.900 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.900 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.900 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.900 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.900 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.901 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.901 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.901 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.901 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.902 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.902 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.902 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.902 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.902 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.903 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.904 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.904 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.904 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.904 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.904 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.904 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.904 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.904 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.905 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.905 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.905 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.905 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.905 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.906 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.906 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.906 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.906 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.906 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.906 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.906 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.906 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.907 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.907 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-01-31T05:40:21.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-01-31T05:40:21.907 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.907 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.907 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.907 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-01-31T05:40:21.907 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.908 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.908 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-01-31T05:40:21.908 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.908 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.908 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.908 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-01-31T05:40:21.909 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-01-31T05:40:21.909 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:21.909 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.909 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.909 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-01-31T05:40:21.909 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.909 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.909 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-01-31T05:40:21.910 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.910 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.910 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-01-31T05:40:21.911 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.911 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.911 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.911 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-01-31T05:40:21.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-01-31T05:40:21.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.911 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.911 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.912 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.912 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.912 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.912 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-01-31T05:40:21.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-01-31T05:40:21.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.913 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.913 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.913 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.914 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-01-31T05:40:21.914 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-01-31T05:40:21.914 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.914 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.914 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.914 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.915 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.915 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.915 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.915 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.915 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.915 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.915 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.915 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.915 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.916 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.916 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.916 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.916 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.916 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.916 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.917 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.917 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.917 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.918 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.918 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.918 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.918 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.918 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.918 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.919 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.919 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.919 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.919 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.919 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.919 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.919 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.919 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.920 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.920 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.920 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.920 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.920 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.920 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.920 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.920 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.921 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.921 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.921 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.921 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.922 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.922 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.922 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.922 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.922 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-01-31T05:40:21.923 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-01-31T05:40:21.923 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.923 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.923 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.923 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.923 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.923 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.923 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.924 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.924 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.924 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.924 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-01-31T05:40:21.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-01-31T05:40:21.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.925 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.925 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-01-31T05:40:21.925 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.925 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.925 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.925 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.926 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.926 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-01-31T05:40:21.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-01-31T05:40:21.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:21.926 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.926 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-01-31T05:40:21.926 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.926 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.927 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.927 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.927 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.927 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.927 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-01-31T05:40:21.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-01-31T05:40:21.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.928 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.928 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.928 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-01-31T05:40:21.928 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.928 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.928 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.928 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-01-31T05:40:21.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.929 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.929 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-01-31T05:40:21.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-01-31T05:40:21.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.929 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.929 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.929 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-01-31T05:40:21.930 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.930 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.930 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.930 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-01-31T05:40:21.930 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.930 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.930 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.930 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-01-31T05:40:21.931 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-01-31T05:40:21.931 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.931 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.931 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.931 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-01-31T05:40:21.931 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.931 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.931 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.932 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.932 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.932 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-01-31T05:40:21.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-01-31T05:40:21.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.932 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.933 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-01-31T05:40:21.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.933 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-01-31T05:40:21.933 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-01-31T05:40:21.933 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.933 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.934 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.934 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.934 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.934 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-01-31T05:40:21.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-01-31T05:40:21.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.935 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.935 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-01-31T05:40:21.935 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-01-31T05:40:21.935 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-01-31T05:40:21.935 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.935 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.935 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.935 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.935 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.936 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.936 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-01-31T05:40:21.936 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.936 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.936 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.936 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.937 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.938 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.938 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.938 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.938 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.938 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.938 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.939 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.939 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.939 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.939 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.939 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.940 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.940 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.940 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.940 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.941 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.941 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.941 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.941 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.942 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.942 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.942 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.942 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.942 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.942 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.942 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.942 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.943 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.943 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.943 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.943 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.944 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.944 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.944 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.944 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.944 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.944 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-01-31T05:40:21.944 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-01-31T05:40:21.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:21.945 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.945 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-01-31T05:40:21.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.945 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.945 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.945 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.946 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-01-31T05:40:21.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-01-31T05:40:21.946 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.946 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.946 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.946 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.947 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.947 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.947 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.947 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.947 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-01-31T05:40:21.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-01-31T05:40:21.948 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.948 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.948 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.948 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.948 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.949 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.949 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.949 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.949 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.949 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-01-31T05:40:21.950 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.950 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.950 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.950 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-01-31T05:40:21.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-01-31T05:40:21.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.951 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:21.951 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.951 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.952 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-01-31T05:40:21.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-01-31T05:40:21.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.952 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.952 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.952 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.952 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.953 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.953 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.953 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.954 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.954 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.954 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.954 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.954 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.954 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.955 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.955 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.955 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.955 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.955 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.955 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.955 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.955 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.956 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.956 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.956 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.956 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.956 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.957 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.957 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.957 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.957 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.957 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.957 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.957 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.958 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.958 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.958 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.958 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.958 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.958 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.959 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.959 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.959 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.959 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.959 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.959 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.959 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.959 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.959 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.960 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.960 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.960 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.961 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.961 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.961 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.961 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-01-31T05:40:21.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-01-31T05:40:21.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.962 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.962 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.962 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.962 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.962 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.962 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.962 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.963 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.963 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-01-31T05:40:21.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-01-31T05:40:21.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.963 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.963 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.963 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.964 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.964 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.964 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.964 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.964 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.964 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.964 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-01-31T05:40:21.965 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.965 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.965 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.965 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.966 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.966 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.966 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.966 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.966 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.966 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.966 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.966 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.967 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.967 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.967 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.967 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.967 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.967 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.968 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.968 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.968 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.968 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.968 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.969 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.969 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.969 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.969 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.969 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.969 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.970 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.970 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.970 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.970 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.970 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.970 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-01-31T05:40:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.975 INFO:teuthology.orchestra.run.smithi045.stdout: "drive_group_interval": { 2024-01-31T05:40:21.975 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "drive_group_interval", 2024-01-31T05:40:21.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:21.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300.0", 2024-01-31T05:40:21.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.976 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-01-31T05:40:21.976 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.976 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.976 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.976 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.976 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.977 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.977 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.977 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.977 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.977 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.977 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.977 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.978 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.978 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.978 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.978 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.978 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.978 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.978 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.978 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.979 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.979 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.979 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.979 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.979 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.979 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.980 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.980 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.980 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.980 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.980 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.980 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.980 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.981 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.981 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.981 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.981 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.981 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.981 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.981 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.981 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.981 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.982 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.982 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.982 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.982 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.982 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.982 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.982 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.982 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.983 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.983 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.983 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.983 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.983 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.983 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.984 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.984 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.984 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.984 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.984 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.984 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.984 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.985 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-01-31T05:40:21.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-01-31T05:40:21.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.985 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.985 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-01-31T05:40:21.985 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.986 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.986 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-01-31T05:40:21.986 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.986 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.986 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.986 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.986 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:21.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:21.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-01-31T05:40:21.987 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:21.987 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:21.987 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:21.987 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:21.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:21.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.987 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.988 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.988 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.988 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.988 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.988 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.988 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.988 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.988 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.990 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:21.991 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:21.992 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.993 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-01-31T05:40:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-01-31T05:40:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.996 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-01-31T05:40:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-01-31T05:40:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-01-31T05:40:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:21.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-01-31T05:40:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-01-31T05:40:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:21.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-01-31T05:40:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-01-31T05:40:22.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-01-31T05:40:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-01-31T05:40:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-01-31T05:40:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.002 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.003 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-01-31T05:40:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-01-31T05:40:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.005 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-01-31T05:40:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-01-31T05:40:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-01-31T05:40:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-01-31T05:40:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.007 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.008 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.009 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.009 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-01-31T05:40:22.009 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-01-31T05:40:22.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:22.009 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.009 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.010 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-01-31T05:40:22.010 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-01-31T05:40:22.010 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.010 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.010 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.010 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.010 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.011 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-01-31T05:40:22.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-01-31T05:40:22.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.011 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.011 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.011 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.011 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.012 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.012 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.012 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.012 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-01-31T05:40:22.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-01-31T05:40:22.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.013 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.013 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.013 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.013 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.013 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.013 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.013 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.014 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.014 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-01-31T05:40:22.014 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-01-31T05:40:22.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.014 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.014 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.014 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.014 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.015 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.015 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.015 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.015 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.015 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.015 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-01-31T05:40:22.016 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:22.016 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:22.016 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:22.016 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-01-31T05:40:22.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-01-31T05:40:22.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.016 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.017 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.017 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-01-31T05:40:22.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.018 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-01-31T05:40:22.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-01-31T05:40:22.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.018 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.019 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.019 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-01-31T05:40:22.019 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.019 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:22.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:22.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.020 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.020 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.020 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.020 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.020 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.020 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.020 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.021 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.021 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.021 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.021 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.021 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.021 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.022 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:22.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:22.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.022 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.022 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.022 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.022 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.023 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.023 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.023 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.023 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.023 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.023 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:22.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:22.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.024 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.024 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.024 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:22.024 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.024 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.024 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.024 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.024 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.025 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.025 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.025 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.025 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.025 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.025 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.025 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.025 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.026 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.026 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:22.026 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:22.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.026 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.026 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.027 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.027 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.027 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.027 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.027 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.027 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.027 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-01-31T05:40:22.028 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:22.028 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:22.028 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:22.028 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:22.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:22.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.029 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.029 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.029 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.029 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.029 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.029 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.029 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.029 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.030 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.030 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.030 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.030 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.030 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.030 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.030 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.031 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:22.031 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:22.031 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.031 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.031 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.031 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.031 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.032 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.032 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.032 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.032 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.032 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.032 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:22.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:22.032 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.033 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:22.033 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.033 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.033 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.033 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.033 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.033 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.034 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.034 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.034 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.034 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.034 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.034 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.034 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.034 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.035 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:22.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:22.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.035 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.035 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.035 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.035 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.035 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.036 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.036 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.036 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.036 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.036 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-01-31T05:40:22.037 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:22.037 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:22.037 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:22.037 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:22.037 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:22.037 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.037 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.037 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.038 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.038 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.038 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.038 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.038 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.038 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.038 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.038 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.038 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.039 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.039 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.039 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.039 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.039 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:22.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:22.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.039 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.040 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.040 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.040 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.040 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.040 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.040 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.040 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.040 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.041 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:22.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:22.041 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.041 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.041 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.041 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:22.041 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.041 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.042 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.042 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.042 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.042 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.042 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.042 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.042 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.042 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.042 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.043 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.043 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.043 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.043 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:22.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:22.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.043 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.043 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.044 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.044 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.044 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.044 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.044 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.044 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.044 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.044 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.044 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.045 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.045 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-01-31T05:40:22.045 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:22.045 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:22.045 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:22.045 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-01-31T05:40:22.045 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-01-31T05:40:22.046 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.046 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.046 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.046 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-01-31T05:40:22.046 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.046 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.046 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.046 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.046 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.047 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.047 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.047 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:22.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:22.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:22.047 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.047 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.047 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-01-31T05:40:22.048 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.048 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.048 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.048 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.048 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.048 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.048 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.048 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:22.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:22.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.049 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.049 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.049 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.049 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.049 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.049 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.049 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.050 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.050 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.050 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.050 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.050 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.050 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.050 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.050 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.050 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.050 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.051 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:22.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:22.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.051 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.051 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.051 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.051 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.052 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.052 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.052 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.052 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.052 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.052 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.052 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:22.053 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:22.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.053 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.053 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.053 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:22.053 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.053 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.053 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.054 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.054 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.054 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.054 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.054 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.054 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.054 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.054 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.054 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.055 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.055 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:22.055 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:22.055 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.055 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.055 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.055 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.055 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.056 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.056 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.056 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.056 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.056 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.056 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.056 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.056 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-01-31T05:40:22.057 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:22.057 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:22.057 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:22.057 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-01-31T05:40:22.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-01-31T05:40:22.057 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.058 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.058 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.058 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:22.058 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.058 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.058 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.058 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-01-31T05:40:22.058 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.058 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.059 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.059 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-01-31T05:40:22.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-01-31T05:40:22.059 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.059 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.059 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.059 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:22.060 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.060 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.060 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.060 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-01-31T05:40:22.060 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.060 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.060 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.060 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-01-31T05:40:22.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-01-31T05:40:22.061 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.061 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.061 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.061 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-01-31T05:40:22.061 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.061 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.061 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.061 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-01-31T05:40:22.061 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.062 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.062 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.062 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-01-31T05:40:22.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-01-31T05:40:22.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.062 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.062 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.063 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.063 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-01-31T05:40:22.063 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.063 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.063 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.063 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-01-31T05:40:22.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-01-31T05:40:22.064 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.064 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.064 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.064 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.064 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.064 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.064 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.064 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-01-31T05:40:22.065 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.065 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-01-31T05:40:22.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-01-31T05:40:22.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.065 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.065 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.066 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.066 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.066 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.066 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.066 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.066 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.067 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-01-31T05:40:22.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-01-31T05:40:22.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.067 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.067 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.067 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.067 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.068 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.068 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.068 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.068 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.068 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-01-31T05:40:22.068 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-01-31T05:40:22.068 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.068 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.069 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.069 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-01-31T05:40:22.069 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.069 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.069 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.069 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.070 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-01-31T05:40:22.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-01-31T05:40:22.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.070 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.070 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.070 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.070 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.070 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.070 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:22.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-01-31T05:40:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-01-31T05:40:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-01-31T05:40:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-01-31T05:40:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-01-31T05:40:22.073 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-01-31T05:40:22.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-01-31T05:40:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-01-31T05:40:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-01-31T05:40:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.079 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.079 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.079 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.079 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.079 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.079 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.080 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:22.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.082 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.083 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.085 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-01-31T05:40:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-01-31T05:40:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.086 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-01-31T05:40:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-01-31T05:40:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.087 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-01-31T05:40:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-01-31T05:40:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-01-31T05:40:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-01-31T05:40:22.090 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.091 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.092 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.093 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.093 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.093 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.093 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.093 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:22.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:22.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.093 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.094 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.094 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.094 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.094 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.094 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.094 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.094 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.095 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:22.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:22.095 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.095 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.095 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:22.095 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.095 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.095 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.096 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.096 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.096 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.096 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.096 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.096 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.096 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.096 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.097 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.097 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.097 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:22.097 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:22.097 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.097 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.097 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.098 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.098 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.098 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.098 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.098 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.098 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.098 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.098 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.098 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.098 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.099 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-01-31T05:40:22.099 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:22.099 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:22.099 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:22.099 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:22.099 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:22.099 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.100 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.100 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.100 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.100 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.100 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.100 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.100 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.100 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.101 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.101 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.101 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.101 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.101 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.103 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.103 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.103 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.103 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.103 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:22.103 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:22.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.103 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.104 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.104 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:22.104 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.104 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.104 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.104 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.104 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.104 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.105 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.105 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.105 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.106 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.107 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.107 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.107 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:22.107 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:22.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.107 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.108 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.108 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.108 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.108 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.108 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.108 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.108 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.108 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.109 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.109 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-01-31T05:40:22.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-01-31T05:40:22.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:22.109 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.109 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.109 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-01-31T05:40:22.109 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.109 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.110 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.110 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-01-31T05:40:22.110 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.110 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.110 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.110 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-01-31T05:40:22.110 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-01-31T05:40:22.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.111 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.111 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.111 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.111 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.111 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-01-31T05:40:22.111 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.111 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.112 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.112 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-01-31T05:40:22.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-01-31T05:40:22.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:22.112 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.112 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.112 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-01-31T05:40:22.112 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.113 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-01-31T05:40:22.113 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.113 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.113 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.113 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.113 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-01-31T05:40:22.114 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-01-31T05:40:22.114 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-01-31T05:40:22.114 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-01-31T05:40:22.114 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-01-31T05:40:22.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-01-31T05:40:22.114 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-01-31T05:40:22.114 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.115 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-01-31T05:40:22.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.115 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.115 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.115 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.115 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.116 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-01-31T05:40:22.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-01-31T05:40:22.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.116 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.116 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.116 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.117 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.117 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.117 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.117 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.117 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.117 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-01-31T05:40:22.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-01-31T05:40:22.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-01-31T05:40:22.118 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.118 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.118 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-01-31T05:40:22.118 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.118 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.118 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.118 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.118 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.118 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.119 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.119 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-01-31T05:40:22.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-01-31T05:40:22.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.119 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.119 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.119 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.120 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.120 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.120 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.120 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.120 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.120 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.120 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.120 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.121 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.121 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.121 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.121 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.121 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.121 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-01-31T05:40:22.121 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-01-31T05:40:22.121 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.122 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.122 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.122 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.122 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.122 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.122 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.122 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.122 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.122 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.122 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.123 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-01-31T05:40:22.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-01-31T05:40:22.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.123 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-01-31T05:40:22.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.124 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.124 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-01-31T05:40:22.124 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-01-31T05:40:22.124 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-01-31T05:40:22.124 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-01-31T05:40:22.124 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-01-31T05:40:22.124 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-01-31T05:40:22.124 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-01-31T05:40:22.124 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.125 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.125 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-01-31T05:40:22.125 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-01-31T05:40:22.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-01-31T05:40:22.126 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.126 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-01-31T05:40:22.126 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-01-31T05:40:22.126 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.126 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.126 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.126 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.126 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.126 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.127 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-01-31T05:40:22.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-01-31T05:40:22.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.127 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.127 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.127 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-01-31T05:40:22.127 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.128 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.128 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.128 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.128 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.128 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.128 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-01-31T05:40:22.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-01-31T05:40:22.129 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-01-31T05:40:22.129 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.129 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-01-31T05:40:22.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.129 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.130 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-01-31T05:40:22.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-01-31T05:40:22.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-01-31T05:40:22.130 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-01-31T05:40:22.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-01-31T05:40:22.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-01-31T05:40:22.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-01-31T05:40:22.131 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-01-31T05:40:22.131 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-01-31T05:40:22.131 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-01-31T05:40:22.131 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-01-31T05:40:22.131 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-01-31T05:40:22.131 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-01-31T05:40:22.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.132 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.132 INFO:teuthology.orchestra.run.smithi045.stdout: "services": { 2024-01-31T05:40:22.132 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard": "https://172.21.15.45:8443/" 2024-01-31T05:40:22.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.133 INFO:teuthology.orchestra.run.smithi045.stdout: "always_on_modules": { 2024-01-31T05:40:22.133 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus": [ 2024-01-31T05:40:22.133 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-01-31T05:40:22.133 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-01-31T05:40:22.133 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-01-31T05:40:22.133 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-01-31T05:40:22.133 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-01-31T05:40:22.133 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-01-31T05:40:22.133 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-01-31T05:40:22.134 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-01-31T05:40:22.134 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-01-31T05:40:22.134 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-01-31T05:40:22.134 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.134 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific": [ 2024-01-31T05:40:22.134 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-01-31T05:40:22.134 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-01-31T05:40:22.134 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-01-31T05:40:22.134 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-01-31T05:40:22.135 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-01-31T05:40:22.135 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-01-31T05:40:22.135 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-01-31T05:40:22.135 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-01-31T05:40:22.135 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-01-31T05:40:22.135 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-01-31T05:40:22.135 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.135 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy": [ 2024-01-31T05:40:22.135 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-01-31T05:40:22.136 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-01-31T05:40:22.136 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-01-31T05:40:22.136 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-01-31T05:40:22.136 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-01-31T05:40:22.136 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-01-31T05:40:22.136 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-01-31T05:40:22.136 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-01-31T05:40:22.136 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-01-31T05:40:22.137 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-01-31T05:40:22.137 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.137 INFO:teuthology.orchestra.run.smithi045.stdout: "reef": [ 2024-01-31T05:40:22.137 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-01-31T05:40:22.137 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-01-31T05:40:22.137 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-01-31T05:40:22.137 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-01-31T05:40:22.137 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-01-31T05:40:22.137 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-01-31T05:40:22.137 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-01-31T05:40:22.138 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-01-31T05:40:22.138 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-01-31T05:40:22.138 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-01-31T05:40:22.138 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-01-31T05:40:22.138 INFO:teuthology.orchestra.run.smithi045.stdout: "squid": [ 2024-01-31T05:40:22.138 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-01-31T05:40:22.138 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-01-31T05:40:22.138 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-01-31T05:40:22.139 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-01-31T05:40:22.139 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-01-31T05:40:22.139 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-01-31T05:40:22.139 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-01-31T05:40:22.139 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-01-31T05:40:22.139 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-01-31T05:40:22.139 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-01-31T05:40:22.139 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-01-31T05:40:22.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.140 INFO:teuthology.orchestra.run.smithi045.stdout: "last_failure_osd_epoch": 89, 2024-01-31T05:40:22.140 INFO:teuthology.orchestra.run.smithi045.stdout: "active_clients": [ 2024-01-31T05:40:22.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.140 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-01-31T05:40:22.140 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-01-31T05:40:22.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.140 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-01-31T05:40:22.140 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-01-31T05:40:22.141 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1388771895 2024-01-31T05:40:22.141 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.141 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-01-31T05:40:22.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "libcephsqlite", 2024-01-31T05:40:22.141 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-01-31T05:40:22.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-01-31T05:40:22.141 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-01-31T05:40:22.142 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4235854266 2024-01-31T05:40:22.142 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.142 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-01-31T05:40:22.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.142 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-01-31T05:40:22.142 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-01-31T05:40:22.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-01-31T05:40:22.143 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-01-31T05:40:22.143 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3945301955 2024-01-31T05:40:22.143 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.143 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-01-31T05:40:22.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.143 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-01-31T05:40:22.143 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-01-31T05:40:22.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-01-31T05:40:22.144 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-01-31T05:40:22.144 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 961011290 2024-01-31T05:40:22.144 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.144 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-01-31T05:40:22.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-01-31T05:40:22.145 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-01-31T05:40:22.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.145 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-01-31T05:40:22.145 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-01-31T05:40:22.145 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2341086585 2024-01-31T05:40:22.145 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.145 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-01-31T05:40:22.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:22.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-01-31T05:40:22.146 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-01-31T05:40:22.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-01-31T05:40:22.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-01-31T05:40:22.146 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-01-31T05:40:22.146 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2447864626 2024-01-31T05:40:22.146 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.146 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-01-31T05:40:22.146 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:22.147 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-01-31T05:40:22.147 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-01-31T05:40:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.057+0000 7fd001fd0640 1 -- 172.21.15.45:0/2555775095 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fcfe80a1450 msgr2=0x7fcfe80a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.057+0000 7fd001fd0640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fcfe80a1450 0x7fcfe80a3910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006010 tx=0x7fcfec004e90 comp rx=0 tx=0).stop 2024-01-31T05:40:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.057+0000 7fd001fd0640 1 -- 172.21.15.45:0/2555775095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcffc104f30 msgr2=0x7fcffc198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.057+0000 7fd001fd0640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcffc104f30 0x7fcffc198690 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400d740 tx=0x7fcfe400dc00 comp rx=0 tx=0).stop 2024-01-31T05:40:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.057+0000 7fd001fd0640 1 -- 172.21.15.45:0/2555775095 shutdown_connections 2024-01-31T05:40:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.057+0000 7fd001fd0640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fcfe80a1450 0x7fcfe80a3910 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.057+0000 7fd001fd0640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcffc104f30 0x7fcffc198690 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.057+0000 7fd001fd0640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcffc198c20 0x7fcffc1a31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.057+0000 7fd001fd0640 1 --2- 172.21.15.45:0/2555775095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcffc0ffe60 0x7fcffc198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.057+0000 7fd001fd0640 1 -- 172.21.15.45:0/2555775095 >> 172.21.15.45:0/2555775095 conn(0x7fcffc076cf0 msgr2=0x7fcffc0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.057+0000 7fd001fd0640 1 -- 172.21.15.45:0/2555775095 shutdown_connections 2024-01-31T05:40:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:20.058+0000 7fd001fd0640 1 -- 172.21.15.45:0/2555775095 wait complete. 2024-01-31T05:40:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:19 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1048360977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:40:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:19 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1426239174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:40:22.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:19 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/4022256762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:40:22.155 DEBUG:tasks.check_counter:Getting stats from a 2024-01-31T05:40:22.156 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph daemon mds.a perf dump 2024-01-31T05:40:22.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:20 smithi045 ceph-mon[31585]: pgmap v562: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 244 op/s 2024-01-31T05:40:22.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:20 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2555775095' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-01-31T05:40:22.157 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:19 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1048360977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:40:22.157 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:19 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1426239174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:40:22.157 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:19 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/4022256762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:40:22.157 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:20 smithi186 ceph-mon[35659]: pgmap v562: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 244 op/s 2024-01-31T05:40:22.158 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:20 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2555775095' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-01-31T05:40:22.158 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:19 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1048360977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:40:22.158 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:19 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1426239174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:40:22.158 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:19 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/4022256762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:40:22.158 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:20 smithi156 ceph-mon[35720]: pgmap v562: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 244 op/s 2024-01-31T05:40:22.158 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:20 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2555775095' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-01-31T05:40:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:40:23.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:22 smithi045 ceph-mon[31585]: pgmap v563: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 244 op/s 2024-01-31T05:40:23.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:22 smithi156 ceph-mon[35720]: pgmap v563: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 244 op/s 2024-01-31T05:40:23.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:22 smithi186 ceph-mon[35659]: pgmap v563: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 244 op/s 2024-01-31T05:40:24.272 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-01-31T05:40:24.272 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-01-31T05:40:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2024-01-31T05:40:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2024-01-31T05:40:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3040, 2024-01-31T05:40:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2024-01-31T05:40:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-01-31T05:40:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-01-31T05:40:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001755795, 2024-01-31T05:40:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000049498, 2024-01-31T05:40:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000744856, 2024-01-31T05:40:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-31T05:40:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2024-01-31T05:40:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000847597, 2024-01-31T05:40:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000282532 2024-01-31T05:40:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-01-31T05:40:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-01-31T05:40:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-01-31T05:40:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 328, 2024-01-31T05:40:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 317, 2024-01-31T05:40:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1589728, 2024-01-31T05:40:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 190976, 2024-01-31T05:40:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-01-31T05:40:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-01-31T05:40:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.050233179, 2024-01-31T05:40:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.017457147, 2024-01-31T05:40:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.035164253, 2024-01-31T05:40:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.005458068, 2024-01-31T05:40:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 316, 2024-01-31T05:40:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.011284247, 2024-01-31T05:40:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000035709 2024-01-31T05:40:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1589728, 2024-01-31T05:40:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 190976 2024-01-31T05:40:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-01-31T05:40:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3, 2024-01-31T05:40:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 244, 2024-01-31T05:40:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 480, 2024-01-31T05:40:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 274592, 2024-01-31T05:40:24.279 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-01-31T05:40:24.279 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-01-31T05:40:24.279 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.016035362, 2024-01-31T05:40:24.279 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.010975256, 2024-01-31T05:40:24.279 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002328714, 2024-01-31T05:40:24.279 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-31T05:40:24.279 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 241, 2024-01-31T05:40:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.015025726, 2024-01-31T05:40:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000062347 2024-01-31T05:40:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 480, 2024-01-31T05:40:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 274592 2024-01-31T05:40:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-01-31T05:40:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-01-31T05:40:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-01-31T05:40:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-01-31T05:40:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-01-31T05:40:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-01-31T05:40:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-01-31T05:40:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-01-31T05:40:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-01-31T05:40:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-01-31T05:40:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-31T05:40:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-01-31T05:40:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-31T05:40:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-01-31T05:40:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-01-31T05:40:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-01-31T05:40:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-31T05:40:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-01-31T05:40:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-01-31T05:40:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-01-31T05:40:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-01-31T05:40:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-01-31T05:40:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-31T05:40:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-01-31T05:40:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-31T05:40:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-01-31T05:40:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-01-31T05:40:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-01-31T05:40:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-01-31T05:40:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-01-31T05:40:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-31T05:40:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-01-31T05:40:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-01-31T05:40:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-01-31T05:40:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-01-31T05:40:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-01-31T05:40:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-01-31T05:40:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-01-31T05:40:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-01-31T05:40:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-01-31T05:40:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148714, 2024-01-31T05:40:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 30, 2024-01-31T05:40:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-01-31T05:40:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-01-31T05:40:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-01-31T05:40:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-01-31T05:40:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-01-31T05:40:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-01-31T05:40:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-01-31T05:40:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-01-31T05:40:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-01-31T05:40:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-01-31T05:40:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-01-31T05:40:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-01-31T05:40:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-01-31T05:40:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-01-31T05:40:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-01-31T05:40:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-01-31T05:40:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-01-31T05:40:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-01-31T05:40:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-01-31T05:40:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-01-31T05:40:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-31T05:40:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-01-31T05:40:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-01-31T05:40:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-01-31T05:40:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 331, 2024-01-31T05:40:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1543451, 2024-01-31T05:40:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 331, 2024-01-31T05:40:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-01-31T05:40:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-01-31T05:40:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 331, 2024-01-31T05:40:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1543451, 2024-01-31T05:40:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-01-31T05:40:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:24.291 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-01-31T05:40:25.043 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-01-31T05:40:25.043 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-01-31T05:40:25.043 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-01-31T05:40:25.043 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-01-31T05:40:25.043 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-01-31T05:40:25.043 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-01-31T05:40:25.043 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-01-31T05:40:25.044 DEBUG:tasks.check_counter:Getting stats from d 2024-01-31T05:40:25.044 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph daemon mds.d perf dump 2024-01-31T05:40:25.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:24 smithi156 ceph-mon[35720]: pgmap v564: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 4.5 MiB/s wr, 176 op/s 2024-01-31T05:40:25.326 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:40:25.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:24 smithi186 ceph-mon[35659]: pgmap v564: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 4.5 MiB/s wr, 176 op/s 2024-01-31T05:40:25.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:24 smithi045 ceph-mon[31585]: pgmap v564: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 4.5 MiB/s wr, 176 op/s 2024-01-31T05:40:27.070 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:26 smithi045 ceph-mon[31585]: pgmap v565: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 4.1 MiB/s wr, 161 op/s 2024-01-31T05:40:27.086 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-01-31T05:40:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-01-31T05:40:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 334, 2024-01-31T05:40:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 323, 2024-01-31T05:40:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1591360, 2024-01-31T05:40:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 194720, 2024-01-31T05:40:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-01-31T05:40:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-01-31T05:40:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.052450292, 2024-01-31T05:40:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.016878221, 2024-01-31T05:40:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.037002822, 2024-01-31T05:40:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.005672307, 2024-01-31T05:40:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 322, 2024-01-31T05:40:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.011972495, 2024-01-31T05:40:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000037181 2024-01-31T05:40:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1591360, 2024-01-31T05:40:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 194720 2024-01-31T05:40:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-01-31T05:40:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 7, 2024-01-31T05:40:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 250, 2024-01-31T05:40:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3520, 2024-01-31T05:40:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 275744, 2024-01-31T05:40:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-01-31T05:40:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-01-31T05:40:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.015485189, 2024-01-31T05:40:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.010455872, 2024-01-31T05:40:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002477480, 2024-01-31T05:40:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-31T05:40:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 248, 2024-01-31T05:40:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.011101397, 2024-01-31T05:40:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000044763 2024-01-31T05:40:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3520, 2024-01-31T05:40:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 275744 2024-01-31T05:40:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-01-31T05:40:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-01-31T05:40:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-01-31T05:40:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-01-31T05:40:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-01-31T05:40:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-01-31T05:40:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-01-31T05:40:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.000420355, 2024-01-31T05:40:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-01-31T05:40:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000347995, 2024-01-31T05:40:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-31T05:40:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-31T05:40:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-01-31T05:40:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-01-31T05:40:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-01-31T05:40:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-01-31T05:40:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-01-31T05:40:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-01-31T05:40:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-01-31T05:40:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-01-31T05:40:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-01-31T05:40:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-01-31T05:40:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-01-31T05:40:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-31T05:40:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-01-31T05:40:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-31T05:40:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-01-31T05:40:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-01-31T05:40:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-01-31T05:40:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-31T05:40:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-01-31T05:40:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-01-31T05:40:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-01-31T05:40:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-01-31T05:40:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-01-31T05:40:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-31T05:40:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-01-31T05:40:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-31T05:40:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-01-31T05:40:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-01-31T05:40:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-01-31T05:40:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-01-31T05:40:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-01-31T05:40:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-31T05:40:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-01-31T05:40:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-01-31T05:40:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-01-31T05:40:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-01-31T05:40:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-01-31T05:40:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-01-31T05:40:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-01-31T05:40:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-01-31T05:40:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-01-31T05:40:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148714, 2024-01-31T05:40:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 30, 2024-01-31T05:40:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-01-31T05:40:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-01-31T05:40:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-01-31T05:40:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-01-31T05:40:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-01-31T05:40:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-01-31T05:40:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-01-31T05:40:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-01-31T05:40:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-01-31T05:40:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-01-31T05:40:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-01-31T05:40:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-01-31T05:40:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-01-31T05:40:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-01-31T05:40:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-01-31T05:40:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-01-31T05:40:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-01-31T05:40:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-01-31T05:40:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-01-31T05:40:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-01-31T05:40:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-31T05:40:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-01-31T05:40:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-01-31T05:40:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-01-31T05:40:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 337, 2024-01-31T05:40:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1544231, 2024-01-31T05:40:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 337, 2024-01-31T05:40:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-01-31T05:40:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-01-31T05:40:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 337, 2024-01-31T05:40:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1544231, 2024-01-31T05:40:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-01-31T05:40:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:27.105 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-01-31T05:40:27.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:26 smithi156 ceph-mon[35720]: pgmap v565: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 4.1 MiB/s wr, 161 op/s 2024-01-31T05:40:27.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:26 smithi186 ceph-mon[35659]: pgmap v565: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 4.1 MiB/s wr, 161 op/s 2024-01-31T05:40:27.910 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-01-31T05:40:27.910 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-01-31T05:40:27.910 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-01-31T05:40:27.911 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-01-31T05:40:27.911 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-01-31T05:40:27.911 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-01-31T05:40:27.911 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-01-31T05:40:27.911 DEBUG:tasks.check_counter:Getting stats from g 2024-01-31T05:40:27.911 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph daemon mds.g perf dump 2024-01-31T05:40:28.154 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:40:29.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:28 smithi156 ceph-mon[35720]: pgmap v566: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.1 MiB/s wr, 218 op/s 2024-01-31T05:40:29.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:28 smithi186 ceph-mon[35659]: pgmap v566: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.1 MiB/s wr, 218 op/s 2024-01-31T05:40:29.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:28 smithi045 ceph-mon[31585]: pgmap v566: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.1 MiB/s wr, 218 op/s 2024-01-31T05:40:29.926 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-01-31T05:40:29.926 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-01-31T05:40:29.926 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 335, 2024-01-31T05:40:29.926 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 322, 2024-01-31T05:40:29.927 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1593584, 2024-01-31T05:40:29.927 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 193296, 2024-01-31T05:40:29.927 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-01-31T05:40:29.927 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-01-31T05:40:29.927 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.048389428, 2024-01-31T05:40:29.927 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.015935659, 2024-01-31T05:40:29.927 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.033230613, 2024-01-31T05:40:29.927 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.005691159, 2024-01-31T05:40:29.928 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:29.928 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 322, 2024-01-31T05:40:29.928 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.011319003, 2024-01-31T05:40:29.928 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000035152 2024-01-31T05:40:29.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:29.928 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:29.928 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:29.928 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:29.929 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:29.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:29.929 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1593584, 2024-01-31T05:40:29.929 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 193296 2024-01-31T05:40:29.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:29.929 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-01-31T05:40:29.929 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3, 2024-01-31T05:40:29.929 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 245, 2024-01-31T05:40:29.930 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 480, 2024-01-31T05:40:29.930 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 274768, 2024-01-31T05:40:29.930 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-01-31T05:40:29.930 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-01-31T05:40:29.930 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.015875600, 2024-01-31T05:40:29.930 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.011194661, 2024-01-31T05:40:29.930 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002187795, 2024-01-31T05:40:29.930 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-31T05:40:29.930 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:29.931 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 242, 2024-01-31T05:40:29.931 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.009908131, 2024-01-31T05:40:29.931 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000040942 2024-01-31T05:40:29.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:29.931 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:29.931 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:29.931 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:29.931 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:29.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:29.932 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 480, 2024-01-31T05:40:29.932 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 274768 2024-01-31T05:40:29.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:29.932 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-01-31T05:40:29.932 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-01-31T05:40:29.932 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-01-31T05:40:29.933 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-01-31T05:40:29.933 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-01-31T05:40:29.933 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-01-31T05:40:29.933 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-01-31T05:40:29.933 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.000430310, 2024-01-31T05:40:29.933 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-01-31T05:40:29.933 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000358476, 2024-01-31T05:40:29.933 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-31T05:40:29.934 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:29.934 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:29.934 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:29.934 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:29.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:29.934 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:29.934 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:29.934 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:29.935 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:29.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:29.935 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-31T05:40:29.935 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-01-31T05:40:29.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:29.935 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-01-31T05:40:29.935 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-01-31T05:40:29.935 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-01-31T05:40:29.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:29.936 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-01-31T05:40:29.936 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-01-31T05:40:29.936 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-01-31T05:40:29.936 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-01-31T05:40:29.936 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-01-31T05:40:29.936 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-01-31T05:40:29.936 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-01-31T05:40:29.936 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-31T05:40:29.937 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-01-31T05:40:29.937 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-31T05:40:29.937 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-01-31T05:40:29.937 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-01-31T05:40:29.937 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-01-31T05:40:29.937 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-31T05:40:29.937 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-01-31T05:40:29.937 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-01-31T05:40:29.938 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-01-31T05:40:29.938 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-01-31T05:40:29.938 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-01-31T05:40:29.938 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-31T05:40:29.938 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-01-31T05:40:29.938 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-31T05:40:29.938 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-01-31T05:40:29.938 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-01-31T05:40:29.939 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-01-31T05:40:29.939 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-01-31T05:40:29.939 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-01-31T05:40:29.939 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-31T05:40:29.939 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-01-31T05:40:29.939 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-01-31T05:40:29.939 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-01-31T05:40:29.939 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-01-31T05:40:29.940 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-01-31T05:40:29.940 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-01-31T05:40:29.940 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-01-31T05:40:29.940 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-01-31T05:40:29.940 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-01-31T05:40:29.940 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148771, 2024-01-31T05:40:29.940 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 32, 2024-01-31T05:40:29.941 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-01-31T05:40:29.941 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-01-31T05:40:29.941 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-01-31T05:40:29.941 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-01-31T05:40:29.941 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-01-31T05:40:29.941 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-01-31T05:40:29.941 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-01-31T05:40:29.941 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-01-31T05:40:29.942 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-01-31T05:40:29.942 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-01-31T05:40:29.942 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-01-31T05:40:29.942 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-01-31T05:40:29.942 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-01-31T05:40:29.942 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-01-31T05:40:29.942 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-01-31T05:40:29.942 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-01-31T05:40:29.942 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-01-31T05:40:29.943 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-01-31T05:40:29.943 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-01-31T05:40:29.943 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-01-31T05:40:29.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:29.943 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-31T05:40:29.943 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-01-31T05:40:29.943 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-01-31T05:40:29.943 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-01-31T05:40:29.944 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 334, 2024-01-31T05:40:29.944 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1543841, 2024-01-31T05:40:29.944 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:29.944 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 334, 2024-01-31T05:40:29.944 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-01-31T05:40:29.944 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-01-31T05:40:29.944 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 334, 2024-01-31T05:40:29.944 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1543841, 2024-01-31T05:40:29.945 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-01-31T05:40:29.945 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:29.945 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:29.945 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:29.945 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:29.945 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:29.945 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-01-31T05:40:30.591 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-01-31T05:40:30.591 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-01-31T05:40:30.591 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-01-31T05:40:30.591 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-01-31T05:40:30.591 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-01-31T05:40:30.592 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-01-31T05:40:30.592 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-01-31T05:40:30.592 DEBUG:tasks.check_counter:Getting stats from j 2024-01-31T05:40:30.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph daemon mds.j perf dump 2024-01-31T05:40:30.835 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.a/config 2024-01-31T05:40:31.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:30 smithi156 ceph-mon[35720]: pgmap v567: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.6 MiB/s wr, 128 op/s 2024-01-31T05:40:31.315 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:30 smithi045 ceph-mon[31585]: pgmap v567: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.6 MiB/s wr, 128 op/s 2024-01-31T05:40:31.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:30 smithi186 ceph-mon[35659]: pgmap v567: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.6 MiB/s wr, 128 op/s 2024-01-31T05:40:32.556 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-01-31T05:40:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-01-31T05:40:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2024-01-31T05:40:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2024-01-31T05:40:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3040, 2024-01-31T05:40:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2024-01-31T05:40:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-01-31T05:40:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-01-31T05:40:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001659242, 2024-01-31T05:40:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000043625, 2024-01-31T05:40:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000703550, 2024-01-31T05:40:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-31T05:40:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2024-01-31T05:40:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000823634, 2024-01-31T05:40:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000274544 2024-01-31T05:40:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-01-31T05:40:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-01-31T05:40:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-01-31T05:40:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3, 2024-01-31T05:40:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 250, 2024-01-31T05:40:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 480, 2024-01-31T05:40:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 275648, 2024-01-31T05:40:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-01-31T05:40:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-01-31T05:40:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.015732904, 2024-01-31T05:40:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.011256045, 2024-01-31T05:40:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001872730, 2024-01-31T05:40:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-31T05:40:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 247, 2024-01-31T05:40:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.012218640, 2024-01-31T05:40:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000049468 2024-01-31T05:40:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 480, 2024-01-31T05:40:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 275648 2024-01-31T05:40:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-01-31T05:40:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 336, 2024-01-31T05:40:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 325, 2024-01-31T05:40:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1591904, 2024-01-31T05:40:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 195968, 2024-01-31T05:40:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-01-31T05:40:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-01-31T05:40:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.049548577, 2024-01-31T05:40:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.017508355, 2024-01-31T05:40:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.034593778, 2024-01-31T05:40:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.005732906, 2024-01-31T05:40:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 324, 2024-01-31T05:40:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.011527878, 2024-01-31T05:40:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000035579 2024-01-31T05:40:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1591904, 2024-01-31T05:40:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 195968 2024-01-31T05:40:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-01-31T05:40:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-01-31T05:40:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-01-31T05:40:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-01-31T05:40:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-01-31T05:40:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-01-31T05:40:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-01-31T05:40:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-01-31T05:40:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-01-31T05:40:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-01-31T05:40:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-31T05:40:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-01-31T05:40:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-31T05:40:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-01-31T05:40:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-01-31T05:40:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-01-31T05:40:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-31T05:40:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-01-31T05:40:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-01-31T05:40:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-01-31T05:40:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-01-31T05:40:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-01-31T05:40:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-31T05:40:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-01-31T05:40:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-31T05:40:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-01-31T05:40:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-01-31T05:40:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-01-31T05:40:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-01-31T05:40:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-01-31T05:40:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-31T05:40:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-01-31T05:40:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-01-31T05:40:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-01-31T05:40:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-01-31T05:40:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-01-31T05:40:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-01-31T05:40:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-01-31T05:40:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-01-31T05:40:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-01-31T05:40:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148704, 2024-01-31T05:40:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 29, 2024-01-31T05:40:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-01-31T05:40:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-01-31T05:40:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-01-31T05:40:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-01-31T05:40:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-01-31T05:40:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-01-31T05:40:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-01-31T05:40:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-01-31T05:40:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-01-31T05:40:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-01-31T05:40:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-01-31T05:40:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-01-31T05:40:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-01-31T05:40:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-01-31T05:40:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-01-31T05:40:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-01-31T05:40:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-01-31T05:40:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-01-31T05:40:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-01-31T05:40:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-01-31T05:40:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-01-31T05:40:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-31T05:40:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-01-31T05:40:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-01-31T05:40:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-01-31T05:40:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 339, 2024-01-31T05:40:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1544491, 2024-01-31T05:40:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 339, 2024-01-31T05:40:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-01-31T05:40:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-01-31T05:40:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 339, 2024-01-31T05:40:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1544491, 2024-01-31T05:40:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-01-31T05:40:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-01-31T05:40:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-01-31T05:40:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-01-31T05:40:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-01-31T05:40:32.575 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-01-31T05:40:33.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:32 smithi156 ceph-mon[35720]: pgmap v568: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.6 MiB/s wr, 128 op/s 2024-01-31T05:40:33.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:32 smithi186 ceph-mon[35659]: pgmap v568: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.6 MiB/s wr, 128 op/s 2024-01-31T05:40:33.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:32 smithi045 ceph-mon[31585]: pgmap v568: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.6 MiB/s wr, 128 op/s 2024-01-31T05:40:33.456 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-01-31T05:40:33.456 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-01-31T05:40:33.456 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-01-31T05:40:33.456 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-01-31T05:40:33.457 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-01-31T05:40:33.457 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-01-31T05:40:33.457 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-01-31T05:40:33.457 DEBUG:tasks.check_counter:Getting stats from b 2024-01-31T05:40:33.458 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph daemon mds.b perf dump 2024-01-31T05:40:33.697 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.b/config 2024-01-31T05:40:35.107 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:34 smithi156 ceph-mon[35720]: pgmap v569: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 4.6 MiB/s wr, 178 op/s 2024-01-31T05:40:35.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:34 smithi186 ceph-mon[35659]: pgmap v569: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 4.6 MiB/s wr, 178 op/s 2024-01-31T05:40:35.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:34 smithi045 ceph-mon[31585]: pgmap v569: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 4.6 MiB/s wr, 178 op/s 2024-01-31T05:40:35.492 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-01-31T05:40:35.492 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-01-31T05:40:35.493 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 494406, 2024-01-31T05:40:35.493 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 251140, 2024-01-31T05:40:35.493 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 219205257, 2024-01-31T05:40:35.493 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 54510423, 2024-01-31T05:40:35.493 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 13, 2024-01-31T05:40:35.493 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2024-01-31T05:40:35.493 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 20.720308062, 2024-01-31T05:40:35.494 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 8.275246921, 2024-01-31T05:40:35.494 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 17.291511656, 2024-01-31T05:40:35.494 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.151251943, 2024-01-31T05:40:35.494 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:35.494 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 251127, 2024-01-31T05:40:35.494 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3.803946517, 2024-01-31T05:40:35.494 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000015147 2024-01-31T05:40:35.494 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.495 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:35.495 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 738085, 2024-01-31T05:40:35.495 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.081289626, 2024-01-31T05:40:35.495 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000110 2024-01-31T05:40:35.495 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.495 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 563280, 2024-01-31T05:40:35.495 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 182144 2024-01-31T05:40:35.495 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.496 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-01-31T05:40:35.496 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 80481, 2024-01-31T05:40:35.496 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 96259, 2024-01-31T05:40:35.496 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 26537348, 2024-01-31T05:40:35.496 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 41131626, 2024-01-31T05:40:35.496 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 173, 2024-01-31T05:40:35.496 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2024-01-31T05:40:35.497 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 5.731929852, 2024-01-31T05:40:35.497 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 2.359806180, 2024-01-31T05:40:35.497 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 4.370178810, 2024-01-31T05:40:35.497 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.141242402, 2024-01-31T05:40:35.497 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:35.497 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 96257, 2024-01-31T05:40:35.497 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2.276048680, 2024-01-31T05:40:35.497 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000023645 2024-01-31T05:40:35.498 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.498 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:35.498 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 159143, 2024-01-31T05:40:35.498 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.020786623, 2024-01-31T05:40:35.498 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000130 2024-01-31T05:40:35.498 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.498 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 25575456, 2024-01-31T05:40:35.498 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 32773840 2024-01-31T05:40:35.498 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.499 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-01-31T05:40:35.499 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 50898, 2024-01-31T05:40:35.499 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 31573, 2024-01-31T05:40:35.499 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 15578033, 2024-01-31T05:40:35.499 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 12114924, 2024-01-31T05:40:35.499 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 8, 2024-01-31T05:40:35.499 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2024-01-31T05:40:35.500 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 2.369044033, 2024-01-31T05:40:35.500 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.895323168, 2024-01-31T05:40:35.500 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 1.866232332, 2024-01-31T05:40:35.500 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.091778721, 2024-01-31T05:40:35.500 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:35.500 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 31572, 2024-01-31T05:40:35.500 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.915377993, 2024-01-31T05:40:35.500 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000028993 2024-01-31T05:40:35.501 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.501 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:35.501 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 76972, 2024-01-31T05:40:35.501 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.009559550, 2024-01-31T05:40:35.501 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000124 2024-01-31T05:40:35.501 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.501 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1701296, 2024-01-31T05:40:35.501 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 682976 2024-01-31T05:40:35.501 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.502 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-01-31T05:40:35.502 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-01-31T05:40:35.502 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-01-31T05:40:35.502 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.502 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-01-31T05:40:35.502 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-01-31T05:40:35.502 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-01-31T05:40:35.502 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7743, 2024-01-31T05:40:35.502 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 4.263310598, 2024-01-31T05:40:35.503 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000550601 2024-01-31T05:40:35.503 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:35.503 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.503 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-01-31T05:40:35.503 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-01-31T05:40:35.503 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-01-31T05:40:35.503 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3065, 2024-01-31T05:40:35.503 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.188143488, 2024-01-31T05:40:35.504 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000061384 2024-01-31T05:40:35.504 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:35.504 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.504 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-01-31T05:40:35.504 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 63071, 2024-01-31T05:40:35.504 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 9850, 2024-01-31T05:40:35.504 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-01-31T05:40:35.504 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 9850, 2024-01-31T05:40:35.505 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 46.984949098, 2024-01-31T05:40:35.505 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.004770045 2024-01-31T05:40:35.505 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.505 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-01-31T05:40:35.505 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 53221, 2024-01-31T05:40:35.505 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-01-31T05:40:35.505 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-01-31T05:40:35.505 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 718, 2024-01-31T05:40:35.505 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2024-01-31T05:40:35.506 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2024-01-31T05:40:35.506 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 256, 2024-01-31T05:40:35.506 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 87, 2024-01-31T05:40:35.506 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 155, 2024-01-31T05:40:35.506 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 14, 2024-01-31T05:40:35.506 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 124, 2024-01-31T05:40:35.506 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 214, 2024-01-31T05:40:35.506 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 26, 2024-01-31T05:40:35.506 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 26, 2024-01-31T05:40:35.507 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 9, 2024-01-31T05:40:35.507 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 74577, 2024-01-31T05:40:35.507 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 12003, 2024-01-31T05:40:35.507 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 53221, 2024-01-31T05:40:35.507 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 108, 2024-01-31T05:40:35.507 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-01-31T05:40:35.507 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-01-31T05:40:35.507 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 475, 2024-01-31T05:40:35.507 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-01-31T05:40:35.508 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-01-31T05:40:35.508 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 10, 2024-01-31T05:40:35.508 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 152, 2024-01-31T05:40:35.508 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 3, 2024-01-31T05:40:35.508 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 18, 2024-01-31T05:40:35.508 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-01-31T05:40:35.508 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2024-01-31T05:40:35.509 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-01-31T05:40:35.509 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 673, 2024-01-31T05:40:35.509 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 211539274, 2024-01-31T05:40:35.509 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 6, 2024-01-31T05:40:35.509 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2024-01-31T05:40:35.509 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-01-31T05:40:35.509 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-01-31T05:40:35.509 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-01-31T05:40:35.510 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2024-01-31T05:40:35.510 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2024-01-31T05:40:35.510 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2024-01-31T05:40:35.510 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2024-01-31T05:40:35.510 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 4, 2024-01-31T05:40:35.510 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 9068, 2024-01-31T05:40:35.510 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 14978, 2024-01-31T05:40:35.510 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-01-31T05:40:35.510 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 187, 2024-01-31T05:40:35.511 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-31T05:40:35.511 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 11502, 2024-01-31T05:40:35.511 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 1411, 2024-01-31T05:40:35.511 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 637, 2024-01-31T05:40:35.511 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 8798 2024-01-31T05:40:35.511 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.511 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-01-31T05:40:35.511 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 161, 2024-01-31T05:40:35.511 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 343, 2024-01-31T05:40:35.512 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 216, 2024-01-31T05:40:35.512 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 275, 2024-01-31T05:40:35.512 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 197, 2024-01-31T05:40:35.512 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 5, 2024-01-31T05:40:35.512 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-01-31T05:40:35.512 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-01-31T05:40:35.512 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 1657, 2024-01-31T05:40:35.512 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 1652, 2024-01-31T05:40:35.513 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-01-31T05:40:35.513 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-01-31T05:40:35.513 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-01-31T05:40:35.513 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-01-31T05:40:35.513 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-01-31T05:40:35.513 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-01-31T05:40:35.513 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-01-31T05:40:35.514 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2024-01-31T05:40:35.514 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 10, 2024-01-31T05:40:35.514 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-01-31T05:40:35.514 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2024-01-31T05:40:35.514 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-01-31T05:40:35.514 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-01-31T05:40:35.514 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.514 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-01-31T05:40:35.515 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-01-31T05:40:35.515 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 9568, 2024-01-31T05:40:35.515 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 9455, 2024-01-31T05:40:35.515 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 9455, 2024-01-31T05:40:35.515 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 113, 2024-01-31T05:40:35.515 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-01-31T05:40:35.515 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 212, 2024-01-31T05:40:35.515 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 86, 2024-01-31T05:40:35.516 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 85, 2024-01-31T05:40:35.516 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 85, 2024-01-31T05:40:35.516 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 1, 2024-01-31T05:40:35.516 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 1, 2024-01-31T05:40:35.516 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-01-31T05:40:35.516 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 1, 2024-01-31T05:40:35.516 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 23690643, 2024-01-31T05:40:35.516 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 23957268, 2024-01-31T05:40:35.517 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2024-01-31T05:40:35.517 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-01-31T05:40:35.517 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3194, 2024-01-31T05:40:35.517 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 9.702052032, 2024-01-31T05:40:35.517 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003037586 2024-01-31T05:40:35.517 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.517 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2024-01-31T05:40:35.518 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.518 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-01-31T05:40:35.518 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 191, 2024-01-31T05:40:35.518 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 1947, 2024-01-31T05:40:35.518 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 1756, 2024-01-31T05:40:35.518 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 210, 2024-01-31T05:40:35.518 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 351, 2024-01-31T05:40:35.518 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 141, 2024-01-31T05:40:35.519 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 259, 2024-01-31T05:40:35.519 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 2900, 2024-01-31T05:40:35.519 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 2641, 2024-01-31T05:40:35.519 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 22, 2024-01-31T05:40:35.519 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 1840, 2024-01-31T05:40:35.519 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 1818, 2024-01-31T05:40:35.519 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 68472, 2024-01-31T05:40:35.520 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207228 2024-01-31T05:40:35.520 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.520 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-01-31T05:40:35.520 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 70327, 2024-01-31T05:40:35.520 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 42, 2024-01-31T05:40:35.520 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 63071, 2024-01-31T05:40:35.520 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 407, 2024-01-31T05:40:35.521 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 1827, 2024-01-31T05:40:35.521 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-01-31T05:40:35.521 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1741, 2024-01-31T05:40:35.521 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 11.418901701, 2024-01-31T05:40:35.521 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.006558817 2024-01-31T05:40:35.521 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.521 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-01-31T05:40:35.521 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 216, 2024-01-31T05:40:35.522 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.115722369, 2024-01-31T05:40:35.522 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000535751 2024-01-31T05:40:35.522 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.522 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-01-31T05:40:35.522 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.522 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.522 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.522 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.523 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-01-31T05:40:35.523 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.523 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.523 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.523 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.523 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-01-31T05:40:35.523 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2701, 2024-01-31T05:40:35.524 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 14.161709883, 2024-01-31T05:40:35.524 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.005243135 2024-01-31T05:40:35.524 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.524 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-01-31T05:40:35.524 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.524 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.524 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.525 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.525 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-01-31T05:40:35.525 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.525 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.525 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.525 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.525 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-01-31T05:40:35.525 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.525 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.526 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.526 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.526 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-01-31T05:40:35.526 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.526 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.526 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.526 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.526 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-01-31T05:40:35.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.527 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.527 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.527 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-01-31T05:40:35.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 18, 2024-01-31T05:40:35.527 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.002918678, 2024-01-31T05:40:35.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000162148 2024-01-31T05:40:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-01-31T05:40:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 20, 2024-01-31T05:40:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.023477853, 2024-01-31T05:40:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001173892 2024-01-31T05:40:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-01-31T05:40:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.529 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.529 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.529 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.529 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-01-31T05:40:35.529 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 18, 2024-01-31T05:40:35.529 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 7.690554533, 2024-01-31T05:40:35.529 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.427253029 2024-01-31T05:40:35.529 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.529 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-01-31T05:40:35.530 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.530 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.530 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.530 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.530 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-01-31T05:40:35.530 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2682, 2024-01-31T05:40:35.530 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2.836536363, 2024-01-31T05:40:35.530 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001057619 2024-01-31T05:40:35.530 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.530 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-01-31T05:40:35.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 531, 2024-01-31T05:40:35.531 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 4.430554859, 2024-01-31T05:40:35.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.008343794 2024-01-31T05:40:35.531 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.531 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-01-31T05:40:35.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.531 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.531 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.532 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-01-31T05:40:35.532 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.532 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.532 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.532 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.532 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-01-31T05:40:35.532 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-01-31T05:40:35.533 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.034243153, 2024-01-31T05:40:35.533 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.034243153 2024-01-31T05:40:35.533 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.533 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-01-31T05:40:35.533 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 12, 2024-01-31T05:40:35.533 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1.982769051, 2024-01-31T05:40:35.533 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.165230754 2024-01-31T05:40:35.533 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.533 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-01-31T05:40:35.534 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.534 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.534 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.534 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.534 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-01-31T05:40:35.534 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-01-31T05:40:35.535 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000975748, 2024-01-31T05:40:35.535 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000975748 2024-01-31T05:40:35.535 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.535 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-01-31T05:40:35.535 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.535 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.536 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.536 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.536 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-01-31T05:40:35.536 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 244, 2024-01-31T05:40:35.536 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.139454387, 2024-01-31T05:40:35.536 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000571534 2024-01-31T05:40:35.536 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.536 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-01-31T05:40:35.536 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.537 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.537 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.537 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.537 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-01-31T05:40:35.537 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 8, 2024-01-31T05:40:35.537 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.022143285, 2024-01-31T05:40:35.537 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002767910 2024-01-31T05:40:35.537 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.538 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-01-31T05:40:35.538 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.538 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.538 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.538 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.538 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-01-31T05:40:35.538 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1657, 2024-01-31T05:40:35.538 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 4.124987235, 2024-01-31T05:40:35.539 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002489431 2024-01-31T05:40:35.539 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.539 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-01-31T05:40:35.539 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-01-31T05:40:35.539 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-01-31T05:40:35.539 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.539 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.539 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.539 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:35.540 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.540 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-01-31T05:40:35.540 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2024-01-31T05:40:35.540 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 6, 2024-01-31T05:40:35.540 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 4, 2024-01-31T05:40:35.540 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2024-01-31T05:40:35.541 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-01-31T05:40:35.541 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 2303, 2024-01-31T05:40:35.541 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 1151, 2024-01-31T05:40:35.541 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 1068, 2024-01-31T05:40:35.541 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-01-31T05:40:35.541 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.541 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-01-31T05:40:35.542 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 360, 2024-01-31T05:40:35.542 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 360, 2024-01-31T05:40:35.542 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-01-31T05:40:35.542 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-01-31T05:40:35.542 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-01-31T05:40:35.542 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-01-31T05:40:35.542 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-31T05:40:35.542 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-01-31T05:40:35.542 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-31T05:40:35.543 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-01-31T05:40:35.543 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-01-31T05:40:35.543 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-01-31T05:40:35.543 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-31T05:40:35.543 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-01-31T05:40:35.543 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-01-31T05:40:35.543 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-01-31T05:40:35.543 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-01-31T05:40:35.544 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-01-31T05:40:35.544 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-31T05:40:35.544 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-01-31T05:40:35.544 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-31T05:40:35.544 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-01-31T05:40:35.544 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-01-31T05:40:35.544 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-01-31T05:40:35.544 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-01-31T05:40:35.545 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-01-31T05:40:35.545 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-31T05:40:35.545 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-01-31T05:40:35.545 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-01-31T05:40:35.545 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-01-31T05:40:35.545 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-01-31T05:40:35.546 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-01-31T05:40:35.546 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-01-31T05:40:35.546 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-01-31T05:40:35.546 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-01-31T05:40:35.546 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-01-31T05:40:35.546 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 381945, 2024-01-31T05:40:35.546 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 291, 2024-01-31T05:40:35.546 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 616, 2024-01-31T05:40:35.546 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 7, 2024-01-31T05:40:35.547 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-01-31T05:40:35.547 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-01-31T05:40:35.547 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-01-31T05:40:35.547 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-01-31T05:40:35.547 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-01-31T05:40:35.547 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-01-31T05:40:35.547 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8160, 2024-01-31T05:40:35.547 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 186, 2024-01-31T05:40:35.547 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-01-31T05:40:35.548 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-01-31T05:40:35.548 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-01-31T05:40:35.548 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-01-31T05:40:35.548 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 991092, 2024-01-31T05:40:35.548 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 8631, 2024-01-31T05:40:35.548 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-01-31T05:40:35.548 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-01-31T05:40:35.548 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-01-31T05:40:35.548 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-01-31T05:40:35.549 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.549 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-01-31T05:40:35.549 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-01-31T05:40:35.549 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-01-31T05:40:35.549 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 8156, 2024-01-31T05:40:35.549 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 23403363, 2024-01-31T05:40:35.549 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 37, 2024-01-31T05:40:35.550 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 8156, 2024-01-31T05:40:35.550 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-01-31T05:40:35.550 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 8156, 2024-01-31T05:40:35.550 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 59.692069807, 2024-01-31T05:40:35.550 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.007318792 2024-01-31T05:40:35.550 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.550 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-01-31T05:40:35.550 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-01-31T05:40:35.550 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 8156, 2024-01-31T05:40:35.551 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 11998 2024-01-31T05:40:35.551 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.551 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 8156, 2024-01-31T05:40:35.551 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 190, 2024-01-31T05:40:35.551 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 7966, 2024-01-31T05:40:35.551 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-01-31T05:40:35.551 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-01-31T05:40:35.551 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 683, 2024-01-31T05:40:35.551 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 1104, 2024-01-31T05:40:35.552 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 190, 2024-01-31T05:40:35.552 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 3384, 2024-01-31T05:40:35.552 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 398, 2024-01-31T05:40:35.552 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-01-31T05:40:35.552 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-01-31T05:40:35.552 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-01-31T05:40:35.552 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2024-01-31T05:40:35.552 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 2172, 2024-01-31T05:40:35.552 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-01-31T05:40:35.553 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-01-31T05:40:35.553 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-01-31T05:40:35.553 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 0, 2024-01-31T05:40:35.553 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 2208, 2024-01-31T05:40:35.553 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-01-31T05:40:35.553 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-01-31T05:40:35.553 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-01-31T05:40:35.553 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-01-31T05:40:35.554 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-01-31T05:40:35.554 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-01-31T05:40:35.554 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-01-31T05:40:35.554 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-01-31T05:40:35.554 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-01-31T05:40:35.554 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-01-31T05:40:35.554 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-01-31T05:40:35.554 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-01-31T05:40:35.554 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-01-31T05:40:35.555 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-01-31T05:40:35.555 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-01-31T05:40:35.555 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-01-31T05:40:35.555 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-01-31T05:40:35.555 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-01-31T05:40:35.555 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-01-31T05:40:35.555 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-01-31T05:40:35.555 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-01-31T05:40:35.556 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-01-31T05:40:35.556 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-01-31T05:40:35.556 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-01-31T05:40:35.556 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-01-31T05:40:35.556 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-01-31T05:40:35.556 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 119, 2024-01-31T05:40:35.556 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-01-31T05:40:35.556 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 34, 2024-01-31T05:40:35.557 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2024-01-31T05:40:35.557 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2024-01-31T05:40:35.557 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-01-31T05:40:35.557 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-01-31T05:40:35.557 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 1662, 2024-01-31T05:40:35.557 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 0, 2024-01-31T05:40:35.557 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 197 2024-01-31T05:40:35.557 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.557 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-01-31T05:40:35.558 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2024-01-31T05:40:35.558 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 3, 2024-01-31T05:40:35.558 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 3, 2024-01-31T05:40:35.558 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-01-31T05:40:35.558 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.558 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-01-31T05:40:35.558 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-01-31T05:40:35.558 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.558 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-01-31T05:40:35.558 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-01-31T05:40:35.559 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 10, 2024-01-31T05:40:35.559 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-01-31T05:40:35.559 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 10, 2024-01-31T05:40:35.559 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 1652, 2024-01-31T05:40:35.559 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 1652, 2024-01-31T05:40:35.559 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-01-31T05:40:35.559 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.559 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-31T05:40:35.560 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-01-31T05:40:35.560 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-01-31T05:40:35.560 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-01-31T05:40:35.560 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 625781, 2024-01-31T05:40:35.560 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 201298690, 2024-01-31T05:40:35.560 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:35.560 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 625781, 2024-01-31T05:40:35.560 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-01-31T05:40:35.560 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-01-31T05:40:35.561 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 625781, 2024-01-31T05:40:35.561 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 201298690, 2024-01-31T05:40:35.561 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-01-31T05:40:35.561 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.561 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.561 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.561 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:35.561 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.562 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-01-31T05:40:35.562 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-01-31T05:40:35.562 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-01-31T05:40:35.562 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-01-31T05:40:35.563 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-01-31T05:40:35.563 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-01-31T05:40:35.563 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:35.563 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-01-31T05:40:35.563 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 8156, 2024-01-31T05:40:35.563 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 23570295, 2024-01-31T05:40:35.563 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 8156, 2024-01-31T05:40:35.563 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 23570295, 2024-01-31T05:40:35.564 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-01-31T05:40:35.564 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.564 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.564 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.564 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:35.564 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.564 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-01-31T05:40:35.565 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-01-31T05:40:35.565 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-01-31T05:40:35.565 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-01-31T05:40:35.565 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-01-31T05:40:35.565 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-01-31T05:40:35.565 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:35.565 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-01-31T05:40:35.565 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 8156, 2024-01-31T05:40:35.565 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 8156, 2024-01-31T05:40:35.566 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 8156, 2024-01-31T05:40:35.566 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 8156, 2024-01-31T05:40:35.566 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-01-31T05:40:35.566 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.566 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.566 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.566 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:35.567 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.567 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-01-31T05:40:35.567 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-01-31T05:40:35.567 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-01-31T05:40:35.567 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-01-31T05:40:35.567 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 1652, 2024-01-31T05:40:35.567 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 166932, 2024-01-31T05:40:35.567 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:35.567 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 1652, 2024-01-31T05:40:35.568 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-01-31T05:40:35.568 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-01-31T05:40:35.568 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 190, 2024-01-31T05:40:35.568 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 166932, 2024-01-31T05:40:35.568 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-01-31T05:40:35.568 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.568 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.568 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.568 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:35.569 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:35.569 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x558844cf46e0": { 2024-01-31T05:40:35.569 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 2396, 2024-01-31T05:40:35.569 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-01-31T05:40:35.569 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-01-31T05:40:35.569 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 9568, 2024-01-31T05:40:35.569 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 19762964, 2024-01-31T05:40:35.569 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:35.569 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 9568, 2024-01-31T05:40:35.570 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-01-31T05:40:35.570 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-01-31T05:40:35.570 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 3194, 2024-01-31T05:40:35.570 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 19760568, 2024-01-31T05:40:35.570 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-01-31T05:40:35.570 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:35.570 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:35.570 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:35.570 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:35.571 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:35.571 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-01-31T05:40:36.307 INFO:tasks.check_counter:Daemon mds.b mds.exported=10 2024-01-31T05:40:36.307 INFO:tasks.check_counter:Daemon mds.b mds.imported=3 2024-01-31T05:40:36.307 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=161 2024-01-31T05:40:36.308 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=343 2024-01-31T05:40:36.308 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-01-31T05:40:36.308 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=18 2024-01-31T05:40:36.308 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=12 2024-01-31T05:40:36.308 DEBUG:tasks.check_counter:Getting stats from e 2024-01-31T05:40:36.308 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph daemon mds.e perf dump 2024-01-31T05:40:36.548 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.b/config 2024-01-31T05:40:37.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:36 smithi156 ceph-mon[35720]: pgmap v570: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.0 MiB/s wr, 106 op/s 2024-01-31T05:40:37.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:36 smithi186 ceph-mon[35659]: pgmap v570: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.0 MiB/s wr, 106 op/s 2024-01-31T05:40:37.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:36 smithi045 ceph-mon[31585]: pgmap v570: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.0 MiB/s wr, 106 op/s 2024-01-31T05:40:38.399 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-01-31T05:40:38.400 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-01-31T05:40:38.400 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2228, 2024-01-31T05:40:38.400 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2228, 2024-01-31T05:40:38.400 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 680069171, 2024-01-31T05:40:38.400 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 692176, 2024-01-31T05:40:38.401 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 8, 2024-01-31T05:40:38.401 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2024-01-31T05:40:38.401 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.435621906, 2024-01-31T05:40:38.401 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.043756735, 2024-01-31T05:40:38.401 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.333805330, 2024-01-31T05:40:38.401 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.044580970, 2024-01-31T05:40:38.401 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:38.401 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2227, 2024-01-31T05:40:38.402 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.047774473, 2024-01-31T05:40:38.402 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000021452 2024-01-31T05:40:38.402 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.402 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:38.402 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.402 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.402 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.402 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.402 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-01-31T05:40:38.403 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 448 2024-01-31T05:40:38.403 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.403 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-01-31T05:40:38.403 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1956, 2024-01-31T05:40:38.403 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1931, 2024-01-31T05:40:38.403 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 497015729, 2024-01-31T05:40:38.403 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 690261, 2024-01-31T05:40:38.403 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2024-01-31T05:40:38.403 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2024-01-31T05:40:38.404 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.388802975, 2024-01-31T05:40:38.404 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.044091687, 2024-01-31T05:40:38.404 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.306681453, 2024-01-31T05:40:38.404 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.038070305, 2024-01-31T05:40:38.404 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:38.404 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1930, 2024-01-31T05:40:38.404 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.044713825, 2024-01-31T05:40:38.404 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000023167 2024-01-31T05:40:38.405 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.405 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:38.405 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.405 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.405 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.405 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.405 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1682240, 2024-01-31T05:40:38.405 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 202608 2024-01-31T05:40:38.406 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.406 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-01-31T05:40:38.406 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2967, 2024-01-31T05:40:38.406 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 3214, 2024-01-31T05:40:38.406 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 602441636, 2024-01-31T05:40:38.406 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1519203, 2024-01-31T05:40:38.406 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 8, 2024-01-31T05:40:38.406 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2024-01-31T05:40:38.406 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.509163899, 2024-01-31T05:40:38.407 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.067885817, 2024-01-31T05:40:38.407 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.348106146, 2024-01-31T05:40:38.407 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.068831093, 2024-01-31T05:40:38.407 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-01-31T05:40:38.407 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3211, 2024-01-31T05:40:38.407 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.089766119, 2024-01-31T05:40:38.407 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000027955 2024-01-31T05:40:38.407 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.408 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-01-31T05:40:38.408 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.408 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.408 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.408 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.408 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 320, 2024-01-31T05:40:38.408 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 597088 2024-01-31T05:40:38.408 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.409 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-01-31T05:40:38.409 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-01-31T05:40:38.409 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-01-31T05:40:38.409 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.409 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-01-31T05:40:38.409 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-01-31T05:40:38.409 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-01-31T05:40:38.409 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 9945, 2024-01-31T05:40:38.409 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.299778022, 2024-01-31T05:40:38.410 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000030143 2024-01-31T05:40:38.410 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:38.410 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.410 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-01-31T05:40:38.410 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-01-31T05:40:38.410 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-01-31T05:40:38.410 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.410 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.410 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.411 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:38.411 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.411 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-01-31T05:40:38.411 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2024-01-31T05:40:38.411 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2024-01-31T05:40:38.411 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-01-31T05:40:38.411 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.411 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.412 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.412 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.412 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-01-31T05:40:38.412 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2024-01-31T05:40:38.412 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-01-31T05:40:38.412 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-01-31T05:40:38.412 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2024-01-31T05:40:38.412 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2024-01-31T05:40:38.413 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2024-01-31T05:40:38.413 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 73725, 2024-01-31T05:40:38.413 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 35688, 2024-01-31T05:40:38.413 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 38037, 2024-01-31T05:40:38.413 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 0, 2024-01-31T05:40:38.413 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 22742, 2024-01-31T05:40:38.413 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 0, 2024-01-31T05:40:38.413 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2024-01-31T05:40:38.414 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2024-01-31T05:40:38.414 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 15, 2024-01-31T05:40:38.414 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2024-01-31T05:40:38.414 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2024-01-31T05:40:38.414 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2024-01-31T05:40:38.414 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2024-01-31T05:40:38.414 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-01-31T05:40:38.414 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-01-31T05:40:38.415 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2024-01-31T05:40:38.415 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-01-31T05:40:38.415 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-01-31T05:40:38.415 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2024-01-31T05:40:38.415 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2024-01-31T05:40:38.415 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2024-01-31T05:40:38.415 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2024-01-31T05:40:38.415 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-01-31T05:40:38.415 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2024-01-31T05:40:38.416 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-01-31T05:40:38.416 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2024-01-31T05:40:38.416 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2024-01-31T05:40:38.416 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2024-01-31T05:40:38.416 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2024-01-31T05:40:38.416 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-01-31T05:40:38.416 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-01-31T05:40:38.416 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-01-31T05:40:38.417 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2024-01-31T05:40:38.417 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2024-01-31T05:40:38.417 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2024-01-31T05:40:38.417 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2024-01-31T05:40:38.417 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2024-01-31T05:40:38.417 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2024-01-31T05:40:38.417 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2024-01-31T05:40:38.417 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-01-31T05:40:38.418 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-01-31T05:40:38.418 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-31T05:40:38.418 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2024-01-31T05:40:38.418 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2024-01-31T05:40:38.418 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2024-01-31T05:40:38.418 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2024-01-31T05:40:38.418 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.418 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-01-31T05:40:38.419 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2024-01-31T05:40:38.419 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2024-01-31T05:40:38.419 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2024-01-31T05:40:38.419 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2024-01-31T05:40:38.419 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2024-01-31T05:40:38.419 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2024-01-31T05:40:38.419 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-01-31T05:40:38.419 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-01-31T05:40:38.420 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2024-01-31T05:40:38.420 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2024-01-31T05:40:38.420 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-01-31T05:40:38.420 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-01-31T05:40:38.420 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-01-31T05:40:38.420 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-01-31T05:40:38.420 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-01-31T05:40:38.420 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-01-31T05:40:38.420 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-01-31T05:40:38.421 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2024-01-31T05:40:38.421 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2024-01-31T05:40:38.421 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-01-31T05:40:38.421 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2024-01-31T05:40:38.421 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-01-31T05:40:38.421 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-01-31T05:40:38.421 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.421 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-01-31T05:40:38.421 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-01-31T05:40:38.422 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2024-01-31T05:40:38.422 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2024-01-31T05:40:38.422 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2024-01-31T05:40:38.422 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 322525, 2024-01-31T05:40:38.422 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-01-31T05:40:38.422 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2024-01-31T05:40:38.422 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2024-01-31T05:40:38.422 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2024-01-31T05:40:38.423 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2024-01-31T05:40:38.423 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 136, 2024-01-31T05:40:38.423 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 28, 2024-01-31T05:40:38.423 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-01-31T05:40:38.423 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2024-01-31T05:40:38.423 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 1255438933, 2024-01-31T05:40:38.423 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 1780172594, 2024-01-31T05:40:38.423 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 1780166276, 2024-01-31T05:40:38.423 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-01-31T05:40:38.424 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.424 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.424 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.424 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.424 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 322525 2024-01-31T05:40:38.424 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.424 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-01-31T05:40:38.424 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 3331, 2024-01-31T05:40:38.424 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 73417, 2024-01-31T05:40:38.425 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 70086, 2024-01-31T05:40:38.425 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 203, 2024-01-31T05:40:38.425 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 444, 2024-01-31T05:40:38.425 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 241, 2024-01-31T05:40:38.425 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 73755, 2024-01-31T05:40:38.425 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 74020, 2024-01-31T05:40:38.425 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 265, 2024-01-31T05:40:38.425 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2024-01-31T05:40:38.426 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2024-01-31T05:40:38.426 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2024-01-31T05:40:38.426 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 129720, 2024-01-31T05:40:38.426 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207228 2024-01-31T05:40:38.426 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.426 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-01-31T05:40:38.426 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2024-01-31T05:40:38.426 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2024-01-31T05:40:38.427 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2024-01-31T05:40:38.427 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2024-01-31T05:40:38.427 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2024-01-31T05:40:38.427 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-01-31T05:40:38.427 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.427 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.427 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.427 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.427 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-01-31T05:40:38.428 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.428 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.428 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.428 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.428 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-01-31T05:40:38.428 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.428 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.428 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.429 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.429 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-01-31T05:40:38.429 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.429 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.429 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.429 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.429 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-01-31T05:40:38.429 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.430 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.430 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.430 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.430 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-01-31T05:40:38.430 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.430 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.430 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.430 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.430 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-01-31T05:40:38.430 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.431 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.431 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.431 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.431 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-01-31T05:40:38.431 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.431 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.431 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.431 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.432 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-01-31T05:40:38.432 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.432 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.432 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.432 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.432 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-01-31T05:40:38.432 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.432 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.432 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.433 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.433 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-01-31T05:40:38.433 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.433 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.433 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.433 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.433 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-01-31T05:40:38.434 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.434 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.434 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.434 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.434 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-01-31T05:40:38.434 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.435 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.435 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.435 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.435 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-01-31T05:40:38.435 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.435 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.435 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.435 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.435 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-01-31T05:40:38.436 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.436 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.436 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.436 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.436 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-01-31T05:40:38.436 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.436 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.436 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.436 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.437 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-01-31T05:40:38.437 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.437 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.437 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.437 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.437 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-01-31T05:40:38.437 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.437 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.438 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.438 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.438 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-01-31T05:40:38.438 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.438 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.438 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.438 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.438 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-01-31T05:40:38.439 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.439 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.439 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.439 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.439 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-01-31T05:40:38.439 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.439 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.439 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.440 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.440 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-01-31T05:40:38.440 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.440 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.440 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.440 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.440 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-01-31T05:40:38.440 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.440 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.441 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.441 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.441 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-01-31T05:40:38.441 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.441 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.441 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.441 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.441 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-01-31T05:40:38.442 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.442 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.442 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.442 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.442 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-01-31T05:40:38.442 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.442 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.442 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.443 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.443 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-01-31T05:40:38.443 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.443 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.443 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.443 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.443 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-01-31T05:40:38.443 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.443 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.444 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.444 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.444 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-01-31T05:40:38.444 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.444 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.444 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.444 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.444 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-01-31T05:40:38.445 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-01-31T05:40:38.445 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-01-31T05:40:38.445 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.445 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.445 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.445 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:38.445 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.445 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-01-31T05:40:38.445 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2024-01-31T05:40:38.446 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 2, 2024-01-31T05:40:38.446 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 0, 2024-01-31T05:40:38.446 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2024-01-31T05:40:38.446 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-01-31T05:40:38.446 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2024-01-31T05:40:38.446 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2024-01-31T05:40:38.446 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 2427, 2024-01-31T05:40:38.446 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-01-31T05:40:38.446 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.447 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-01-31T05:40:38.447 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-01-31T05:40:38.447 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-01-31T05:40:38.447 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-01-31T05:40:38.447 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-01-31T05:40:38.447 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-01-31T05:40:38.447 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-01-31T05:40:38.447 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-31T05:40:38.448 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-01-31T05:40:38.448 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-31T05:40:38.448 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-01-31T05:40:38.448 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-01-31T05:40:38.448 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-01-31T05:40:38.448 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-31T05:40:38.448 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-01-31T05:40:38.448 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-01-31T05:40:38.448 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-01-31T05:40:38.449 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-01-31T05:40:38.449 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-01-31T05:40:38.449 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-31T05:40:38.449 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-01-31T05:40:38.449 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-31T05:40:38.449 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-01-31T05:40:38.449 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-01-31T05:40:38.449 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-01-31T05:40:38.450 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-01-31T05:40:38.450 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-01-31T05:40:38.450 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-31T05:40:38.450 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-01-31T05:40:38.450 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-01-31T05:40:38.450 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-01-31T05:40:38.450 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-01-31T05:40:38.450 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-01-31T05:40:38.450 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-01-31T05:40:38.451 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-01-31T05:40:38.451 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-01-31T05:40:38.451 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-01-31T05:40:38.451 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 387152, 2024-01-31T05:40:38.451 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 4071, 2024-01-31T05:40:38.451 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2024-01-31T05:40:38.451 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2024-01-31T05:40:38.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-01-31T05:40:38.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-01-31T05:40:38.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-01-31T05:40:38.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-01-31T05:40:38.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-01-31T05:40:38.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-01-31T05:40:38.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8160, 2024-01-31T05:40:38.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 186, 2024-01-31T05:40:38.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-01-31T05:40:38.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-01-31T05:40:38.453 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-01-31T05:40:38.453 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-01-31T05:40:38.453 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 60677018, 2024-01-31T05:40:38.453 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 438295, 2024-01-31T05:40:38.453 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-01-31T05:40:38.453 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-01-31T05:40:38.453 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-01-31T05:40:38.453 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-01-31T05:40:38.453 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.454 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-01-31T05:40:38.454 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-01-31T05:40:38.454 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-01-31T05:40:38.454 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 6757, 2024-01-31T05:40:38.454 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21, 2024-01-31T05:40:38.454 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2024-01-31T05:40:38.454 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 6757, 2024-01-31T05:40:38.454 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-01-31T05:40:38.454 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6757, 2024-01-31T05:40:38.455 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 7.583881410, 2024-01-31T05:40:38.455 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001122374 2024-01-31T05:40:38.455 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.455 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-01-31T05:40:38.455 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-01-31T05:40:38.455 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6757, 2024-01-31T05:40:38.455 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 6758 2024-01-31T05:40:38.455 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.455 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 6757, 2024-01-31T05:40:38.456 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 6757, 2024-01-31T05:40:38.456 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2024-01-31T05:40:38.456 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-01-31T05:40:38.456 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-01-31T05:40:38.456 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 4255, 2024-01-31T05:40:38.456 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2024-01-31T05:40:38.456 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 2500, 2024-01-31T05:40:38.456 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2024-01-31T05:40:38.457 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2024-01-31T05:40:38.457 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-01-31T05:40:38.457 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-01-31T05:40:38.457 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-01-31T05:40:38.457 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2024-01-31T05:40:38.457 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2024-01-31T05:40:38.457 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-01-31T05:40:38.457 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-01-31T05:40:38.458 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-01-31T05:40:38.458 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1, 2024-01-31T05:40:38.458 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2024-01-31T05:40:38.458 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-01-31T05:40:38.458 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-01-31T05:40:38.458 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-01-31T05:40:38.458 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-01-31T05:40:38.458 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-01-31T05:40:38.458 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-01-31T05:40:38.459 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-01-31T05:40:38.459 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-01-31T05:40:38.459 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-01-31T05:40:38.459 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-01-31T05:40:38.459 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-01-31T05:40:38.459 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-01-31T05:40:38.459 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-01-31T05:40:38.459 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-01-31T05:40:38.459 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-01-31T05:40:38.460 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-01-31T05:40:38.460 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-01-31T05:40:38.460 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-01-31T05:40:38.460 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-01-31T05:40:38.460 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-01-31T05:40:38.460 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-01-31T05:40:38.460 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-01-31T05:40:38.460 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-01-31T05:40:38.461 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-01-31T05:40:38.461 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-01-31T05:40:38.461 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-01-31T05:40:38.461 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 119, 2024-01-31T05:40:38.461 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-01-31T05:40:38.461 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 33, 2024-01-31T05:40:38.461 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 11, 2024-01-31T05:40:38.461 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 11, 2024-01-31T05:40:38.461 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-01-31T05:40:38.462 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-01-31T05:40:38.462 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2024-01-31T05:40:38.462 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2024-01-31T05:40:38.462 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2024-01-31T05:40:38.462 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.462 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-01-31T05:40:38.462 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2024-01-31T05:40:38.462 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-01-31T05:40:38.462 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-01-31T05:40:38.463 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-01-31T05:40:38.463 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.463 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-01-31T05:40:38.463 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-01-31T05:40:38.463 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.463 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-01-31T05:40:38.463 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-01-31T05:40:38.463 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2024-01-31T05:40:38.463 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-01-31T05:40:38.464 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2024-01-31T05:40:38.464 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 0, 2024-01-31T05:40:38.464 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2024-01-31T05:40:38.464 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-01-31T05:40:38.464 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.464 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-31T05:40:38.464 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-01-31T05:40:38.464 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-01-31T05:40:38.464 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-01-31T05:40:38.465 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 7147, 2024-01-31T05:40:38.465 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 1778860598, 2024-01-31T05:40:38.465 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:38.465 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 7147, 2024-01-31T05:40:38.465 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-01-31T05:40:38.465 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-01-31T05:40:38.465 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 7147, 2024-01-31T05:40:38.465 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 1778860598, 2024-01-31T05:40:38.466 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-01-31T05:40:38.466 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.466 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.466 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.466 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:38.466 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.466 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-01-31T05:40:38.466 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-01-31T05:40:38.466 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-01-31T05:40:38.467 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-01-31T05:40:38.467 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-01-31T05:40:38.467 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-01-31T05:40:38.467 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:38.467 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-01-31T05:40:38.467 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 6757, 2024-01-31T05:40:38.467 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 1776009472, 2024-01-31T05:40:38.467 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 6757, 2024-01-31T05:40:38.467 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 1776009472, 2024-01-31T05:40:38.468 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-01-31T05:40:38.468 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.468 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.468 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.468 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:38.468 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.468 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-01-31T05:40:38.468 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-01-31T05:40:38.469 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-01-31T05:40:38.469 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-01-31T05:40:38.469 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-01-31T05:40:38.469 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-01-31T05:40:38.469 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:38.469 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-01-31T05:40:38.469 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 6757, 2024-01-31T05:40:38.469 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 6757, 2024-01-31T05:40:38.470 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 6757, 2024-01-31T05:40:38.470 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 6757, 2024-01-31T05:40:38.470 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-01-31T05:40:38.470 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.470 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.470 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.470 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:38.470 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.470 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-01-31T05:40:38.470 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-01-31T05:40:38.471 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-01-31T05:40:38.471 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-01-31T05:40:38.471 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-01-31T05:40:38.471 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-01-31T05:40:38.471 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:38.471 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-01-31T05:40:38.471 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-01-31T05:40:38.471 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-01-31T05:40:38.471 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2024-01-31T05:40:38.472 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2024-01-31T05:40:38.472 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-01-31T05:40:38.472 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.472 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.472 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.472 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:38.472 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-01-31T05:40:38.472 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x5589008fc960": { 2024-01-31T05:40:38.472 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-01-31T05:40:38.473 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-01-31T05:40:38.473 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-01-31T05:40:38.473 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-01-31T05:40:38.473 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-01-31T05:40:38.473 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-01-31T05:40:38.473 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-01-31T05:40:38.473 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-01-31T05:40:38.473 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-01-31T05:40:38.474 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2024-01-31T05:40:38.474 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2024-01-31T05:40:38.474 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-01-31T05:40:38.474 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-01-31T05:40:38.474 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-01-31T05:40:38.474 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-01-31T05:40:38.474 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:38.474 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-01-31T05:40:38.475 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-01-31T05:40:39.126 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-01-31T05:40:39.126 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-01-31T05:40:39.126 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-01-31T05:40:39.126 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-01-31T05:40:39.126 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-01-31T05:40:39.127 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-01-31T05:40:39.127 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-01-31T05:40:39.127 DEBUG:tasks.check_counter:Getting stats from h 2024-01-31T05:40:39.127 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph daemon mds.h perf dump 2024-01-31T05:40:39.154 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:38 smithi156 ceph-mon[35720]: pgmap v571: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 4.8 MiB/s wr, 165 op/s 2024-01-31T05:40:39.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:38 smithi186 ceph-mon[35659]: pgmap v571: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 4.8 MiB/s wr, 165 op/s 2024-01-31T05:40:39.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:38 smithi045 ceph-mon[31585]: pgmap v571: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 4.8 MiB/s wr, 165 op/s 2024-01-31T05:40:39.409 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/mon.b/config 2024-01-31T05:40:41.017 INFO:teuthology.orchestra.run.smithi156.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-01-31T05:40:41.018 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:40 smithi156 ceph-mon[35720]: pgmap v572: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 2.8 MiB/s wr, 108 op/s 2024-01-31T05:40:41.195 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:40 smithi045 ceph-mon[31585]: pgmap v572: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 2.8 MiB/s wr, 108 op/s 2024-01-31T05:40:41.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:40 smithi186 ceph-mon[35659]: pgmap v572: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 2.8 MiB/s wr, 108 op/s 2024-01-31T05:40:41.615 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-01-31T05:40:41.616 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/ceph_manager.py", line 1864, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi156 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 -- ceph daemon mds.h perf dump' 2024-01-31T05:40:41.626 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-01-31T05:40:41.637 INFO:tasks.kclient:Unmounting kernel clients... 2024-01-31T05:40:41.638 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:40:41.638 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:40:41.657 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:40:41.657 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:40:41.714 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:40:41.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.861+0000 7f1f082b1640 1 -- 172.21.15.45:0/3289511986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f00102880 msgr2=0x7f1f0010ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.861+0000 7f1f082b1640 1 --2- 172.21.15.45:0/3289511986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f00102880 0x7f1f0010ad60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009900 tx=0x7f1ef002bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.861+0000 7f1f082b1640 1 -- 172.21.15.45:0/3289511986 shutdown_connections 2024-01-31T05:40:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.861+0000 7f1f082b1640 1 --2- 172.21.15.45:0/3289511986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f00102880 0x7f1f0010ad60 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.861+0000 7f1f082b1640 1 --2- 172.21.15.45:0/3289511986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f0010b330 0x7f1f0010d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.861+0000 7f1f082b1640 1 --2- 172.21.15.45:0/3289511986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f00101f60 0x7f1f00102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.861+0000 7f1f082b1640 1 -- 172.21.15.45:0/3289511986 >> 172.21.15.45:0/3289511986 conn(0x7f1f00076cf0 msgr2=0x7f1f00077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.861+0000 7f1f082b1640 1 -- 172.21.15.45:0/3289511986 shutdown_connections 2024-01-31T05:40:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.861+0000 7f1f082b1640 1 -- 172.21.15.45:0/3289511986 wait complete. 2024-01-31T05:40:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f082b1640 1 Processor -- start 2024-01-31T05:40:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f082b1640 1 -- start start 2024-01-31T05:40:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f082b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f00101f60 0x7f1f00197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f082b1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f00102880 0x7f1f001977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f082b1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f0010b330 0x7f1f0019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f082b1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0019c150 con 0x7f1f00102880 2024-01-31T05:40:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f082b1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0019c2c0 con 0x7f1f00101f60 2024-01-31T05:40:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f082b1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0019c480 con 0x7f1f0010b330 2024-01-31T05:40:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f0584c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f00102880 0x7f1f001977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f0604d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f00101f60 0x7f1f00197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f0604d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f00101f60 0x7f1f00197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:55908/0 (socket says 172.21.15.45:55908) 2024-01-31T05:40:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f0604d640 1 -- 172.21.15.45:0/3676220998 learned_addr learned my addr 172.21.15.45:0/3676220998 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f0584c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f00102880 0x7f1f001977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36978/0 (socket says 172.21.15.45:36978) 2024-01-31T05:40:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.862+0000 7f1f0684e640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f0010b330 0x7f1f0019bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.863+0000 7f1f0584c640 1 -- 172.21.15.45:0/3676220998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f0010b330 msgr2=0x7f1f0019bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.863+0000 7f1f0584c640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f0010b330 0x7f1f0019bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.863+0000 7f1f0584c640 1 -- 172.21.15.45:0/3676220998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f00101f60 msgr2=0x7f1f00197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.863+0000 7f1f0584c640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f00101f60 0x7f1f00197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.863+0000 7f1f0584c640 1 -- 172.21.15.45:0/3676220998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef0009560 con 0x7f1f00102880 2024-01-31T05:40:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.863+0000 7f1f0604d640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f00101f60 0x7f1f00197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:40:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.863+0000 7f1f0584c640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f00102880 0x7f1f001977d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0036040 tx=0x7f1ef002cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.863+0000 7f1ef77fe640 1 -- 172.21.15.45:0/3676220998 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0004390 con 0x7f1f00102880 2024-01-31T05:40:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.863+0000 7f1ef77fe640 1 -- 172.21.15.45:0/3676220998 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ef00044f0 con 0x7f1f00102880 2024-01-31T05:40:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.863+0000 7f1f082b1640 1 -- 172.21.15.45:0/3676220998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0019c700 con 0x7f1f00102880 2024-01-31T05:40:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.863+0000 7f1ef77fe640 1 -- 172.21.15.45:0/3676220998 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0003d10 con 0x7f1f00102880 2024-01-31T05:40:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.863+0000 7f1f082b1640 1 -- 172.21.15.45:0/3676220998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f001a3fe0 con 0x7f1f00102880 2024-01-31T05:40:41.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.864+0000 7f1f082b1640 1 -- 172.21.15.45:0/3676220998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f00067ae0 con 0x7f1f00102880 2024-01-31T05:40:41.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.866+0000 7f1ef77fe640 1 -- 172.21.15.45:0/3676220998 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f1ef0005000 con 0x7f1f00102880 2024-01-31T05:40:41.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.866+0000 7f1ef77fe640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1eec0a14d0 0x7f1eec0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:41.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.867+0000 7f1f0604d640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1eec0a14d0 0x7f1eec0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:41.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.867+0000 7f1ef77fe640 1 -- 172.21.15.45:0/3676220998 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1ef00ef5f0 con 0x7f1f00102880 2024-01-31T05:40:41.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.867+0000 7f1f0604d640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1eec0a14d0 0x7f1eec0a3990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1efc005fd0 tx=0x7f1efc00b480 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:41.871+0000 7f1ef77fe640 1 -- 172.21.15.45:0/3676220998 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1ef00b8840 con 0x7f1f00102880 2024-01-31T05:40:42.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.058+0000 7f1f082b1640 1 -- 172.21.15.45:0/3676220998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f00106420 con 0x7f1f00102880 2024-01-31T05:40:42.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.059+0000 7f1ef77fe640 1 -- 172.21.15.45:0/3676220998 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+23052 (secure 0 0 0) 0x7f1ef00bbe60 con 0x7f1f00102880 2024-01-31T05:40:42.059 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:40:42.060 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":119,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:40:12.712633+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":118,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/1441130788":"2024-01-31T06:27:35.007948+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/782292734":"2024-02-01T05:22:41.447737+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2742663751":"2024-02-01T05:22:41.447737+0000","172.21.15.45:0/1829415949":"2024-02-01T05:22:41.447737+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2788245525":"2024-02-01T05:22:41.447737+0000","172.21.15.45:0/948304328":"2024-02-01T05:22:41.447737+0000","172.21.15.45:6801/2788245525":"2024-02-01T05:22:41.447737+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:40:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.061+0000 7f1f082b1640 1 -- 172.21.15.45:0/3676220998 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1eec0a14d0 msgr2=0x7f1eec0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.061+0000 7f1f082b1640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1eec0a14d0 0x7f1eec0a3990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1efc005fd0 tx=0x7f1efc00b480 comp rx=0 tx=0).stop 2024-01-31T05:40:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.061+0000 7f1f082b1640 1 -- 172.21.15.45:0/3676220998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f00102880 msgr2=0x7f1f001977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.061+0000 7f1f082b1640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f00102880 0x7f1f001977d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0036040 tx=0x7f1ef002cd00 comp rx=0 tx=0).stop 2024-01-31T05:40:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.061+0000 7f1f082b1640 1 -- 172.21.15.45:0/3676220998 shutdown_connections 2024-01-31T05:40:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.061+0000 7f1f082b1640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f1eec0a14d0 0x7f1eec0a3990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.061+0000 7f1f082b1640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f00102880 0x7f1f001977d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.061+0000 7f1f082b1640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f0010b330 0x7f1f0019bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.061+0000 7f1f082b1640 1 --2- 172.21.15.45:0/3676220998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f00101f60 0x7f1f00197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.061+0000 7f1f082b1640 1 -- 172.21.15.45:0/3676220998 >> 172.21.15.45:0/3676220998 conn(0x7f1f00076cf0 msgr2=0x7f1f000fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.061+0000 7f1f082b1640 1 -- 172.21.15.45:0/3676220998 shutdown_connections 2024-01-31T05:40:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.061+0000 7f1f082b1640 1 -- 172.21.15.45:0/3676220998 wait complete. 2024-01-31T05:40:42.073 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:16:10.856581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:20:38.347599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:20:39.270522+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 118, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:40:42.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:40:42.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.199+0000 7f8b1a092640 1 -- 172.21.15.45:0/1614667782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b14106030 msgr2=0x7f8b1410dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.199+0000 7f8b1a092640 1 --2- 172.21.15.45:0/1614667782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b14106030 0x7f8b1410dbe0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009930 tx=0x7f8b0c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.199+0000 7f8b1a092640 1 -- 172.21.15.45:0/1614667782 shutdown_connections 2024-01-31T05:40:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.199+0000 7f8b1a092640 1 --2- 172.21.15.45:0/1614667782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b14106030 0x7f8b1410dbe0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.199+0000 7f8b1a092640 1 --2- 172.21.15.45:0/1614667782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b14068b10 0x7f8b14105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.199+0000 7f8b1a092640 1 --2- 172.21.15.45:0/1614667782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b14068160 0x7f8b14068540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.199+0000 7f8b1a092640 1 -- 172.21.15.45:0/1614667782 >> 172.21.15.45:0/1614667782 conn(0x7f8b14076cf0 msgr2=0x7f8b14077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:42.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.199+0000 7f8b1a092640 1 -- 172.21.15.45:0/1614667782 shutdown_connections 2024-01-31T05:40:42.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.199+0000 7f8b1a092640 1 -- 172.21.15.45:0/1614667782 wait complete. 2024-01-31T05:40:42.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.200+0000 7f8b1a092640 1 Processor -- start 2024-01-31T05:40:42.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.200+0000 7f8b1a092640 1 -- start start 2024-01-31T05:40:42.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.200+0000 7f8b1a092640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b14068160 0x7f8b14199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.200+0000 7f8b1a092640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b14106030 0x7f8b14199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.200+0000 7f8b1a092640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1419dce0 0x7f8b1419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.200+0000 7f8b1a092640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1419e6d0 con 0x7f8b1419dce0 2024-01-31T05:40:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.200+0000 7f8b1a092640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1419e840 con 0x7f8b14106030 2024-01-31T05:40:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.200+0000 7f8b1a092640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1419e9b0 con 0x7f8b14068160 2024-01-31T05:40:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.200+0000 7f8b13fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1419dce0 0x7f8b1419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.200+0000 7f8b13fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1419dce0 0x7f8b1419e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36998/0 (socket says 172.21.15.45:36998) 2024-01-31T05:40:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.200+0000 7f8b13fff640 1 -- 172.21.15.45:0/1050205891 learned_addr learned my addr 172.21.15.45:0/1050205891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.200+0000 7f8b137fe640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b14068160 0x7f8b14199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.201+0000 7f8b12ffd640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b14106030 0x7f8b14199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.201+0000 7f8b13fff640 1 -- 172.21.15.45:0/1050205891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b14068160 msgr2=0x7f8b14199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.201+0000 7f8b13fff640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b14068160 0x7f8b14199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.201+0000 7f8b13fff640 1 -- 172.21.15.45:0/1050205891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b14106030 msgr2=0x7f8b14199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.201+0000 7f8b13fff640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b14106030 0x7f8b14199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.201+0000 7f8b13fff640 1 -- 172.21.15.45:0/1050205891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c009560 con 0x7f8b1419dce0 2024-01-31T05:40:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.201+0000 7f8b13fff640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1419dce0 0x7f8b1419e190 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c005940 tx=0x7f8b0c004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.201+0000 7f8b10ff9640 1 -- 172.21.15.45:0/1050205891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0042d0 con 0x7f8b1419dce0 2024-01-31T05:40:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.201+0000 7f8b10ff9640 1 -- 172.21.15.45:0/1050205891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b0c004430 con 0x7f8b1419dce0 2024-01-31T05:40:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.201+0000 7f8b1a092640 1 -- 172.21.15.45:0/1050205891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b141a5f90 con 0x7f8b1419dce0 2024-01-31T05:40:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.201+0000 7f8b10ff9640 1 -- 172.21.15.45:0/1050205891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c003c50 con 0x7f8b1419dce0 2024-01-31T05:40:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.201+0000 7f8b1a092640 1 -- 172.21.15.45:0/1050205891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b141a6410 con 0x7f8b1419dce0 2024-01-31T05:40:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.202+0000 7f8b1a092640 1 -- 172.21.15.45:0/1050205891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b14067ae0 con 0x7f8b1419dce0 2024-01-31T05:40:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.203+0000 7f8b10ff9640 1 -- 172.21.15.45:0/1050205891 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f8b0c005000 con 0x7f8b1419dce0 2024-01-31T05:40:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.204+0000 7f8b10ff9640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8afc0a1450 0x7f8afc0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.204+0000 7f8b137fe640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8afc0a1450 0x7f8afc0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.204+0000 7f8b10ff9640 1 -- 172.21.15.45:0/1050205891 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8b0c0efad0 con 0x7f8b1419dce0 2024-01-31T05:40:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.204+0000 7f8b137fe640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8afc0a1450 0x7f8afc0a3910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8b04005fd0 tx=0x7f8b04009680 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.207+0000 7f8b10ff9640 1 -- 172.21.15.45:0/1050205891 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8b0c0b8d20 con 0x7f8b1419dce0 2024-01-31T05:40:42.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.389+0000 7f8b1a092640 1 -- 172.21.15.45:0/1050205891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b14068be0 con 0x7f8b1419dce0 2024-01-31T05:40:42.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.389+0000 7f8b10ff9640 1 -- 172.21.15.45:0/1050205891 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b0c0bc340 con 0x7f8b1419dce0 2024-01-31T05:40:42.389 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-01-31T05:40:42.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.391+0000 7f8b1a092640 1 -- 172.21.15.45:0/1050205891 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8afc0a1450 msgr2=0x7f8afc0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.391+0000 7f8b1a092640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8afc0a1450 0x7f8afc0a3910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8b04005fd0 tx=0x7f8b04009680 comp rx=0 tx=0).stop 2024-01-31T05:40:42.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.391+0000 7f8b1a092640 1 -- 172.21.15.45:0/1050205891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1419dce0 msgr2=0x7f8b1419e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.391+0000 7f8b1a092640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1419dce0 0x7f8b1419e190 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c005940 tx=0x7f8b0c004dd0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.392+0000 7f8b1a092640 1 -- 172.21.15.45:0/1050205891 shutdown_connections 2024-01-31T05:40:42.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.392+0000 7f8b1a092640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8afc0a1450 0x7f8afc0a3910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.392+0000 7f8b1a092640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1419dce0 0x7f8b1419e190 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.392+0000 7f8b1a092640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b14106030 0x7f8b14199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.392+0000 7f8b1a092640 1 --2- 172.21.15.45:0/1050205891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b14068160 0x7f8b14199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.392+0000 7f8b1a092640 1 -- 172.21.15.45:0/1050205891 >> 172.21.15.45:0/1050205891 conn(0x7f8b14076cf0 msgr2=0x7f8b1410bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:42.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.392+0000 7f8b1a092640 1 -- 172.21.15.45:0/1050205891 shutdown_connections 2024-01-31T05:40:42.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.392+0000 7f8b1a092640 1 -- 172.21.15.45:0/1050205891 wait complete. 2024-01-31T05:40:42.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:40:42.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.508+0000 7f473cb1f640 1 -- 172.21.15.45:0/3355384619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4738100780 msgr2=0x7f47381047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.508+0000 7f473cb1f640 1 --2- 172.21.15.45:0/3355384619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4738100780 0x7f47381047c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4720009930 tx=0x7f472002bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:42.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.509+0000 7f473cb1f640 1 -- 172.21.15.45:0/3355384619 shutdown_connections 2024-01-31T05:40:42.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.509+0000 7f473cb1f640 1 --2- 172.21.15.45:0/3355384619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4738100780 0x7f47381047c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.509+0000 7f473cb1f640 1 --2- 172.21.15.45:0/3355384619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4738104ea0 0x7f473810ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.509+0000 7f473cb1f640 1 --2- 172.21.15.45:0/3355384619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47380ffdd0 0x7f47381001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.509+0000 7f473cb1f640 1 -- 172.21.15.45:0/3355384619 >> 172.21.15.45:0/3355384619 conn(0x7f4738076cf0 msgr2=0x7f4738077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:42.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.509+0000 7f473cb1f640 1 -- 172.21.15.45:0/3355384619 shutdown_connections 2024-01-31T05:40:42.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.509+0000 7f473cb1f640 1 -- 172.21.15.45:0/3355384619 wait complete. 2024-01-31T05:40:42.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.509+0000 7f473cb1f640 1 Processor -- start 2024-01-31T05:40:42.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.509+0000 7f473cb1f640 1 -- start start 2024-01-31T05:40:42.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.509+0000 7f473cb1f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47380ffdd0 0x7f47381980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:42.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473cb1f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4738104ea0 0x7f47381985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:42.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473cb1f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4738198b70 0x7f47381a3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:42.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473cb1f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47381a37c0 con 0x7f47380ffdd0 2024-01-31T05:40:42.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473cb1f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47381a3930 con 0x7f4738104ea0 2024-01-31T05:40:42.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473cb1f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47381a3af0 con 0x7f4738198b70 2024-01-31T05:40:42.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47380ffdd0 0x7f47381980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:42.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47380ffdd0 0x7f47381980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37016/0 (socket says 172.21.15.45:37016) 2024-01-31T05:40:42.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473659c640 1 -- 172.21.15.45:0/2325534167 learned_addr learned my addr 172.21.15.45:0/2325534167 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:42.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f4735d9b640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4738104ea0 0x7f47381985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:42.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f4736d9d640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4738198b70 0x7f47381a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:42.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473659c640 1 -- 172.21.15.45:0/2325534167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4738198b70 msgr2=0x7f47381a3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473659c640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4738198b70 0x7f47381a3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473659c640 1 -- 172.21.15.45:0/2325534167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4738104ea0 msgr2=0x7f47381985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473659c640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4738104ea0 0x7f47381985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473659c640 1 -- 172.21.15.45:0/2325534167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4720009560 con 0x7f47380ffdd0 2024-01-31T05:40:42.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473659c640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47380ffdd0 0x7f47381980a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f472800e9f0 tx=0x7f472800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:42.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f472f7fe640 1 -- 172.21.15.45:0/2325534167 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728009700 con 0x7f47380ffdd0 2024-01-31T05:40:42.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f472f7fe640 1 -- 172.21.15.45:0/2325534167 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f472800cd80 con 0x7f47380ffdd0 2024-01-31T05:40:42.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473cb1f640 1 -- 172.21.15.45:0/2325534167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47381a3dd0 con 0x7f47380ffdd0 2024-01-31T05:40:42.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f472f7fe640 1 -- 172.21.15.45:0/2325534167 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728018860 con 0x7f47380ffdd0 2024-01-31T05:40:42.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.510+0000 7f473cb1f640 1 -- 172.21.15.45:0/2325534167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47381a4250 con 0x7f47380ffdd0 2024-01-31T05:40:42.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.511+0000 7f473cb1f640 1 -- 172.21.15.45:0/2325534167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4738067ae0 con 0x7f47380ffdd0 2024-01-31T05:40:42.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.512+0000 7f472f7fe640 1 -- 172.21.15.45:0/2325534167 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f47280189c0 con 0x7f47380ffdd0 2024-01-31T05:40:42.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.513+0000 7f472f7fe640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f47240a1450 0x7f47240a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:42.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.513+0000 7f4735d9b640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f47240a1450 0x7f47240a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:42.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.513+0000 7f472f7fe640 1 -- 172.21.15.45:0/2325534167 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4728014070 con 0x7f47380ffdd0 2024-01-31T05:40:42.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.514+0000 7f4735d9b640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f47240a1450 0x7f47240a3910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4720000c00 tx=0x7f4720005fd0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:42.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.517+0000 7f472f7fe640 1 -- 172.21.15.45:0/2325534167 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f47280968b0 con 0x7f47380ffdd0 2024-01-31T05:40:42.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.712+0000 7f473cb1f640 1 -- 172.21.15.45:0/2325534167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4738100780 con 0x7f47380ffdd0 2024-01-31T05:40:42.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.712+0000 7f472f7fe640 1 -- 172.21.15.45:0/2325534167 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4728099ed0 con 0x7f47380ffdd0 2024-01-31T05:40:42.712 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-01-31T05:40:42.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.714+0000 7f473cb1f640 1 -- 172.21.15.45:0/2325534167 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f47240a1450 msgr2=0x7f47240a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.714+0000 7f473cb1f640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f47240a1450 0x7f47240a3910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4720000c00 tx=0x7f4720005fd0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.714+0000 7f473cb1f640 1 -- 172.21.15.45:0/2325534167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47380ffdd0 msgr2=0x7f47381980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.714+0000 7f473cb1f640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47380ffdd0 0x7f47381980a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f472800e9f0 tx=0x7f472800eeb0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.714+0000 7f473cb1f640 1 -- 172.21.15.45:0/2325534167 shutdown_connections 2024-01-31T05:40:42.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.714+0000 7f473cb1f640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f47240a1450 0x7f47240a3910 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.714+0000 7f473cb1f640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4738198b70 0x7f47381a3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.714+0000 7f473cb1f640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4738104ea0 0x7f47381985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.714+0000 7f473cb1f640 1 --2- 172.21.15.45:0/2325534167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47380ffdd0 0x7f47381980a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.714+0000 7f473cb1f640 1 -- 172.21.15.45:0/2325534167 >> 172.21.15.45:0/2325534167 conn(0x7f4738076cf0 msgr2=0x7f47380fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:42.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.714+0000 7f473cb1f640 1 -- 172.21.15.45:0/2325534167 shutdown_connections 2024-01-31T05:40:42.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.714+0000 7f473cb1f640 1 -- 172.21.15.45:0/2325534167 wait complete. 2024-01-31T05:40:42.725 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:40:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.831+0000 7f8f1afee640 1 -- 172.21.15.45:0/1969262951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f14069610 msgr2=0x7f8f140699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.831+0000 7f8f1afee640 1 --2- 172.21.15.45:0/1969262951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f14069610 0x7f8f140699f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009cd0 tx=0x7f8f0c02c0f0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.831+0000 7f8f1afee640 1 -- 172.21.15.45:0/1969262951 shutdown_connections 2024-01-31T05:40:42.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.831+0000 7f8f1afee640 1 --2- 172.21.15.45:0/1969262951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f14106050 0x7f8f1410dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.831+0000 7f8f1afee640 1 --2- 172.21.15.45:0/1969262951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f14069fc0 0x7f8f14105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.831+0000 7f8f1afee640 1 --2- 172.21.15.45:0/1969262951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f14069610 0x7f8f140699f0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.831+0000 7f8f1afee640 1 -- 172.21.15.45:0/1969262951 >> 172.21.15.45:0/1969262951 conn(0x7f8f14076cc0 msgr2=0x7f8f140770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:42.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.831+0000 7f8f1afee640 1 -- 172.21.15.45:0/1969262951 shutdown_connections 2024-01-31T05:40:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.831+0000 7f8f1afee640 1 -- 172.21.15.45:0/1969262951 wait complete. 2024-01-31T05:40:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1afee640 1 Processor -- start 2024-01-31T05:40:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1afee640 1 -- start start 2024-01-31T05:40:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1afee640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f14069610 0x7f8f14197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1afee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f14106050 0x7f8f14197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1afee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f1419baf0 0x7f8f1419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1afee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1419c4e0 con 0x7f8f1419baf0 2024-01-31T05:40:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1afee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1419c650 con 0x7f8f14106050 2024-01-31T05:40:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1afee640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1419c7c0 con 0x7f8f14069610 2024-01-31T05:40:42.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1958b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f1419baf0 0x7f8f1419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:42.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1958b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f1419baf0 0x7f8f1419bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37040/0 (socket says 172.21.15.45:37040) 2024-01-31T05:40:42.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1958b640 1 -- 172.21.15.45:0/2240824025 learned_addr learned my addr 172.21.15.45:0/2240824025 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:42.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f13fff640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f14106050 0x7f8f14197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:42.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f18d8a640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f14069610 0x7f8f14197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:42.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1958b640 1 -- 172.21.15.45:0/2240824025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f14069610 msgr2=0x7f8f14197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1958b640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f14069610 0x7f8f14197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1958b640 1 -- 172.21.15.45:0/2240824025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f14106050 msgr2=0x7f8f14197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:42.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1958b640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f14106050 0x7f8f14197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:42.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1958b640 1 -- 172.21.15.45:0/2240824025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c009900 con 0x7f8f1419baf0 2024-01-31T05:40:42.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f18d8a640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f14069610 0x7f8f14197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:40:42.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.832+0000 7f8f1958b640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f1419baf0 0x7f8f1419bfa0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00d950 tx=0x7f8efc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:42.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.833+0000 7f8f11ffb640 1 -- 172.21.15.45:0/2240824025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc009a30 con 0x7f8f1419baf0 2024-01-31T05:40:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.833+0000 7f8f11ffb640 1 -- 172.21.15.45:0/2240824025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8efc009b90 con 0x7f8f1419baf0 2024-01-31T05:40:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.833+0000 7f8f1afee640 1 -- 172.21.15.45:0/2240824025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f141a3da0 con 0x7f8f1419baf0 2024-01-31T05:40:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.833+0000 7f8f11ffb640 1 -- 172.21.15.45:0/2240824025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc003c00 con 0x7f8f1419baf0 2024-01-31T05:40:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.833+0000 7f8f1afee640 1 -- 172.21.15.45:0/2240824025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f141a4200 con 0x7f8f1419baf0 2024-01-31T05:40:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.834+0000 7f8f1afee640 1 -- 172.21.15.45:0/2240824025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f141023e0 con 0x7f8f1419baf0 2024-01-31T05:40:42.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.835+0000 7f8f11ffb640 1 -- 172.21.15.45:0/2240824025 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f8efc009d00 con 0x7f8f1419baf0 2024-01-31T05:40:42.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.835+0000 7f8f11ffb640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8f000a1450 0x7f8f000a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:42.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.835+0000 7f8f18d8a640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8f000a1450 0x7f8f000a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:42.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.835+0000 7f8f11ffb640 1 -- 172.21.15.45:0/2240824025 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8efc0ccb20 con 0x7f8f1419baf0 2024-01-31T05:40:42.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.836+0000 7f8f18d8a640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8f000a1450 0x7f8f000a3910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c006010 tx=0x7f8f0c005c50 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:42.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:42.840+0000 7f8f11ffb640 1 -- 172.21.15.45:0/2240824025 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8efc095c60 con 0x7f8f1419baf0 2024-01-31T05:40:43.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.019+0000 7f8f1afee640 1 -- 172.21.15.45:0/2240824025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f141092a0 con 0x7f8f1419baf0 2024-01-31T05:40:43.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.019+0000 7f8f11ffb640 1 -- 172.21.15.45:0/2240824025 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f141092a0 con 0x7f8f1419baf0 2024-01-31T05:40:43.020 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-01-31T05:40:43.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.021+0000 7f8f1afee640 1 -- 172.21.15.45:0/2240824025 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8f000a1450 msgr2=0x7f8f000a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:43.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.021+0000 7f8f1afee640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8f000a1450 0x7f8f000a3910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c006010 tx=0x7f8f0c005c50 comp rx=0 tx=0).stop 2024-01-31T05:40:43.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.021+0000 7f8f1afee640 1 -- 172.21.15.45:0/2240824025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f1419baf0 msgr2=0x7f8f1419bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:43.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.021+0000 7f8f1afee640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f1419baf0 0x7f8f1419bfa0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00d950 tx=0x7f8efc00de10 comp rx=0 tx=0).stop 2024-01-31T05:40:43.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.021+0000 7f8f1afee640 1 -- 172.21.15.45:0/2240824025 shutdown_connections 2024-01-31T05:40:43.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.021+0000 7f8f1afee640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8f000a1450 0x7f8f000a3910 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.021+0000 7f8f1afee640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f1419baf0 0x7f8f1419bfa0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.021+0000 7f8f1afee640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f14106050 0x7f8f14197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.021+0000 7f8f1afee640 1 --2- 172.21.15.45:0/2240824025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f14069610 0x7f8f14197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.021+0000 7f8f1afee640 1 -- 172.21.15.45:0/2240824025 >> 172.21.15.45:0/2240824025 conn(0x7f8f14076cc0 msgr2=0x7f8f140fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:43.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.022+0000 7f8f1afee640 1 -- 172.21.15.45:0/2240824025 shutdown_connections 2024-01-31T05:40:43.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.022+0000 7f8f1afee640 1 -- 172.21.15.45:0/2240824025 wait complete. 2024-01-31T05:40:43.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:42 smithi045 ceph-mon[31585]: pgmap v573: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 2.8 MiB/s wr, 108 op/s 2024-01-31T05:40:43.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:42 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/3676220998' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:40:43.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:42 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1050205891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:40:43.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:42 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2325534167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:40:43.033 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:40:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.136+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/1782793816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c34106070 msgr2=0x7f7c3410dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.136+0000 7f7c3c3f1640 1 --2- 172.21.15.45:0/1782793816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c34106070 0x7f7c3410dbf0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009cd0 tx=0x7f7c3002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.136+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/1782793816 shutdown_connections 2024-01-31T05:40:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.136+0000 7f7c3c3f1640 1 --2- 172.21.15.45:0/1782793816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c34106070 0x7f7c3410dbf0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.136+0000 7f7c3c3f1640 1 --2- 172.21.15.45:0/1782793816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c340ffe10 0x7f7c34105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.136+0000 7f7c3c3f1640 1 --2- 172.21.15.45:0/1782793816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c340ff460 0x7f7c340ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.136+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/1782793816 >> 172.21.15.45:0/1782793816 conn(0x7f7c340761b0 msgr2=0x7f7c340fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.136+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/1782793816 shutdown_connections 2024-01-31T05:40:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.136+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/1782793816 wait complete. 2024-01-31T05:40:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3c3f1640 1 Processor -- start 2024-01-31T05:40:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3c3f1640 1 -- start start 2024-01-31T05:40:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3c3f1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c340ff460 0x7f7c341971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3c3f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c34106070 0x7f7c34197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3c3f1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c3419ba50 0x7f7c3419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3c3f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3419c440 con 0x7f7c34106070 2024-01-31T05:40:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3c3f1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3419c5b0 con 0x7f7c340ff460 2024-01-31T05:40:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3c3f1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3419c720 con 0x7f7c3419ba50 2024-01-31T05:40:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3998c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c34106070 0x7f7c34197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3998c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c34106070 0x7f7c34197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37050/0 (socket says 172.21.15.45:37050) 2024-01-31T05:40:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3998c640 1 -- 172.21.15.45:0/932725677 learned_addr learned my addr 172.21.15.45:0/932725677 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3a98e640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c3419ba50 0x7f7c3419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3a18d640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c340ff460 0x7f7c341971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3998c640 1 -- 172.21.15.45:0/932725677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c3419ba50 msgr2=0x7f7c3419bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3998c640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c3419ba50 0x7f7c3419bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3998c640 1 -- 172.21.15.45:0/932725677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c340ff460 msgr2=0x7f7c341971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3998c640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c340ff460 0x7f7c341971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3998c640 1 -- 172.21.15.45:0/932725677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30009900 con 0x7f7c34106070 2024-01-31T05:40:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3a18d640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c340ff460 0x7f7c341971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:40:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.137+0000 7f7c3998c640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c34106070 0x7f7c34197720 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400c8c0 tx=0x7f7c2400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.138+0000 7f7c2b7fe640 1 -- 172.21.15.45:0/932725677 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24007bb0 con 0x7f7c34106070 2024-01-31T05:40:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.138+0000 7f7c2b7fe640 1 -- 172.21.15.45:0/932725677 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c24007d10 con 0x7f7c34106070 2024-01-31T05:40:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.138+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/932725677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c341a3d00 con 0x7f7c34106070 2024-01-31T05:40:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.138+0000 7f7c2b7fe640 1 -- 172.21.15.45:0/932725677 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24018710 con 0x7f7c34106070 2024-01-31T05:40:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.138+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/932725677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c341a41b0 con 0x7f7c34106070 2024-01-31T05:40:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.139+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/932725677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c34067f30 con 0x7f7c34106070 2024-01-31T05:40:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.140+0000 7f7c2b7fe640 1 -- 172.21.15.45:0/932725677 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f7c2400f450 con 0x7f7c34106070 2024-01-31T05:40:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.140+0000 7f7c2b7fe640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7c200a1450 0x7f7c200a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.140+0000 7f7c3a18d640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7c200a1450 0x7f7c200a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.140+0000 7f7c2b7fe640 1 -- 172.21.15.45:0/932725677 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7c240cd0f0 con 0x7f7c34106070 2024-01-31T05:40:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.141+0000 7f7c3a18d640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7c200a1450 0x7f7c200a3910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0058c0 tx=0x7f7c2c005d30 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.145+0000 7f7c2b7fe640 1 -- 172.21.15.45:0/932725677 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7c24096230 con 0x7f7c34106070 2024-01-31T05:40:43.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:42 smithi156 ceph-mon[35720]: pgmap v573: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 2.8 MiB/s wr, 108 op/s 2024-01-31T05:40:43.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:42 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/3676220998' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:40:43.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:42 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1050205891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:40:43.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:42 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2325534167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:40:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.334+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/932725677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7c341984e0 con 0x7f7c34106070 2024-01-31T05:40:43.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.335+0000 7f7c2b7fe640 1 -- 172.21.15.45:0/932725677 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v119) v1 ==== 81+0+1001 (secure 0 0 0) 0x7f7c24099850 con 0x7f7c34106070 2024-01-31T05:40:43.335 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1441130788 2024-01-31T06:27:35.007948+0000 2024-01-31T05:40:43.335 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3852874833 2024-02-01T05:13:46.510704+0000 2024-01-31T05:40:43.335 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/2712881513 2024-02-01T05:13:20.248709+0000 2024-01-31T05:40:43.336 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1162729130 2024-02-01T05:13:20.248709+0000 2024-01-31T05:40:43.336 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3020673835 2024-02-01T05:13:20.248709+0000 2024-01-31T05:40:43.336 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/4050317147 2024-02-01T05:13:46.510704+0000 2024-01-31T05:40:43.336 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/4050317147 2024-02-01T05:13:46.510704+0000 2024-01-31T05:40:43.336 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/782292734 2024-02-01T05:22:41.447737+0000 2024-01-31T05:40:43.336 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/745566822 2024-02-01T05:13:46.510704+0000 2024-01-31T05:40:43.337 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3329080521 2024-02-01T05:13:46.510704+0000 2024-01-31T05:40:43.337 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2742663751 2024-02-01T05:22:41.447737+0000 2024-01-31T05:40:43.337 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1829415949 2024-02-01T05:22:41.447737+0000 2024-01-31T05:40:43.337 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/2712881513 2024-02-01T05:13:20.248709+0000 2024-01-31T05:40:43.337 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2019451955 2024-02-01T05:13:20.248709+0000 2024-01-31T05:40:43.337 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/2788245525 2024-02-01T05:22:41.447737+0000 2024-01-31T05:40:43.338 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/948304328 2024-02-01T05:22:41.447737+0000 2024-01-31T05:40:43.338 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/2788245525 2024-02-01T05:22:41.447737+0000 2024-01-31T05:40:43.339 INFO:teuthology.orchestra.run.smithi045.stderr:listed 17 entries 2024-01-31T05:40:43.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.337+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/932725677 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7c200a1450 msgr2=0x7f7c200a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.337+0000 7f7c3c3f1640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7c200a1450 0x7f7c200a3910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0058c0 tx=0x7f7c2c005d30 comp rx=0 tx=0).stop 2024-01-31T05:40:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.337+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/932725677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c34106070 msgr2=0x7f7c34197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.337+0000 7f7c3c3f1640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c34106070 0x7f7c34197720 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400c8c0 tx=0x7f7c2400cd80 comp rx=0 tx=0).stop 2024-01-31T05:40:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.337+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/932725677 shutdown_connections 2024-01-31T05:40:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.337+0000 7f7c3c3f1640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f7c200a1450 0x7f7c200a3910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.337+0000 7f7c3c3f1640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c34106070 0x7f7c34197720 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.337+0000 7f7c3c3f1640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c3419ba50 0x7f7c3419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.337+0000 7f7c3c3f1640 1 --2- 172.21.15.45:0/932725677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c340ff460 0x7f7c341971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.337+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/932725677 >> 172.21.15.45:0/932725677 conn(0x7f7c340761b0 msgr2=0x7f7c34104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.337+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/932725677 shutdown_connections 2024-01-31T05:40:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:43.337+0000 7f7c3c3f1640 1 -- 172.21.15.45:0/932725677 wait complete. 2024-01-31T05:40:43.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:42 smithi186 ceph-mon[35659]: pgmap v573: 97 pgs: 97 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 2.8 MiB/s wr, 108 op/s 2024-01-31T05:40:43.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:42 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/3676220998' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:40:43.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:42 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1050205891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-31T05:40:43.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:42 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2325534167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-31T05:40:43.348 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-31T05:40:43.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:40:43.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-31T05:40:43.813 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-01-31T05:40:43.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:40:43.814 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-31T05:40:43.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:40:43.874 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-01-31T05:40:43.909 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-31T05:40:43.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:40:43.910 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:40:43.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:40:43.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-01-31T05:40:44.063 INFO:tasks.kclient:Unmounting kernel clients... 2024-01-31T05:40:44.063 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-01-31T05:40:44.063 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:40:44.073 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:43 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2240824025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:40:44.073 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:43 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/932725677' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-31T05:40:44.080 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498940k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249468k,nr_inodes=812367,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/ecfdd65236a34b088a447a5df5402327e91770588ad83729fd60e61a5b155f5e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/6QPJLT6RRWAKRQQU3NULCIBNGD:/var/lib/containers/storage/overlay/l/C4ZACFB3Y4MJJ7HMRZJJB45CZL,upperdir=/var/lib/containers/storage/overlay/ecfdd65236a34b088a447a5df5402327e91770588ad83729fd60e61a5b155f5e/diff,workdir=/var/lib/containers/storage/overlay/ecfdd65236a34b088a447a5df5402327e91770588ad83729fd60e61a5b155f5e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5e32800eb2fbddf5f48eb20f00636d849d1b5a3b703dbfc3373faeb0eb493897/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/6QPJLT6RRWAKRQQU3NULCIBNGD:/var/lib/containers/storage/overlay/l/C4ZACFB3Y4MJJ7HMRZJJB45CZL,upperdir=/var/lib/containers/storage/overlay/5e32800eb2fbddf5f48eb20f00636d849d1b5a3b703dbfc3373faeb0eb493897/diff,workdir=/var/lib/containers/storage/overlay/5e32800eb2fbddf5f48eb20f00636d849d1b5a3b703dbfc3373faeb0eb493897/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249468k,nr_inodes=812367,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/5d6fa0250a19395ab49682e5784b493058258dc48eeb169fbe144c3a0debcf1d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/6QPJLT6RRWAKRQQU3NULCIBNGD:/var/lib/containers/storage/overlay/l/C4ZACFB3Y4MJJ7HMRZJJB45CZL,upperdir=/var/lib/containers/storage/overlay/5d6fa0250a19395ab49682e5784b493058258dc48eeb169fbe144c3a0debcf1d/diff,workdir=/var/lib/containers/storage/overlay/5d6fa0250a19395ab49682e5784b493058258dc48eeb169fbe144c3a0debcf1d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b3f1cefc011897c37787223f901a320b374dc21dbe2fbe7d251035107625485b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/6QPJLT6RRWAKRQQU3NULCIBNGD:/var/lib/containers/storage/overlay/l/C4ZACFB3Y4MJJ7HMRZJJB45CZL,upperdir=/var/lib/containers/storage/overlay/b3f1cefc011897c37787223f901a320b374dc21dbe2fbe7d251035107625485b/diff,workdir=/var/lib/containers/storage/overlay/b3f1cefc011897c37787223f901a320b374dc21dbe2fbe7d251035107625485b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0fdd1e5e7e50750975a6b262f9fa3123bfab8fab7af10fd1e78605452ca3317f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/6QPJLT6RRWAKRQQU3NULCIBNGD:/var/lib/containers/storage/overlay/l/C4ZACFB3Y4MJJ7HMRZJJB45CZL,upperdir=/var/lib/containers/storage/overlay/0fdd1e5e7e50750975a6b262f9fa3123bfab8fab7af10fd1e78605452ca3317f/diff,workdir=/var/lib/containers/storage/overlay/0fdd1e5e7e50750975a6b262f9fa3123bfab8fab7af10fd1e78605452ca3317f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/26a18f73e6986d480b158b19c6c9f644e7a1d039a02d69a8b5075bb7e0cabc7b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/6QPJLT6RRWAKRQQU3NULCIBNGD:/var/lib/containers/storage/overlay/l/C4ZACFB3Y4MJJ7HMRZJJB45CZL,upperdir=/var/lib/containers/storage/overlay/26a18f73e6986d480b158b19c6c9f644e7a1d039a02d69a8b5075bb7e0cabc7b/diff,workdir=/var/lib/containers/storage/overlay/26a18f73e6986d480b158b19c6c9f644e7a1d039a02d69a8b5075bb7e0cabc7b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d8be0f186b5f1ddb612f92563f146324d7c507e6e79f5464f60f9f0447630c43/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/6QPJLT6RRWAKRQQU3NULCIBNGD:/var/lib/containers/storage/overlay/l/C4ZACFB3Y4MJJ7HMRZJJB45CZL,upperdir=/var/lib/containers/storage/overlay/d8be0f186b5f1ddb612f92563f146324d7c507e6e79f5464f60f9f0447630c43/diff,workdir=/var/lib/containers/storage/overlay/d8be0f186b5f1ddb612f92563f146324d7c507e6e79f5464f60f9f0447630c43/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/10d1024e34cfef12145180b035ea4eef8345b6051f725b5d90d52ebe54b67edf/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/6QPJLT6RRWAKRQQU3NULCIBNGD:/var/lib/containers/storage/overlay/l/C4ZACFB3Y4MJJ7HMRZJJB45CZL,upperdir=/var/lib/containers/storage/overlay/10d1024e34cfef12145180b035ea4eef8345b6051f725b5d90d52ebe54b67edf/diff,workdir=/var/lib/containers/storage/overlay/10d1024e34cfef12145180b035ea4eef8345b6051f725b5d90d52ebe54b67edf/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/eb9e463ad230f18291d42d2e83e5278215b8f92088b23370c0bb401cc5ab950e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/6QPJLT6RRWAKRQQU3NULCIBNGD:/var/lib/containers/storage/overlay/l/C4ZACFB3Y4MJJ7HMRZJJB45CZL,upperdir=/var/lib/containers/storage/overlay/eb9e463ad230f18291d42d2e83e5278215b8f92088b23370c0bb401cc5ab950e/diff,workdir=/var/lib/containers/storage/overlay/eb9e463ad230f18291d42d2e83e5278215b8f92088b23370c0bb401cc5ab950e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ef49e4326b1a54961b540c86ae879b6b38c95c121bca15fd0c147cf4e4134daa/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/6QPJLT6RRWAKRQQU3NULCIBNGD:/var/lib/containers/storage/overlay/l/C4ZACFB3Y4MJJ7HMRZJJB45CZL,upperdir=/var/lib/containers/storage/overlay/ef49e4326b1a54961b540c86ae879b6b38c95c121bca15fd0c147cf4e4134daa/diff,workdir=/var/lib/containers/storage/overlay/ef49e4326b1a54961b540c86ae879b6b38c95c121bca15fd0c147cf4e4134daa/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498940k,nr_inodes=819200,mode=755,inode64 0 0 2024-01-31T05:40:44.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:40:44.081 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-01-31T05:40:44.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:40:44.137 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-01-31T05:40:44.218 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-01-31T05:40:44.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:40:44.221 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-01-31T05:40:44.231 INFO:tasks.cephadm:Teardown begin 2024-01-31T05:40:44.231 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-31T05:40:44.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:40:44.232 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:43 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2240824025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:40:44.232 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:43 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/932725677' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-31T05:40:44.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:43 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2240824025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-31T05:40:44.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:43 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/932725677' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-31T05:40:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.363+0000 7f8cc29a7640 1 -- 172.21.15.45:0/3718018699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cbc10b330 msgr2=0x7f8cbc10d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.363+0000 7f8cc29a7640 1 --2- 172.21.15.45:0/3718018699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cbc10b330 0x7f8cbc10d7d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009ca0 tx=0x7f8cb402c0f0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.363+0000 7f8cc29a7640 1 -- 172.21.15.45:0/3718018699 shutdown_connections 2024-01-31T05:40:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.363+0000 7f8cc29a7640 1 --2- 172.21.15.45:0/3718018699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cbc10b330 0x7f8cbc10d7d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.363+0000 7f8cc29a7640 1 --2- 172.21.15.45:0/3718018699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cbc102880 0x7f8cbc10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.363+0000 7f8cc29a7640 1 --2- 172.21.15.45:0/3718018699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cbc101f60 0x7f8cbc102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.363+0000 7f8cc29a7640 1 -- 172.21.15.45:0/3718018699 >> 172.21.15.45:0/3718018699 conn(0x7f8cbc076cf0 msgr2=0x7f8cbc077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.364+0000 7f8cc29a7640 1 -- 172.21.15.45:0/3718018699 shutdown_connections 2024-01-31T05:40:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.364+0000 7f8cc29a7640 1 -- 172.21.15.45:0/3718018699 wait complete. 2024-01-31T05:40:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.364+0000 7f8cc29a7640 1 Processor -- start 2024-01-31T05:40:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.364+0000 7f8cc29a7640 1 -- start start 2024-01-31T05:40:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.364+0000 7f8cc29a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cbc101f60 0x7f8cbc1971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.364+0000 7f8cc29a7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cbc102880 0x7f8cbc197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:44.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.364+0000 7f8cc29a7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cbc10b330 0x7f8cbc19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:44.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.364+0000 7f8cc29a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc19c0b0 con 0x7f8cbc101f60 2024-01-31T05:40:44.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.364+0000 7f8cc29a7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc19c220 con 0x7f8cbc10b330 2024-01-31T05:40:44.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.364+0000 7f8cc29a7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc19c3e0 con 0x7f8cbc102880 2024-01-31T05:40:44.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cbbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cbc101f60 0x7f8cbc1971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:44.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cbbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cbc101f60 0x7f8cbc1971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37080/0 (socket says 172.21.15.45:37080) 2024-01-31T05:40:44.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cbbfff640 1 -- 172.21.15.45:0/4214720079 learned_addr learned my addr 172.21.15.45:0/4214720079 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cc0f44640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cbc10b330 0x7f8cbc19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cbb7fe640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cbc102880 0x7f8cbc197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cbbfff640 1 -- 172.21.15.45:0/4214720079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cbc102880 msgr2=0x7f8cbc197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cbbfff640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cbc102880 0x7f8cbc197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cbbfff640 1 -- 172.21.15.45:0/4214720079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cbc10b330 msgr2=0x7f8cbc19bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cbbfff640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cbc10b330 0x7f8cbc19bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cbbfff640 1 -- 172.21.15.45:0/4214720079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb4009900 con 0x7f8cbc101f60 2024-01-31T05:40:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cbbfff640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cbc101f60 0x7f8cbc1971f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000d950 tx=0x7f8cb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cb97fa640 1 -- 172.21.15.45:0/4214720079 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0009c30 con 0x7f8cbc101f60 2024-01-31T05:40:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cb97fa640 1 -- 172.21.15.45:0/4214720079 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cb0009d90 con 0x7f8cbc101f60 2024-01-31T05:40:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cc29a7640 1 -- 172.21.15.45:0/4214720079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc19c6c0 con 0x7f8cbc101f60 2024-01-31T05:40:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cb97fa640 1 -- 172.21.15.45:0/4214720079 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb000f740 con 0x7f8cbc101f60 2024-01-31T05:40:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.365+0000 7f8cc29a7640 1 -- 172.21.15.45:0/4214720079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc1a3f40 con 0x7f8cbc101f60 2024-01-31T05:40:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.366+0000 7f8cc29a7640 1 -- 172.21.15.45:0/4214720079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cbc067ae0 con 0x7f8cbc101f60 2024-01-31T05:40:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.367+0000 7f8cb97fa640 1 -- 172.21.15.45:0/4214720079 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f8cb0010a20 con 0x7f8cbc101f60 2024-01-31T05:40:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.368+0000 7f8cb97fa640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8ca40a1450 0x7f8ca40a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.368+0000 7f8cb97fa640 1 -- 172.21.15.45:0/4214720079 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8cb00ccb20 con 0x7f8cbc101f60 2024-01-31T05:40:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.368+0000 7f8cbb7fe640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8ca40a1450 0x7f8ca40a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.368+0000 7f8cbb7fe640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8ca40a1450 0x7f8ca40a3910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc10d090 tx=0x7f8cac005f90 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.372+0000 7f8cb97fa640 1 -- 172.21.15.45:0/4214720079 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8cb0095d10 con 0x7f8cbc101f60 2024-01-31T05:40:44.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.590+0000 7f8cc29a7640 1 -- 172.21.15.45:0/4214720079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cbc106420 con 0x7f8cbc101f60 2024-01-31T05:40:44.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.590+0000 7f8cb97fa640 1 -- 172.21.15.45:0/4214720079 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10552 (secure 0 0 0) 0x7f8cb0099330 con 0x7f8cbc101f60 2024-01-31T05:40:44.590 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:40:44.591 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1801709710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1801709710},{"type":"v1","addr":"172.21.15.45:6839","nonce":1801709710}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/1148586810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1148586810},{"type":"v1","addr":"172.21.15.45:6841","nonce":1148586810}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-31T05:20:40.057668+0000","modified":"2024-01-31T05:40:17.714051+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/3821255404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3821255404},{"type":"v1","addr":"172.21.15.186:6839","nonce":3821255404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/3137398384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3137398384},{"type":"v1","addr":"172.21.15.156:6839","nonce":3137398384}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:40:44.591 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 34 2024-01-31T05:40:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.592+0000 7f8cc29a7640 1 -- 172.21.15.45:0/4214720079 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8ca40a1450 msgr2=0x7f8ca40a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.592+0000 7f8cc29a7640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8ca40a1450 0x7f8ca40a3910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc10d090 tx=0x7f8cac005f90 comp rx=0 tx=0).stop 2024-01-31T05:40:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.592+0000 7f8cc29a7640 1 -- 172.21.15.45:0/4214720079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cbc101f60 msgr2=0x7f8cbc1971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.592+0000 7f8cc29a7640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cbc101f60 0x7f8cbc1971f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000d950 tx=0x7f8cb000de10 comp rx=0 tx=0).stop 2024-01-31T05:40:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.592+0000 7f8cc29a7640 1 -- 172.21.15.45:0/4214720079 shutdown_connections 2024-01-31T05:40:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.592+0000 7f8cc29a7640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f8ca40a1450 0x7f8ca40a3910 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.592+0000 7f8cc29a7640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cbc10b330 0x7f8cbc19bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.592+0000 7f8cc29a7640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cbc102880 0x7f8cbc197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.592+0000 7f8cc29a7640 1 --2- 172.21.15.45:0/4214720079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cbc101f60 0x7f8cbc1971f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.592+0000 7f8cc29a7640 1 -- 172.21.15.45:0/4214720079 >> 172.21.15.45:0/4214720079 conn(0x7f8cbc076cf0 msgr2=0x7f8cbc0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.592+0000 7f8cc29a7640 1 -- 172.21.15.45:0/4214720079 shutdown_connections 2024-01-31T05:40:44.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.592+0000 7f8cc29a7640 1 -- 172.21.15.45:0/4214720079 wait complete. 2024-01-31T05:40:44.603 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:40:44.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.730+0000 7fcca5742640 1 -- 172.21.15.45:0/1464905305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca010b330 msgr2=0x7fcca010d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:44.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.730+0000 7fcca5742640 1 --2- 172.21.15.45:0/1464905305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca010b330 0x7fcca010d7d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009ca0 tx=0x7fcc9002c0f0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.731+0000 7fcca5742640 1 -- 172.21.15.45:0/1464905305 shutdown_connections 2024-01-31T05:40:44.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.731+0000 7fcca5742640 1 --2- 172.21.15.45:0/1464905305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca010b330 0x7fcca010d7d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.731+0000 7fcca5742640 1 --2- 172.21.15.45:0/1464905305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcca0102880 0x7fcca010ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.731+0000 7fcca5742640 1 --2- 172.21.15.45:0/1464905305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcca0101f60 0x7fcca0102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.731+0000 7fcca5742640 1 -- 172.21.15.45:0/1464905305 >> 172.21.15.45:0/1464905305 conn(0x7fcca0076cf0 msgr2=0x7fcca0077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.731+0000 7fcca5742640 1 -- 172.21.15.45:0/1464905305 shutdown_connections 2024-01-31T05:40:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.731+0000 7fcca5742640 1 -- 172.21.15.45:0/1464905305 wait complete. 2024-01-31T05:40:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.731+0000 7fcca5742640 1 Processor -- start 2024-01-31T05:40:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.731+0000 7fcca5742640 1 -- start start 2024-01-31T05:40:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.731+0000 7fcca5742640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcca0101f60 0x7fcca0197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcca5742640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca0102880 0x7fcca01977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcca5742640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcca010b330 0x7fcca019bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcca5742640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca019c0e0 con 0x7fcca0102880 2024-01-31T05:40:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcca5742640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca019c250 con 0x7fcca0101f60 2024-01-31T05:40:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcca5742640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca019c410 con 0x7fcca010b330 2024-01-31T05:40:44.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca0102880 0x7fcca01977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:44.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcca0101f60 0x7fcca0197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:44.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca0102880 0x7fcca01977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37096/0 (socket says 172.21.15.45:37096) 2024-01-31T05:40:44.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcca0101f60 0x7fcca0197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.45:55990/0 (socket says 172.21.15.45:55990) 2024-01-31T05:40:44.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9e7fc640 1 -- 172.21.15.45:0/1461365136 learned_addr learned my addr 172.21.15.45:0/1461365136 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:44.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9f7fe640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcca010b330 0x7fcca019bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:44.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9e7fc640 1 -- 172.21.15.45:0/1461365136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcca010b330 msgr2=0x7fcca019bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:44.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9e7fc640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcca010b330 0x7fcca019bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9e7fc640 1 -- 172.21.15.45:0/1461365136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcca0101f60 msgr2=0x7fcca0197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:44.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9e7fc640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcca0101f60 0x7fcca0197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9e7fc640 1 -- 172.21.15.45:0/1461365136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc90009900 con 0x7fcca0102880 2024-01-31T05:40:44.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9effd640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcca0101f60 0x7fcca0197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:40:44.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc9e7fc640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca0102880 0x7fcca01977b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400e9f0 tx=0x7fcc9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:44.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc87fff640 1 -- 172.21.15.45:0/1461365136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94009700 con 0x7fcca0102880 2024-01-31T05:40:44.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc87fff640 1 -- 172.21.15.45:0/1461365136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc9400cd80 con 0x7fcca0102880 2024-01-31T05:40:44.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcca5742640 1 -- 172.21.15.45:0/1461365136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca019c6f0 con 0x7fcca0102880 2024-01-31T05:40:44.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcc87fff640 1 -- 172.21.15.45:0/1461365136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94018860 con 0x7fcca0102880 2024-01-31T05:40:44.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.732+0000 7fcca5742640 1 -- 172.21.15.45:0/1461365136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca01a3f70 con 0x7fcca0102880 2024-01-31T05:40:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.733+0000 7fcca5742640 1 -- 172.21.15.45:0/1461365136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca0067ae0 con 0x7fcca0102880 2024-01-31T05:40:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.734+0000 7fcc87fff640 1 -- 172.21.15.45:0/1461365136 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fcc940189c0 con 0x7fcca0102880 2024-01-31T05:40:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.735+0000 7fcc87fff640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fcc880a1450 0x7fcc880a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.735+0000 7fcc9effd640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fcc880a1450 0x7fcc880a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.735+0000 7fcc87fff640 1 -- 172.21.15.45:0/1461365136 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fcc94014070 con 0x7fcca0102880 2024-01-31T05:40:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.735+0000 7fcc9effd640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fcc880a1450 0x7fcc880a3910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009fc0 tx=0x7fcc98005f90 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.739+0000 7fcc87fff640 1 -- 172.21.15.45:0/1461365136 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fcc94052600 con 0x7fcca0102880 2024-01-31T05:40:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.958+0000 7fcca5742640 1 -- 172.21.15.45:0/1461365136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcca0106420 con 0x7fcca0102880 2024-01-31T05:40:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.959+0000 7fcc87fff640 1 -- 172.21.15.45:0/1461365136 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10552 (secure 0 0 0) 0x7fcc94096420 con 0x7fcca0102880 2024-01-31T05:40:44.959 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:40:44.960 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/363421179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":363421179},{"type":"v1","addr":"172.21.15.45:6835","nonce":363421179}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/176161833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":176161833},{"type":"v1","addr":"172.21.15.45:6837","nonce":176161833}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1801709710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1801709710},{"type":"v1","addr":"172.21.15.45:6839","nonce":1801709710}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/1148586810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1148586810},{"type":"v1","addr":"172.21.15.45:6841","nonce":1148586810}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2627326005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2627326005},{"type":"v1","addr":"172.21.15.156:6833","nonce":2627326005}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-31T05:20:40.057668+0000","modified":"2024-01-31T05:40:17.714051+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/3707036230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3707036230},{"type":"v1","addr":"172.21.15.186:6833","nonce":3707036230}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4045803513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4045803513},{"type":"v1","addr":"172.21.15.156:6835","nonce":4045803513}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/696958495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":696958495},{"type":"v1","addr":"172.21.15.186:6835","nonce":696958495}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6837/2559285393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2559285393},{"type":"v1","addr":"172.21.15.186:6837","nonce":2559285393}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":5,"addr":"172.21.15.186:6839/3821255404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3821255404},{"type":"v1","addr":"172.21.15.186:6839","nonce":3821255404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/3137398384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3137398384},{"type":"v1","addr":"172.21.15.156:6839","nonce":3137398384}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:40:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 34 2024-01-31T05:40:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.961+0000 7fcca5742640 1 -- 172.21.15.45:0/1461365136 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fcc880a1450 msgr2=0x7fcc880a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.961+0000 7fcca5742640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fcc880a1450 0x7fcc880a3910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009fc0 tx=0x7fcc98005f90 comp rx=0 tx=0).stop 2024-01-31T05:40:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.961+0000 7fcca5742640 1 -- 172.21.15.45:0/1461365136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca0102880 msgr2=0x7fcca01977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.961+0000 7fcca5742640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca0102880 0x7fcca01977b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400e9f0 tx=0x7fcc9400eeb0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.961+0000 7fcca5742640 1 -- 172.21.15.45:0/1461365136 shutdown_connections 2024-01-31T05:40:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.961+0000 7fcca5742640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7fcc880a1450 0x7fcc880a3910 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.961+0000 7fcca5742640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca0102880 0x7fcca01977b0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.961+0000 7fcca5742640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcca010b330 0x7fcca019bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.961+0000 7fcca5742640 1 --2- 172.21.15.45:0/1461365136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcca0101f60 0x7fcca0197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.961+0000 7fcca5742640 1 -- 172.21.15.45:0/1461365136 >> 172.21.15.45:0/1461365136 conn(0x7fcca0076cf0 msgr2=0x7fcca00fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.961+0000 7fcca5742640 1 -- 172.21.15.45:0/1461365136 shutdown_connections 2024-01-31T05:40:44.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:44.961+0000 7fcca5742640 1 -- 172.21.15.45:0/1461365136 wait complete. 2024-01-31T05:40:44.972 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:40:45.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.076+0000 7f26111c6640 1 -- 172.21.15.45:0/2731439680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260c0ffe20 msgr2=0x7f260c100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.076+0000 7f26111c6640 1 --2- 172.21.15.45:0/2731439680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260c0ffe20 0x7f260c100200 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009930 tx=0x7f25fc02bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.076+0000 7f26111c6640 1 -- 172.21.15.45:0/2731439680 shutdown_connections 2024-01-31T05:40:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.076+0000 7f26111c6640 1 --2- 172.21.15.45:0/2731439680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260c104ef0 0x7f260c10cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.076+0000 7f26111c6640 1 --2- 172.21.15.45:0/2731439680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f260c1007d0 0x7f260c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.076+0000 7f26111c6640 1 --2- 172.21.15.45:0/2731439680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260c0ffe20 0x7f260c100200 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.076+0000 7f26111c6640 1 -- 172.21.15.45:0/2731439680 >> 172.21.15.45:0/2731439680 conn(0x7f260c076c90 msgr2=0x7f260c0770a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.076+0000 7f26111c6640 1 -- 172.21.15.45:0/2731439680 shutdown_connections 2024-01-31T05:40:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.076+0000 7f26111c6640 1 -- 172.21.15.45:0/2731439680 wait complete. 2024-01-31T05:40:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.077+0000 7f26111c6640 1 Processor -- start 2024-01-31T05:40:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.077+0000 7f26111c6640 1 -- start start 2024-01-31T05:40:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.077+0000 7f26111c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260c0ffe20 0x7f260c1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.077+0000 7f26111c6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f260c104ef0 0x7f260c197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.077+0000 7f26111c6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260c19bb90 0x7f260c19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.077+0000 7f26111c6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c19c580 con 0x7f260c0ffe20 2024-01-31T05:40:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.077+0000 7f26111c6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c19c6f0 con 0x7f260c104ef0 2024-01-31T05:40:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.077+0000 7f26111c6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c19c860 con 0x7f260c19bb90 2024-01-31T05:40:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.077+0000 7f260ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260c0ffe20 0x7f260c1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.077+0000 7f260ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260c0ffe20 0x7f260c1972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37120/0 (socket says 172.21.15.45:37120) 2024-01-31T05:40:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.077+0000 7f260ad9d640 1 -- 172.21.15.45:0/782821289 learned_addr learned my addr 172.21.15.45:0/782821289 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f2603fff640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f260c104ef0 0x7f260c197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f260b59e640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260c19bb90 0x7f260c19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f260ad9d640 1 -- 172.21.15.45:0/782821289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260c19bb90 msgr2=0x7f260c19c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f260ad9d640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260c19bb90 0x7f260c19c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f260ad9d640 1 -- 172.21.15.45:0/782821289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f260c104ef0 msgr2=0x7f260c197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f260ad9d640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f260c104ef0 0x7f260c197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f260ad9d640 1 -- 172.21.15.45:0/782821289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc009560 con 0x7f260c0ffe20 2024-01-31T05:40:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f260b59e640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260c19bb90 0x7f260c19c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:40:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f260ad9d640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260c0ffe20 0x7f260c1972d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f25fc005940 tx=0x7f25fc004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f2608d99640 1 -- 172.21.15.45:0/782821289 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc0042d0 con 0x7f260c0ffe20 2024-01-31T05:40:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f2608d99640 1 -- 172.21.15.45:0/782821289 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25fc004430 con 0x7f260c0ffe20 2024-01-31T05:40:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f26111c6640 1 -- 172.21.15.45:0/782821289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260c1a3e40 con 0x7f260c0ffe20 2024-01-31T05:40:45.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f2608d99640 1 -- 172.21.15.45:0/782821289 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc003cc0 con 0x7f260c0ffe20 2024-01-31T05:40:45.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.078+0000 7f26111c6640 1 -- 172.21.15.45:0/782821289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260c1a4260 con 0x7f260c0ffe20 2024-01-31T05:40:45.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.079+0000 7f26111c6640 1 -- 172.21.15.45:0/782821289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260c1012f0 con 0x7f260c0ffe20 2024-01-31T05:40:45.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.080+0000 7f2608d99640 1 -- 172.21.15.45:0/782821289 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f25fc005000 con 0x7f260c0ffe20 2024-01-31T05:40:45.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.081+0000 7f2608d99640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f25f80a1450 0x7f25f80a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:45.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.081+0000 7f2603fff640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f25f80a1450 0x7f25f80a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:45.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.081+0000 7f2608d99640 1 -- 172.21.15.45:0/782821289 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f25fc0ef770 con 0x7f260c0ffe20 2024-01-31T05:40:45.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.081+0000 7f2603fff640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f25f80a1450 0x7f25f80a3910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f260c0ffab0 tx=0x7f25f400b410 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:45.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.085+0000 7f2608d99640 1 -- 172.21.15.45:0/782821289 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f25fc0b8960 con 0x7f260c0ffe20 2024-01-31T05:40:45.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:44 smithi156 ceph-mon[35720]: pgmap v574: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 4.0 MiB/s wr, 154 op/s 2024-01-31T05:40:45.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:44 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/4214720079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:40:45.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.267+0000 7f26111c6640 1 -- 172.21.15.45:0/782821289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f260c1007d0 con 0x7f260c0ffe20 2024-01-31T05:40:45.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.269+0000 7f2608d99640 1 -- 172.21.15.45:0/782821289 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+23052 (secure 0 0 0) 0x7f25fc0bbf80 con 0x7f260c0ffe20 2024-01-31T05:40:45.269 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-01-31T05:40:45.270 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":119,"fsid":"3a1f4122-bff7-11ee-95b3-87774f69a715","created":"2024-01-31T05:12:51.341846+0000","modified":"2024-01-31T05:40:12.712633+0000","last_up_change":"2024-01-31T05:19:49.727812+0000","last_in_change":"2024-01-31T05:19:29.863201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:16:10.856581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:20:38.347599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:20:39.270522+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":118,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5b06f7af-6fee-4838-afe2-f178362f0a88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6803","nonce":4026202784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6805","nonce":4026202784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6809","nonce":4026202784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4026202784},{"type":"v1","addr":"172.21.15.45:6807","nonce":4026202784}]},"public_addr":"172.21.15.45:6803/4026202784","cluster_addr":"172.21.15.45:6805/4026202784","heartbeat_back_addr":"172.21.15.45:6809/4026202784","heartbeat_front_addr":"172.21.15.45:6807/4026202784","state":["exists","up"]},{"osd":1,"uuid":"2ae3ab3f-342a-4e5a-93a2-a674831b30bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6801","nonce":2623092514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6803","nonce":2623092514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6807","nonce":2623092514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2623092514},{"type":"v1","addr":"172.21.15.156:6805","nonce":2623092514}]},"public_addr":"172.21.15.156:6801/2623092514","cluster_addr":"172.21.15.156:6803/2623092514","heartbeat_back_addr":"172.21.15.156:6807/2623092514","heartbeat_front_addr":"172.21.15.156:6805/2623092514","state":["exists","up"]},{"osd":2,"uuid":"6c8acbd3-748d-4107-85e4-766d8820c5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6801","nonce":811586396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6803","nonce":811586396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6807","nonce":811586396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":811586396},{"type":"v1","addr":"172.21.15.186:6805","nonce":811586396}]},"public_addr":"172.21.15.186:6801/811586396","cluster_addr":"172.21.15.186:6803/811586396","heartbeat_back_addr":"172.21.15.186:6807/811586396","heartbeat_front_addr":"172.21.15.186:6805/811586396","state":["exists","up"]},{"osd":3,"uuid":"d36dc9a9-b8cb-4e69-8ebc-a9e0ffa29d4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6811","nonce":3653915645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6813","nonce":3653915645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6817","nonce":3653915645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3653915645},{"type":"v1","addr":"172.21.15.45:6815","nonce":3653915645}]},"public_addr":"172.21.15.45:6811/3653915645","cluster_addr":"172.21.15.45:6813/3653915645","heartbeat_back_addr":"172.21.15.45:6817/3653915645","heartbeat_front_addr":"172.21.15.45:6815/3653915645","state":["exists","up"]},{"osd":4,"uuid":"4ae77293-4355-4aac-87a5-220b0754e21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6809","nonce":2291851752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6811","nonce":2291851752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6815","nonce":2291851752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2291851752},{"type":"v1","addr":"172.21.15.156:6813","nonce":2291851752}]},"public_addr":"172.21.15.156:6809/2291851752","cluster_addr":"172.21.15.156:6811/2291851752","heartbeat_back_addr":"172.21.15.156:6815/2291851752","heartbeat_front_addr":"172.21.15.156:6813/2291851752","state":["exists","up"]},{"osd":5,"uuid":"69641e46-7955-4b81-990e-bf656f6d7ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6809","nonce":2142758307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6811","nonce":2142758307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6815","nonce":2142758307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2142758307},{"type":"v1","addr":"172.21.15.186:6813","nonce":2142758307}]},"public_addr":"172.21.15.186:6809/2142758307","cluster_addr":"172.21.15.186:6811/2142758307","heartbeat_back_addr":"172.21.15.186:6815/2142758307","heartbeat_front_addr":"172.21.15.186:6813/2142758307","state":["exists","up"]},{"osd":6,"uuid":"9449230f-ba33-45ee-993a-12a376e2c9ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6819","nonce":2753791681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6821","nonce":2753791681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6825","nonce":2753791681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2753791681},{"type":"v1","addr":"172.21.15.45:6823","nonce":2753791681}]},"public_addr":"172.21.15.45:6819/2753791681","cluster_addr":"172.21.15.45:6821/2753791681","heartbeat_back_addr":"172.21.15.45:6825/2753791681","heartbeat_front_addr":"172.21.15.45:6823/2753791681","state":["exists","up"]},{"osd":7,"uuid":"de8fa2cd-6e29-40b8-836f-90a64e4b09a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6817","nonce":2686361879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6819","nonce":2686361879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6823","nonce":2686361879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2686361879},{"type":"v1","addr":"172.21.15.156:6821","nonce":2686361879}]},"public_addr":"172.21.15.156:6817/2686361879","cluster_addr":"172.21.15.156:6819/2686361879","heartbeat_back_addr":"172.21.15.156:6823/2686361879","heartbeat_front_addr":"172.21.15.156:6821/2686361879","state":["exists","up"]},{"osd":8,"uuid":"023fed4e-b2c5-4931-9bc8-04461d8bd939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6817","nonce":2054139759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6819","nonce":2054139759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6823","nonce":2054139759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2054139759},{"type":"v1","addr":"172.21.15.186:6821","nonce":2054139759}]},"public_addr":"172.21.15.186:6817/2054139759","cluster_addr":"172.21.15.186:6819/2054139759","heartbeat_back_addr":"172.21.15.186:6823/2054139759","heartbeat_front_addr":"172.21.15.186:6821/2054139759","state":["exists","up"]},{"osd":9,"uuid":"bfaec4e4-25e7-40b8-a571-0b8a0bc40e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6827","nonce":717478931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6829","nonce":717478931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6833","nonce":717478931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":717478931},{"type":"v1","addr":"172.21.15.45:6831","nonce":717478931}]},"public_addr":"172.21.15.45:6827/717478931","cluster_addr":"172.21.15.45:6829/717478931","heartbeat_back_addr":"172.21.15.45:6833/717478931","heartbeat_front_addr":"172.21.15.45:6831/717478931","state":["exists","up"]},{"osd":10,"uuid":"2ed07ce9-24a6-40b1-9a8d-2b51af4d70cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6825","nonce":2036478526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6827","nonce":2036478526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6831","nonce":2036478526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2036478526},{"type":"v1","addr":"172.21.15.156:6829","nonce":2036478526}]},"public_addr":"172.21.15.156:6825/2036478526","cluster_addr":"172.21.15.156:6827/2036478526","heartbeat_back_addr":"172.21.15.156:6831/2036478526","heartbeat_front_addr":"172.21.15.156:6829/2036478526","state":["exists","up"]},{"osd":11,"uuid":"a26f7c8a-679e-43d2-b923-01f3358ffcc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6825","nonce":3877143910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6827","nonce":3877143910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6831","nonce":3877143910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3877143910},{"type":"v1","addr":"172.21.15.186:6829","nonce":3877143910}]},"public_addr":"172.21.15.186:6825/3877143910","cluster_addr":"172.21.15.186:6827/3877143910","heartbeat_back_addr":"172.21.15.186:6831/3877143910","heartbeat_front_addr":"172.21.15.186:6829/3877143910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:19.222800+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:15:43.333930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:07.549658+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:31.217823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:16:55.074318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:19.747816+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:17:43.874967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:08.256160+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:32.787850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:18:58.020167+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:22.789821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:19:47.907581+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/1441130788":"2024-01-31T06:27:35.007948+0000","172.21.15.45:0/3852874833":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6800/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/1162729130":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/3020673835":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:6801/4050317147":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/782292734":"2024-02-01T05:22:41.447737+0000","172.21.15.45:0/745566822":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/3329080521":"2024-02-01T05:13:46.510704+0000","172.21.15.45:0/2742663751":"2024-02-01T05:22:41.447737+0000","172.21.15.45:0/1829415949":"2024-02-01T05:22:41.447737+0000","172.21.15.45:6801/2712881513":"2024-02-01T05:13:20.248709+0000","172.21.15.45:0/2019451955":"2024-02-01T05:13:20.248709+0000","172.21.15.45:6800/2788245525":"2024-02-01T05:22:41.447737+0000","172.21.15.45:0/948304328":"2024-02-01T05:22:41.447737+0000","172.21.15.45:6801/2788245525":"2024-02-01T05:22:41.447737+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:40:45.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.271+0000 7f26111c6640 1 -- 172.21.15.45:0/782821289 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f25f80a1450 msgr2=0x7f25f80a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.271+0000 7f26111c6640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f25f80a1450 0x7f25f80a3910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f260c0ffab0 tx=0x7f25f400b410 comp rx=0 tx=0).stop 2024-01-31T05:40:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.271+0000 7f26111c6640 1 -- 172.21.15.45:0/782821289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260c0ffe20 msgr2=0x7f260c1972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.271+0000 7f26111c6640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260c0ffe20 0x7f260c1972d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f25fc005940 tx=0x7f25fc004dd0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.271+0000 7f26111c6640 1 -- 172.21.15.45:0/782821289 shutdown_connections 2024-01-31T05:40:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.271+0000 7f26111c6640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f25f80a1450 0x7f25f80a3910 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.271+0000 7f26111c6640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260c19bb90 0x7f260c19c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.271+0000 7f26111c6640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f260c104ef0 0x7f260c197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.271+0000 7f26111c6640 1 --2- 172.21.15.45:0/782821289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260c0ffe20 0x7f260c1972d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.271+0000 7f26111c6640 1 -- 172.21.15.45:0/782821289 >> 172.21.15.45:0/782821289 conn(0x7f260c076c90 msgr2=0x7f260c0696e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:45.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.271+0000 7f26111c6640 1 -- 172.21.15.45:0/782821289 shutdown_connections 2024-01-31T05:40:45.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.271+0000 7f26111c6640 1 -- 172.21.15.45:0/782821289 wait complete. 2024-01-31T05:40:45.281 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:44 smithi045 ceph-mon[31585]: pgmap v574: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 4.0 MiB/s wr, 154 op/s 2024-01-31T05:40:45.281 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:44 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/4214720079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:40:45.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-31T05:40:45.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:44 smithi186 ceph-mon[35659]: pgmap v574: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 4.0 MiB/s wr, 154 op/s 2024-01-31T05:40:45.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:44 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/4214720079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:40:45.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.385+0000 7ff56901d640 1 -- 172.21.15.45:0/3280302800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff564100780 msgr2=0x7ff5641047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.385+0000 7ff56901d640 1 --2- 172.21.15.45:0/3280302800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff564100780 0x7ff5641047c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009930 tx=0x7ff54c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:45.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.385+0000 7ff56901d640 1 -- 172.21.15.45:0/3280302800 shutdown_connections 2024-01-31T05:40:45.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.385+0000 7ff56901d640 1 --2- 172.21.15.45:0/3280302800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff564100780 0x7ff5641047c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.385+0000 7ff56901d640 1 --2- 172.21.15.45:0/3280302800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff564104ea0 0x7ff56410ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.385+0000 7ff56901d640 1 --2- 172.21.15.45:0/3280302800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5640ffdd0 0x7ff5641001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.385+0000 7ff56901d640 1 -- 172.21.15.45:0/3280302800 >> 172.21.15.45:0/3280302800 conn(0x7ff564076cf0 msgr2=0x7ff564077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:45.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.385+0000 7ff56901d640 1 -- 172.21.15.45:0/3280302800 shutdown_connections 2024-01-31T05:40:45.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.385+0000 7ff56901d640 1 -- 172.21.15.45:0/3280302800 wait complete. 2024-01-31T05:40:45.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.386+0000 7ff56901d640 1 Processor -- start 2024-01-31T05:40:45.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.386+0000 7ff56901d640 1 -- start start 2024-01-31T05:40:45.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56901d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5640ffdd0 0x7ff564197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:45.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56901d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff564104ea0 0x7ff5641977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:45.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56901d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff56419bb50 0x7ff56419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:45.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56901d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56419c540 con 0x7ff56419bb50 2024-01-31T05:40:45.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56901d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56419c6b0 con 0x7ff5640ffdd0 2024-01-31T05:40:45.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56901d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56419c820 con 0x7ff564104ea0 2024-01-31T05:40:45.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff56419bb50 0x7ff56419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:45.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff56419bb50 0x7ff56419c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37138/0 (socket says 172.21.15.45:37138) 2024-01-31T05:40:45.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56359e640 1 -- 172.21.15.45:0/1101219168 learned_addr learned my addr 172.21.15.45:0/1101219168 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:45.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff562d9d640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5640ffdd0 0x7ff564197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:45.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56259c640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff564104ea0 0x7ff5641977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:45.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56359e640 1 -- 172.21.15.45:0/1101219168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff564104ea0 msgr2=0x7ff5641977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56359e640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff564104ea0 0x7ff5641977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56359e640 1 -- 172.21.15.45:0/1101219168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5640ffdd0 msgr2=0x7ff564197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56359e640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5640ffdd0 0x7ff564197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56359e640 1 -- 172.21.15.45:0/1101219168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c009560 con 0x7ff56419bb50 2024-01-31T05:40:45.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56359e640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff56419bb50 0x7ff56419c000 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff554009fd0 tx=0x7ff55400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:45.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff55bfff640 1 -- 172.21.15.45:0/1101219168 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55400bea0 con 0x7ff56419bb50 2024-01-31T05:40:45.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56901d640 1 -- 172.21.15.45:0/1101219168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5641a3e00 con 0x7ff56419bb50 2024-01-31T05:40:45.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff55bfff640 1 -- 172.21.15.45:0/1101219168 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff554009940 con 0x7ff56419bb50 2024-01-31T05:40:45.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff55bfff640 1 -- 172.21.15.45:0/1101219168 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5540106c0 con 0x7ff56419bb50 2024-01-31T05:40:45.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.387+0000 7ff56901d640 1 -- 172.21.15.45:0/1101219168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5641a4280 con 0x7ff56419bb50 2024-01-31T05:40:45.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.388+0000 7ff56901d640 1 -- 172.21.15.45:0/1101219168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff564067ae0 con 0x7ff56419bb50 2024-01-31T05:40:45.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.389+0000 7ff55bfff640 1 -- 172.21.15.45:0/1101219168 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7ff5540041b0 con 0x7ff56419bb50 2024-01-31T05:40:45.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.390+0000 7ff55bfff640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff5500a1450 0x7ff5500a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:45.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.390+0000 7ff55bfff640 1 -- 172.21.15.45:0/1101219168 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff5540ccb20 con 0x7ff56419bb50 2024-01-31T05:40:45.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.390+0000 7ff562d9d640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff5500a1450 0x7ff5500a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:45.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.390+0000 7ff562d9d640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff5500a1450 0x7ff5500a3910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00bbe0 tx=0x7ff55c00b390 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:45.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.394+0000 7ff55bfff640 1 -- 172.21.15.45:0/1101219168 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7ff554095d10 con 0x7ff56419bb50 2024-01-31T05:40:45.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.613+0000 7ff56901d640 1 -- 172.21.15.45:0/1101219168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff5641985e0 con 0x7ff56419bb50 2024-01-31T05:40:45.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.615+0000 7ff55bfff640 1 -- 172.21.15.45:0/1101219168 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff554099330 con 0x7ff56419bb50 2024-01-31T05:40:45.615 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-31T05:40:45.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.618+0000 7ff56901d640 1 -- 172.21.15.45:0/1101219168 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff5500a1450 msgr2=0x7ff5500a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.618+0000 7ff56901d640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff5500a1450 0x7ff5500a3910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00bbe0 tx=0x7ff55c00b390 comp rx=0 tx=0).stop 2024-01-31T05:40:45.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.618+0000 7ff56901d640 1 -- 172.21.15.45:0/1101219168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff56419bb50 msgr2=0x7ff56419c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.618+0000 7ff56901d640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff56419bb50 0x7ff56419c000 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff554009fd0 tx=0x7ff55400b260 comp rx=0 tx=0).stop 2024-01-31T05:40:45.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.618+0000 7ff56901d640 1 -- 172.21.15.45:0/1101219168 shutdown_connections 2024-01-31T05:40:45.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.618+0000 7ff56901d640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7ff5500a1450 0x7ff5500a3910 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.618+0000 7ff56901d640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff56419bb50 0x7ff56419c000 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.618+0000 7ff56901d640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff564104ea0 0x7ff5641977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.618+0000 7ff56901d640 1 --2- 172.21.15.45:0/1101219168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5640ffdd0 0x7ff564197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.618+0000 7ff56901d640 1 -- 172.21.15.45:0/1101219168 >> 172.21.15.45:0/1101219168 conn(0x7ff564076cf0 msgr2=0x7ff564068290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:45.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.618+0000 7ff56901d640 1 -- 172.21.15.45:0/1101219168 shutdown_connections 2024-01-31T05:40:45.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.618+0000 7ff56901d640 1 -- 172.21.15.45:0/1101219168 wait complete. 2024-01-31T05:40:45.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-31T05:40:45.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.735+0000 7f39c1d16640 1 -- 172.21.15.45:0/3850128831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39bc068af0 msgr2=0x7f39bc1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.735+0000 7f39c1d16640 1 --2- 172.21.15.45:0/3850128831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39bc068af0 0x7f39bc1058c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009930 tx=0x7f39b002bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:45.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.736+0000 7f39c1d16640 1 -- 172.21.15.45:0/3850128831 shutdown_connections 2024-01-31T05:40:45.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.736+0000 7f39c1d16640 1 --2- 172.21.15.45:0/3850128831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39bc068af0 0x7f39bc1058c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.736+0000 7f39c1d16640 1 --2- 172.21.15.45:0/3850128831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39bc105fa0 0x7f39bc10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.736+0000 7f39c1d16640 1 --2- 172.21.15.45:0/3850128831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc068140 0x7f39bc068520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.736+0000 7f39c1d16640 1 -- 172.21.15.45:0/3850128831 >> 172.21.15.45:0/3850128831 conn(0x7f39bc076cf0 msgr2=0x7f39bc077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:45.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.736+0000 7f39c1d16640 1 -- 172.21.15.45:0/3850128831 shutdown_connections 2024-01-31T05:40:45.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.736+0000 7f39c1d16640 1 -- 172.21.15.45:0/3850128831 wait complete. 2024-01-31T05:40:45.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.736+0000 7f39c1d16640 1 Processor -- start 2024-01-31T05:40:45.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.736+0000 7f39c1d16640 1 -- start start 2024-01-31T05:40:45.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.736+0000 7f39c1d16640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc068140 0x7f39bc197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:45.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39c1d16640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39bc105fa0 0x7f39bc1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:45.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39c1d16640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39bc19bb40 0x7f39bc19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:45.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39c1d16640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc19c530 con 0x7f39bc19bb40 2024-01-31T05:40:45.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39c1d16640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc19c6a0 con 0x7f39bc105fa0 2024-01-31T05:40:45.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39c1d16640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc19c810 con 0x7f39bc068140 2024-01-31T05:40:45.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39bbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39bc19bb40 0x7f39bc19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:45.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39bbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39bc19bb40 0x7f39bc19bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37152/0 (socket says 172.21.15.45:37152) 2024-01-31T05:40:45.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39bbfff640 1 -- 172.21.15.45:0/1756041068 learned_addr learned my addr 172.21.15.45:0/1756041068 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:45.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39bb7fe640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc068140 0x7f39bc197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:45.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39baffd640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39bc105fa0 0x7f39bc1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:45.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39bbfff640 1 -- 172.21.15.45:0/1756041068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc068140 msgr2=0x7f39bc197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39bbfff640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc068140 0x7f39bc197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39bbfff640 1 -- 172.21.15.45:0/1756041068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39bc105fa0 msgr2=0x7f39bc1977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:45.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39bbfff640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39bc105fa0 0x7f39bc1977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:45.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39bbfff640 1 -- 172.21.15.45:0/1756041068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b0009560 con 0x7f39bc19bb40 2024-01-31T05:40:45.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39bb7fe640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc068140 0x7f39bc197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:40:45.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39bbfff640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39bc19bb40 0x7f39bc19bff0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f39b400eaa0 tx=0x7f39b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:45.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39b8ff9640 1 -- 172.21.15.45:0/1756041068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b400cdb0 con 0x7f39bc19bb40 2024-01-31T05:40:45.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39c1d16640 1 -- 172.21.15.45:0/1756041068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc1a3df0 con 0x7f39bc19bb40 2024-01-31T05:40:45.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39b8ff9640 1 -- 172.21.15.45:0/1756041068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39b4005020 con 0x7f39bc19bb40 2024-01-31T05:40:45.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39b8ff9640 1 -- 172.21.15.45:0/1756041068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b40116d0 con 0x7f39bc19bb40 2024-01-31T05:40:45.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.737+0000 7f39c1d16640 1 -- 172.21.15.45:0/1756041068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc1a4270 con 0x7f39bc19bb40 2024-01-31T05:40:45.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.738+0000 7f39c1d16640 1 -- 172.21.15.45:0/1756041068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc067ae0 con 0x7f39bc19bb40 2024-01-31T05:40:45.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.739+0000 7f39b8ff9640 1 -- 172.21.15.45:0/1756041068 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f39b4005190 con 0x7f39bc19bb40 2024-01-31T05:40:45.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.740+0000 7f39b8ff9640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f39a40a1450 0x7f39a40a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:45.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.740+0000 7f39bb7fe640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f39a40a1450 0x7f39a40a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:45.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.740+0000 7f39b8ff9640 1 -- 172.21.15.45:0/1756041068 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8741+0+0 (secure 0 0 0) 0x7f39b40cd6a0 con 0x7f39bc19bb40 2024-01-31T05:40:45.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.740+0000 7f39bb7fe640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f39a40a1450 0x7f39a40a3910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0058c0 tx=0x7f39ac005850 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:45.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.743+0000 7f39b8ff9640 1 -- 172.21.15.45:0/1756041068 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f39b4096700 con 0x7f39bc19bb40 2024-01-31T05:40:45.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a[31581]: 2024-01-31T05:40:45.614+0000 7fd257f06700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-31T05:40:45.942 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b[66600]: ignoring --setuser ceph since I am not root 2024-01-31T05:40:45.943 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b[66600]: ignoring --setgroup ceph since I am not root 2024-01-31T05:40:45.943 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-mds[66604]: -- 172.21.15.156:0/3193717114 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5562aaa8f520 con 0x5562aaa64c00 2024-01-31T05:40:45.943 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b[66600]: starting mds.b at 2024-01-31T05:40:45.943 INFO:journalctl@ceph.mds.e.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-e[65977]: ignoring --setuser ceph since I am not root 2024-01-31T05:40:45.943 INFO:journalctl@ceph.mds.e.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-e[65977]: ignoring --setgroup ceph since I am not root 2024-01-31T05:40:45.943 INFO:journalctl@ceph.mds.e.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-mds[65981]: -- 172.21.15.156:0/2197091415 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55719e3bf520 con 0x55719e395000 2024-01-31T05:40:45.943 INFO:journalctl@ceph.mds.e.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-e[65977]: starting mds.e at 2024-01-31T05:40:45.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:45.963+0000 7f39c1d16640 1 -- 172.21.15.45:0/1756041068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f39bc1092c0 con 0x7f39bc19bb40 2024-01-31T05:40:46.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1461365136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:40:46.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/782821289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:40:46.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1101219168' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-01-31T05:40:46.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-mon[35659]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-01-31T05:40:46.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-mon[35659]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-31T05:40:46.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1101219168' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-01-31T05:40:46.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-mon[35659]: osdmap e120: 12 total, 12 up, 12 in 2024-01-31T05:40:46.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-mon[35659]: fsmap cephfs:0/3 5 up:standby, 3 failed 2024-01-31T05:40:46.094 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-i[66979]: ignoring --setuser ceph since I am not root 2024-01-31T05:40:46.094 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-i[66979]: ignoring --setgroup ceph since I am not root 2024-01-31T05:40:46.094 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-mds[66983]: -- 172.21.15.186:0/2714526249 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e75b70d520 con 0x55e75b6e3000 2024-01-31T05:40:46.094 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-i[66979]: starting mds.i at 2024-01-31T05:40:46.094 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-c[66673]: ignoring --setuser ceph since I am not root 2024-01-31T05:40:46.095 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-c[66673]: ignoring --setgroup ceph since I am not root 2024-01-31T05:40:46.095 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-mds[66677]: -- 172.21.15.186:0/448193286 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557ae0001520 con 0x557adffd6c00 2024-01-31T05:40:46.095 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-c[66673]: starting mds.c at 2024-01-31T05:40:46.095 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f[66358]: ignoring --setuser ceph since I am not root 2024-01-31T05:40:46.095 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f[66358]: ignoring --setgroup ceph since I am not root 2024-01-31T05:40:46.096 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-mds[66362]: -- 172.21.15.186:0/2692174288 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563359b87520 con 0x563359b5c800 2024-01-31T05:40:46.096 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f[66358]: starting mds.f at 2024-01-31T05:40:46.096 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l[66049]: ignoring --setuser ceph since I am not root 2024-01-31T05:40:46.096 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l[66049]: ignoring --setgroup ceph since I am not root 2024-01-31T05:40:46.096 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-mds[66053]: -- 172.21.15.186:0/2914827899 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557ceb57d520 con 0x557ceb553000 2024-01-31T05:40:46.096 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:40:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l[66049]: starting mds.l at 2024-01-31T05:40:46.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1461365136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:40:46.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/782821289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:40:46.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1101219168' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-01-31T05:40:46.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-mon[35720]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-01-31T05:40:46.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-mon[35720]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-31T05:40:46.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1101219168' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-01-31T05:40:46.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-mon[35720]: osdmap e120: 12 total, 12 up, 12 in 2024-01-31T05:40:46.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:45 smithi156 ceph-mon[35720]: fsmap cephfs:0/3 5 up:standby, 3 failed 2024-01-31T05:40:46.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:45 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1461365136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-31T05:40:46.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:45 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/782821289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-31T05:40:46.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:45 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1101219168' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-01-31T05:40:46.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:45 smithi045 ceph-mon[31585]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-01-31T05:40:46.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:45 smithi045 ceph-mon[31585]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-31T05:40:46.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:45 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1101219168' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-01-31T05:40:46.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:45 smithi045 ceph-mon[31585]: osdmap e120: 12 total, 12 up, 12 in 2024-01-31T05:40:46.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:45 smithi045 ceph-mon[31585]: fsmap cephfs:0/3 5 up:standby, 3 failed 2024-01-31T05:40:46.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.621+0000 7f39b8ff9640 1 -- 172.21.15.45:0/1756041068 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7f39bc1092c0 con 0x7f39bc19bb40 2024-01-31T05:40:46.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.624+0000 7f39c1d16640 1 -- 172.21.15.45:0/1756041068 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f39a40a1450 msgr2=0x7f39a40a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:46.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.624+0000 7f39c1d16640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f39a40a1450 0x7f39a40a3910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0058c0 tx=0x7f39ac005850 comp rx=0 tx=0).stop 2024-01-31T05:40:46.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.624+0000 7f39c1d16640 1 -- 172.21.15.45:0/1756041068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39bc19bb40 msgr2=0x7f39bc19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:46.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.624+0000 7f39c1d16640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39bc19bb40 0x7f39bc19bff0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f39b400eaa0 tx=0x7f39b400ef60 comp rx=0 tx=0).stop 2024-01-31T05:40:46.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.624+0000 7f39c1d16640 1 -- 172.21.15.45:0/1756041068 shutdown_connections 2024-01-31T05:40:46.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.624+0000 7f39c1d16640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f39a40a1450 0x7f39a40a3910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:46.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.624+0000 7f39c1d16640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39bc19bb40 0x7f39bc19bff0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:46.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.624+0000 7f39c1d16640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39bc105fa0 0x7f39bc1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:46.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.624+0000 7f39c1d16640 1 --2- 172.21.15.45:0/1756041068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc068140 0x7f39bc197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:46.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.624+0000 7f39c1d16640 1 -- 172.21.15.45:0/1756041068 >> 172.21.15.45:0/1756041068 conn(0x7f39bc076cf0 msgr2=0x7f39bc0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:46.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.624+0000 7f39c1d16640 1 -- 172.21.15.45:0/1756041068 shutdown_connections 2024-01-31T05:40:46.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.624+0000 7f39c1d16640 1 -- 172.21.15.45:0/1756041068 wait complete. 2024-01-31T05:40:46.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-31T05:40:46.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.753+0000 7f776441d640 1 -- 172.21.15.45:0/772238416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775c1007c0 msgr2=0x7f775c104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:46.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.753+0000 7f776441d640 1 --2- 172.21.15.45:0/772238416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775c1007c0 0x7f775c104800 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7750009930 tx=0x7f775002bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:46.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.753+0000 7f776441d640 1 -- 172.21.15.45:0/772238416 shutdown_connections 2024-01-31T05:40:46.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.753+0000 7f776441d640 1 --2- 172.21.15.45:0/772238416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775c1007c0 0x7f775c104800 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:46.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.753+0000 7f776441d640 1 --2- 172.21.15.45:0/772238416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c104ee0 0x7f775c10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:46.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.753+0000 7f776441d640 1 --2- 172.21.15.45:0/772238416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f775c0ffe10 0x7f775c1001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:46.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.753+0000 7f776441d640 1 -- 172.21.15.45:0/772238416 >> 172.21.15.45:0/772238416 conn(0x7f775c075f70 msgr2=0x7f775c076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:46.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.753+0000 7f776441d640 1 -- 172.21.15.45:0/772238416 shutdown_connections 2024-01-31T05:40:46.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.753+0000 7f776441d640 1 -- 172.21.15.45:0/772238416 wait complete. 2024-01-31T05:40:46.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.753+0000 7f776441d640 1 Processor -- start 2024-01-31T05:40:46.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.753+0000 7f776441d640 1 -- start start 2024-01-31T05:40:46.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f776441d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f775c0ffe10 0x7f775c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:46.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f776441d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c104ee0 0x7f775c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:46.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f776441d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775c19bb00 0x7f775c19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:46.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f776441d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c19c4f0 con 0x7f775c19bb00 2024-01-31T05:40:46.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f776441d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c19c660 con 0x7f775c0ffe10 2024-01-31T05:40:46.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f776441d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c19c7d0 con 0x7f775c104ee0 2024-01-31T05:40:46.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f77629ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775c19bb00 0x7f775c19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:46.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f77629ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775c19bb00 0x7f775c19bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37174/0 (socket says 172.21.15.45:37174) 2024-01-31T05:40:46.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f77629ba640 1 -- 172.21.15.45:0/1812100067 learned_addr learned my addr 172.21.15.45:0/1812100067 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:46.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f77621b9640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f775c0ffe10 0x7f775c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:46.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f77619b8640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c104ee0 0x7f775c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:46.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f77629ba640 1 -- 172.21.15.45:0/1812100067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c104ee0 msgr2=0x7f775c197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:46.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f77629ba640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c104ee0 0x7f775c197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:46.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f77629ba640 1 -- 172.21.15.45:0/1812100067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f775c0ffe10 msgr2=0x7f775c197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:46.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f77629ba640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f775c0ffe10 0x7f775c197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:46.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f77629ba640 1 -- 172.21.15.45:0/1812100067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7750009560 con 0x7f775c19bb00 2024-01-31T05:40:46.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f77629ba640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775c19bb00 0x7f775c19bfb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f774c00eaa0 tx=0x7f774c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:46.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f775b7fe640 1 -- 172.21.15.45:0/1812100067 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c00cdb0 con 0x7f775c19bb00 2024-01-31T05:40:46.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f775b7fe640 1 -- 172.21.15.45:0/1812100067 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f774c005020 con 0x7f775c19bb00 2024-01-31T05:40:46.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f776441d640 1 -- 172.21.15.45:0/1812100067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775c1a3db0 con 0x7f775c19bb00 2024-01-31T05:40:46.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.754+0000 7f775b7fe640 1 -- 172.21.15.45:0/1812100067 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c0116d0 con 0x7f775c19bb00 2024-01-31T05:40:46.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.755+0000 7f776441d640 1 -- 172.21.15.45:0/1812100067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c1a4230 con 0x7f775c19bb00 2024-01-31T05:40:46.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.755+0000 7f776441d640 1 -- 172.21.15.45:0/1812100067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775c1012e0 con 0x7f775c19bb00 2024-01-31T05:40:46.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.757+0000 7f775b7fe640 1 -- 172.21.15.45:0/1812100067 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f774c005190 con 0x7f775c19bb00 2024-01-31T05:40:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.758+0000 7f775b7fe640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f77440a1450 0x7f77440a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.758+0000 7f77621b9640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f77440a1450 0x7f77440a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.758+0000 7f775b7fe640 1 -- 172.21.15.45:0/1812100067 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8741+0+0 (secure 0 0 0) 0x7f774c0cd620 con 0x7f775c19bb00 2024-01-31T05:40:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.758+0000 7f77621b9640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f77440a1450 0x7f77440a3910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7754009b60 tx=0x7f7754005f90 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.761+0000 7f775b7fe640 1 -- 172.21.15.45:0/1812100067 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f774c096680 con 0x7f775c19bb00 2024-01-31T05:40:46.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:46.941+0000 7f776441d640 1 -- 172.21.15.45:0/1812100067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f775c067f30 con 0x7f775c19bb00 2024-01-31T05:40:46.946 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: pgmap v576: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 3.6 MiB/s wr, 125 op/s 2024-01-31T05:40:46.946 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1756041068' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-01-31T05:40:46.946 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-01-31T05:40:46.946 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-01-31T05:40:46.947 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: Cluster is now healthy 2024-01-31T05:40:46.947 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1756041068' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-01-31T05:40:46.947 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.156:6835/2601371319,v1:172.21.15.156:6837/2601371319] up:boot 2024-01-31T05:40:46.947 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.156:6834/370173035,v1:172.21.15.156:6836/370173035] up:boot 2024-01-31T05:40:46.947 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.186:6832/2771625817,v1:172.21.15.186:6834/2771625817] up:boot 2024-01-31T05:40:46.948 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.186:6836/3211892659,v1:172.21.15.186:6838/3211892659] up:boot 2024-01-31T05:40:46.948 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.186:6833/3674236436,v1:172.21.15.186:6835/3674236436] up:boot 2024-01-31T05:40:46.948 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: mds.? [v2:172.21.15.186:6837/3111463418,v1:172.21.15.186:6839/3111463418] up:boot 2024-01-31T05:40:46.948 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: fsmap 11 up:standby 2024-01-31T05:40:46.948 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-31T05:40:46.948 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-31T05:40:46.949 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-31T05:40:46.949 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-31T05:40:46.949 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-31T05:40:46.949 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:46 smithi156 ceph-mon[35720]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-31T05:40:47.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: pgmap v576: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 3.6 MiB/s wr, 125 op/s 2024-01-31T05:40:47.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1756041068' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-01-31T05:40:47.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-01-31T05:40:47.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-01-31T05:40:47.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: Cluster is now healthy 2024-01-31T05:40:47.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1756041068' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-01-31T05:40:47.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.156:6835/2601371319,v1:172.21.15.156:6837/2601371319] up:boot 2024-01-31T05:40:47.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.156:6834/370173035,v1:172.21.15.156:6836/370173035] up:boot 2024-01-31T05:40:47.343 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.186:6832/2771625817,v1:172.21.15.186:6834/2771625817] up:boot 2024-01-31T05:40:47.344 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.186:6836/3211892659,v1:172.21.15.186:6838/3211892659] up:boot 2024-01-31T05:40:47.344 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.186:6833/3674236436,v1:172.21.15.186:6835/3674236436] up:boot 2024-01-31T05:40:47.344 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: mds.? [v2:172.21.15.186:6837/3111463418,v1:172.21.15.186:6839/3111463418] up:boot 2024-01-31T05:40:47.344 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: fsmap 11 up:standby 2024-01-31T05:40:47.344 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-31T05:40:47.344 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-31T05:40:47.344 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-31T05:40:47.344 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-31T05:40:47.345 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-31T05:40:47.345 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:46 smithi186 ceph-mon[35659]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-31T05:40:47.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: pgmap v576: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 3.6 MiB/s wr, 125 op/s 2024-01-31T05:40:47.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1756041068' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-01-31T05:40:47.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-01-31T05:40:47.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-01-31T05:40:47.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: Cluster is now healthy 2024-01-31T05:40:47.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1756041068' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-01-31T05:40:47.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.156:6835/2601371319,v1:172.21.15.156:6837/2601371319] up:boot 2024-01-31T05:40:47.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.156:6834/370173035,v1:172.21.15.156:6836/370173035] up:boot 2024-01-31T05:40:47.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.186:6832/2771625817,v1:172.21.15.186:6834/2771625817] up:boot 2024-01-31T05:40:47.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.186:6836/3211892659,v1:172.21.15.186:6838/3211892659] up:boot 2024-01-31T05:40:47.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.186:6833/3674236436,v1:172.21.15.186:6835/3674236436] up:boot 2024-01-31T05:40:47.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: mds.? [v2:172.21.15.186:6837/3111463418,v1:172.21.15.186:6839/3111463418] up:boot 2024-01-31T05:40:47.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: fsmap 11 up:standby 2024-01-31T05:40:47.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-31T05:40:47.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-31T05:40:47.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-31T05:40:47.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-31T05:40:47.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-31T05:40:47.396 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:46 smithi045 ceph-mon[31585]: from='mgr.15315 172.21.15.45:0/524312289' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-31T05:40:47.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.623+0000 7f775b7fe640 1 -- 172.21.15.45:0/1812100067 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) v1 ==== 181+0+0 (secure 0 0 0) 0x7f774c01f020 con 0x7f775c19bb00 2024-01-31T05:40:47.623 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-01-31T05:40:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.626+0000 7f776441d640 1 -- 172.21.15.45:0/1812100067 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f77440a1450 msgr2=0x7f77440a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.626+0000 7f776441d640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f77440a1450 0x7f77440a3910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7754009b60 tx=0x7f7754005f90 comp rx=0 tx=0).stop 2024-01-31T05:40:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.626+0000 7f776441d640 1 -- 172.21.15.45:0/1812100067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775c19bb00 msgr2=0x7f775c19bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.626+0000 7f776441d640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775c19bb00 0x7f775c19bfb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f774c00eaa0 tx=0x7f774c00ef60 comp rx=0 tx=0).stop 2024-01-31T05:40:47.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.626+0000 7f776441d640 1 -- 172.21.15.45:0/1812100067 shutdown_connections 2024-01-31T05:40:47.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.626+0000 7f776441d640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f77440a1450 0x7f77440a3910 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:47.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.626+0000 7f776441d640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775c19bb00 0x7f775c19bfb0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:47.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.626+0000 7f776441d640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775c104ee0 0x7f775c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:47.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.626+0000 7f776441d640 1 --2- 172.21.15.45:0/1812100067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f775c0ffe10 0x7f775c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:47.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.626+0000 7f776441d640 1 -- 172.21.15.45:0/1812100067 >> 172.21.15.45:0/1812100067 conn(0x7f775c075f70 msgr2=0x7f775c0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:47.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.626+0000 7f776441d640 1 -- 172.21.15.45:0/1812100067 shutdown_connections 2024-01-31T05:40:47.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.627+0000 7f776441d640 1 -- 172.21.15.45:0/1812100067 wait complete. 2024-01-31T05:40:47.648 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-31T05:40:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.772+0000 7f63725dd640 1 -- 172.21.15.45:0/3163771171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f636c068af0 msgr2=0x7f636c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.772+0000 7f63725dd640 1 --2- 172.21.15.45:0/3163771171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f636c068af0 0x7f636c1058c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f635c009930 tx=0x7f635c02bc30 comp rx=0 tx=0).stop 2024-01-31T05:40:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.772+0000 7f63725dd640 1 -- 172.21.15.45:0/3163771171 shutdown_connections 2024-01-31T05:40:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.772+0000 7f63725dd640 1 --2- 172.21.15.45:0/3163771171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f636c068af0 0x7f636c1058c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:47.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.772+0000 7f63725dd640 1 --2- 172.21.15.45:0/3163771171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f636c105fa0 0x7f636c10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:47.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.772+0000 7f63725dd640 1 --2- 172.21.15.45:0/3163771171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f636c068140 0x7f636c068520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:47.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.772+0000 7f63725dd640 1 -- 172.21.15.45:0/3163771171 >> 172.21.15.45:0/3163771171 conn(0x7f636c076cf0 msgr2=0x7f636c077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:47.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.773+0000 7f63725dd640 1 -- 172.21.15.45:0/3163771171 shutdown_connections 2024-01-31T05:40:47.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.773+0000 7f63725dd640 1 -- 172.21.15.45:0/3163771171 wait complete. 2024-01-31T05:40:47.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.773+0000 7f63725dd640 1 Processor -- start 2024-01-31T05:40:47.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.773+0000 7f63725dd640 1 -- start start 2024-01-31T05:40:47.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.773+0000 7f63725dd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f636c068140 0x7f636c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:47.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.773+0000 7f63725dd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f636c105fa0 0x7f636c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:47.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.773+0000 7f63725dd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f636c19bb00 0x7f636c19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:47.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.773+0000 7f63725dd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c19c4f0 con 0x7f636c105fa0 2024-01-31T05:40:47.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.773+0000 7f63725dd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c19c660 con 0x7f636c068140 2024-01-31T05:40:47.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.773+0000 7f63725dd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c19c7d0 con 0x7f636c19bb00 2024-01-31T05:40:47.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f636b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f636c105fa0 0x7f636c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:47.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f636b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f636c105fa0 0x7f636c197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44112/0 (socket says 172.21.15.45:44112) 2024-01-31T05:40:47.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f636b7fe640 1 -- 172.21.15.45:0/2966384031 learned_addr learned my addr 172.21.15.45:0/2966384031 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-01-31T05:40:47.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f6370b7a640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f636c19bb00 0x7f636c19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:47.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f636bfff640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f636c068140 0x7f636c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:47.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f636b7fe640 1 -- 172.21.15.45:0/2966384031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f636c19bb00 msgr2=0x7f636c19bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:47.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f636b7fe640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f636c19bb00 0x7f636c19bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:47.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f636b7fe640 1 -- 172.21.15.45:0/2966384031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f636c068140 msgr2=0x7f636c197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:47.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f636b7fe640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f636c068140 0x7f636c197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:47.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f636b7fe640 1 -- 172.21.15.45:0/2966384031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c009560 con 0x7f636c105fa0 2024-01-31T05:40:47.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f636b7fe640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f636c105fa0 0x7f636c197780 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f635c009230 tx=0x7f635c02cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:47.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f63697fa640 1 -- 172.21.15.45:0/2966384031 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c004310 con 0x7f636c105fa0 2024-01-31T05:40:47.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f63725dd640 1 -- 172.21.15.45:0/2966384031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636c1a3db0 con 0x7f636c105fa0 2024-01-31T05:40:47.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f63697fa640 1 -- 172.21.15.45:0/2966384031 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f635c004470 con 0x7f636c105fa0 2024-01-31T05:40:47.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f63697fa640 1 -- 172.21.15.45:0/2966384031 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c003d80 con 0x7f636c105fa0 2024-01-31T05:40:47.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.774+0000 7f63725dd640 1 -- 172.21.15.45:0/2966384031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636c1a41b0 con 0x7f636c105fa0 2024-01-31T05:40:47.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.775+0000 7f63725dd640 1 -- 172.21.15.45:0/2966384031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636c067ae0 con 0x7f636c105fa0 2024-01-31T05:40:47.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.777+0000 7f63697fa640 1 -- 172.21.15.45:0/2966384031 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f635c005000 con 0x7f636c105fa0 2024-01-31T05:40:47.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.777+0000 7f63697fa640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f63580a1450 0x7f63580a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:40:47.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.778+0000 7f636bfff640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f63580a1450 0x7f63580a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:40:47.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.778+0000 7f63697fa640 1 -- 172.21.15.45:0/2966384031 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8245+0+0 (secure 0 0 0) 0x7f635c0ef980 con 0x7f636c105fa0 2024-01-31T05:40:47.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.778+0000 7f636bfff640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f63580a1450 0x7f63580a3910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f635400bbe0 tx=0x7f635400b390 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:40:47.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.781+0000 7f63697fa640 1 -- 172.21.15.45:0/2966384031 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f635c0b8c80 con 0x7f636c105fa0 2024-01-31T05:40:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:47.962+0000 7f63725dd640 1 -- 172.21.15.45:0/2966384031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f636c068af0 con 0x7f636c105fa0 2024-01-31T05:40:48.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:47 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1812100067' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-31T05:40:48.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:47 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/1812100067' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-01-31T05:40:48.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:47 smithi156 ceph-mon[35720]: osdmap e121: 12 total, 12 up, 12 in 2024-01-31T05:40:48.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:47 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1812100067' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-31T05:40:48.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:47 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/1812100067' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-01-31T05:40:48.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:47 smithi186 ceph-mon[35659]: osdmap e121: 12 total, 12 up, 12 in 2024-01-31T05:40:48.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:47 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1812100067' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-31T05:40:48.393 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:47 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/1812100067' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-01-31T05:40:48.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:47 smithi045 ceph-mon[31585]: osdmap e121: 12 total, 12 up, 12 in 2024-01-31T05:40:48.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.956+0000 7f63697fa640 1 -- 172.21.15.45:0/2966384031 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v122) v1 ==== 169+0+0 (secure 0 0 0) 0x7f635c0bc2a0 con 0x7f636c105fa0 2024-01-31T05:40:48.956 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-01-31T05:40:48.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.959+0000 7f63725dd640 1 -- 172.21.15.45:0/2966384031 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f63580a1450 msgr2=0x7f63580a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:48.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.959+0000 7f63725dd640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f63580a1450 0x7f63580a3910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f635400bbe0 tx=0x7f635400b390 comp rx=0 tx=0).stop 2024-01-31T05:40:48.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.959+0000 7f63725dd640 1 -- 172.21.15.45:0/2966384031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f636c105fa0 msgr2=0x7f636c197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:40:48.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.959+0000 7f63725dd640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f636c105fa0 0x7f636c197780 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f635c009230 tx=0x7f635c02cd20 comp rx=0 tx=0).stop 2024-01-31T05:40:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.959+0000 7f63725dd640 1 -- 172.21.15.45:0/2966384031 shutdown_connections 2024-01-31T05:40:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.959+0000 7f63725dd640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.45:6800/1984996061,v1:172.21.15.45:6801/1984996061] conn(0x7f63580a1450 0x7f63580a3910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.959+0000 7f63725dd640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f636c105fa0 0x7f636c197780 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.959+0000 7f63725dd640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f636c19bb00 0x7f636c19bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.959+0000 7f63725dd640 1 --2- 172.21.15.45:0/2966384031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f636c068140 0x7f636c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:40:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.959+0000 7f63725dd640 1 -- 172.21.15.45:0/2966384031 >> 172.21.15.45:0/2966384031 conn(0x7f636c076cf0 msgr2=0x7f636c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:40:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.959+0000 7f63725dd640 1 -- 172.21.15.45:0/2966384031 shutdown_connections 2024-01-31T05:40:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-01-31T05:40:48.959+0000 7f63725dd640 1 -- 172.21.15.45:0/2966384031 wait complete. 2024-01-31T05:40:48.974 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-31T05:40:49.011 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-31T05:40:49.045 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-31T05:40:49.140 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-01-31T05:40:49.141 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-01-31T05:40:49.156 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-01-31T05:40:49.171 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-01-31T05:40:49.185 INFO:tasks.cephadm:Stopping all daemons... 2024-01-31T05:40:49.185 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-01-31T05:40:49.185 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.a 2024-01-31T05:40:49.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:48 smithi045 ceph-mon[31585]: pgmap v578: 65 pgs: 65 active+clean; 329 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:40:49.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:48 smithi045 ceph-mon[31585]: from='client.? 172.21.15.45:0/2966384031' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-31T05:40:49.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:48 smithi156 ceph-mon[35720]: pgmap v578: 65 pgs: 65 active+clean; 329 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:40:49.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:48 smithi156 ceph-mon[35720]: from='client.? 172.21.15.45:0/2966384031' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-31T05:40:49.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:48 smithi186 ceph-mon[35659]: pgmap v578: 65 pgs: 65 active+clean; 329 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-01-31T05:40:49.342 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:48 smithi186 ceph-mon[35659]: from='client.? 172.21.15.45:0/2966384031' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-31T05:40:49.623 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:49 smithi045 systemd[1]: Stopping Ceph mon.a for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:40:49.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:49 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a[31581]: 2024-01-31T05:40:49.623+0000 7fd25d711700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:40:49.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:49 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a[31581]: 2024-01-31T05:40:49.623+0000 7fd25d711700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-01-31T05:40:49.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:49 smithi045 podman[94125]: 2024-01-31 05:40:49.685650687 +0000 UTC m=+0.306296019 container died 6db52a688cd2808402d8eb14120a95d364d10b03eb95d6a7cc24d269d5770ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux ) 2024-01-31T05:40:50.315 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:50 smithi045 podman[94125]: 2024-01-31 05:40:50.056936613 +0000 UTC m=+0.677581944 container cleanup 6db52a688cd2808402d8eb14120a95d364d10b03eb95d6a7cc24d269d5770ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-01-31T05:40:50.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jan 31 05:40:50 smithi045 bash[94125]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-a 2024-01-31T05:40:50.474 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.a.service' 2024-01-31T05:40:50.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:40:50.563 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-01-31T05:40:50.563 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-01-31T05:40:50.564 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.b 2024-01-31T05:40:50.869 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:50 smithi156 systemd[1]: Stopping Ceph mon.b for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:40:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:50 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-b[35716]: 2024-01-31T05:40:50.868+0000 7f2e11e33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:40:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:50 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-b[35716]: 2024-01-31T05:40:50.868+0000 7f2e11e33700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-01-31T05:40:51.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:50 smithi156 podman[74840]: 2024-01-31 05:40:50.960142708 +0000 UTC m=+0.206733587 container died 5ab2d0d9b7697d4027e447fb39e95c3764485d773a059e459a044a5299bc2d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-b, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-01-31T05:40:51.577 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:51 smithi156 podman[74840]: 2024-01-31 05:40:51.326991804 +0000 UTC m=+0.573582670 container cleanup 5ab2d0d9b7697d4027e447fb39e95c3764485d773a059e459a044a5299bc2d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, ceph=True) 2024-01-31T05:40:51.577 INFO:journalctl@ceph.mon.b.smithi156.stdout:Jan 31 05:40:51 smithi156 bash[74840]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-b 2024-01-31T05:40:51.733 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.b.service' 2024-01-31T05:40:51.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:40:51.780 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-01-31T05:40:51.780 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-01-31T05:40:51.780 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.c 2024-01-31T05:40:52.092 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:51 smithi186 systemd[1]: Stopping Ceph mon.c for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:40:52.434 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:52 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-c[35639]: 2024-01-31T05:40:52.142+0000 7fe81c6d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:40:52.435 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:52 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-c[35639]: 2024-01-31T05:40:52.142+0000 7fe81c6d8700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-01-31T05:40:52.435 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:52 smithi186 podman[73011]: 2024-01-31 05:40:52.173305133 +0000 UTC m=+0.157971962 container died fad13bf4d04caf9e2b6bb10ea4af6b460175b34ab6b93966df9280b250965cfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-c, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.31.3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-31T05:40:52.826 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:52 smithi186 podman[73011]: 2024-01-31 05:40:52.609130651 +0000 UTC m=+0.593797512 container cleanup fad13bf4d04caf9e2b6bb10ea4af6b460175b34ab6b93966df9280b250965cfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-c, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0) 2024-01-31T05:40:52.826 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:52 smithi186 bash[73011]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-c 2024-01-31T05:40:53.065 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.c.service' 2024-01-31T05:40:53.089 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:52 smithi186 podman[73025]: 2024-01-31 05:40:52.827215936 +0000 UTC m=+0.655936993 container remove fad13bf4d04caf9e2b6bb10ea4af6b460175b34ab6b93966df9280b250965cfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-01-31T05:40:53.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:53 smithi186 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.c.service: Deactivated successfully. 2024-01-31T05:40:53.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:53 smithi186 systemd[1]: Stopped Ceph mon.c for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:40:53.090 INFO:journalctl@ceph.mon.c.smithi186.stdout:Jan 31 05:40:53 smithi186 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mon.c.service: Consumed 14.765s CPU time. 2024-01-31T05:40:53.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:40:53.120 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-01-31T05:40:53.120 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-01-31T05:40:53.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.x 2024-01-31T05:40:53.419 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:40:53 smithi045 systemd[1]: Stopping Ceph mgr.x for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:40:53.752 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:40:53 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x[31816]: 2024-01-31T05:40:53.418+0000 7f0b2dbf6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:40:53.752 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:40:53 smithi045 podman[94246]: 2024-01-31 05:40:53.450242052 +0000 UTC m=+0.140134281 container died c66a60e0cf9e3bf90a67dfa658f3233074fd880add7c65606808b9064a0ef448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True) 2024-01-31T05:40:54.095 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:40:53 smithi045 podman[94246]: 2024-01-31 05:40:53.752769257 +0000 UTC m=+0.442661485 container cleanup c66a60e0cf9e3bf90a67dfa658f3233074fd880add7c65606808b9064a0ef448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-01-31T05:40:54.095 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:40:53 smithi045 bash[94246]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x 2024-01-31T05:40:54.095 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jan 31 05:40:53 smithi045 podman[94260]: 2024-01-31 05:40:53.929102465 +0000 UTC m=+0.477280530 container remove c66a60e0cf9e3bf90a67dfa658f3233074fd880add7c65606808b9064a0ef448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-x, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-01-31T05:40:54.162 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.x.service' 2024-01-31T05:40:54.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:40:54.201 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-01-31T05:40:54.201 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-01-31T05:40:54.201 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.y 2024-01-31T05:40:54.518 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:40:54 smithi156 systemd[1]: Stopping Ceph mgr.y for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:40:54.780 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:40:54 smithi156 podman[74960]: 2024-01-31 05:40:54.518792792 +0000 UTC m=+0.129093785 container died d98f0bf733a7088667588fb847b956aa616377726a9eb48f7a4e10532c081e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True) 2024-01-31T05:40:55.173 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:40:54 smithi156 podman[74960]: 2024-01-31 05:40:54.889277732 +0000 UTC m=+0.499578633 container cleanup d98f0bf733a7088667588fb847b956aa616377726a9eb48f7a4e10532c081e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:40:55.173 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Jan 31 05:40:54 smithi156 bash[74960]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-y 2024-01-31T05:40:55.298 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.y.service' 2024-01-31T05:40:55.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:40:55.335 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-01-31T05:40:55.335 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-01-31T05:40:55.335 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.z 2024-01-31T05:40:55.684 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:40:55 smithi186 systemd[1]: Stopping Ceph mgr.z for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:40:55.956 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:40:55 smithi186 podman[73131]: 2024-01-31 05:40:55.684891888 +0000 UTC m=+0.150512571 container died 371604f44df0544fdde8f52139062031aa5e4b8784c77357d1cb15ecf6f70fb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-01-31T05:40:56.306 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:40:56 smithi186 podman[73131]: 2024-01-31 05:40:56.05573335 +0000 UTC m=+0.521354034 container cleanup 371604f44df0544fdde8f52139062031aa5e4b8784c77357d1cb15ecf6f70fb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-01-31T05:40:56.307 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Jan 31 05:40:56 smithi186 bash[73131]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mgr-z 2024-01-31T05:40:56.468 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mgr.z.service' 2024-01-31T05:40:56.505 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:40:56.505 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-01-31T05:40:56.506 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-01-31T05:40:56.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.0 2024-01-31T05:40:56.823 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:40:56 smithi045 systemd[1]: Stopping Ceph osd.0 for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:40:57.143 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:40:56 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0[46341]: 2024-01-31T05:40:56.823+0000 7fe4fece1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:40:57.143 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:40:56 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0[46341]: 2024-01-31T05:40:56.823+0000 7fe4fece1700 -1 osd.0 122 *** Got signal Terminated *** 2024-01-31T05:40:57.144 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:40:56 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0[46341]: 2024-01-31T05:40:56.823+0000 7fe4fece1700 -1 osd.0 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-31T05:41:02.499 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:41:02 smithi045 podman[94365]: 2024-01-31 05:41:02.224579243 +0000 UTC m=+5.522075520 container died 7e46a11e9297844394fd53edf16a69a5a48bb45abadaa75767000dde1af3788c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-01-31T05:41:02.815 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:41:02 smithi045 podman[94365]: 2024-01-31 05:41:02.590205727 +0000 UTC m=+5.887702000 container cleanup 7e46a11e9297844394fd53edf16a69a5a48bb45abadaa75767000dde1af3788c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-01-31T05:41:02.815 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:41:02 smithi045 bash[94365]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0 2024-01-31T05:41:03.129 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:41:02 smithi045 podman[94382]: 2024-01-31 05:41:02.816274931 +0000 UTC m=+0.590817958 container remove 7e46a11e9297844394fd53edf16a69a5a48bb45abadaa75767000dde1af3788c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-01-31T05:41:03.394 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:41:03 smithi045 podman[94474]: 2024-01-31 05:41:03.03262311 +0000 UTC m=+0.018127966 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:41:03.394 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:41:03 smithi045 podman[94474]: 2024-01-31 05:41:03.142326758 +0000 UTC m=+0.127831614 container create 99c040a8ea21a126a555ed58cbf812a7ac48aa74b0f48321838ee279307269b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:41:03.394 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:41:03 smithi045 podman[94474]: 2024-01-31 05:41:03.353523857 +0000 UTC m=+0.339028722 container init 99c040a8ea21a126a555ed58cbf812a7ac48aa74b0f48321838ee279307269b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240123, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True) 2024-01-31T05:41:03.394 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:41:03 smithi045 podman[94474]: 2024-01-31 05:41:03.356706736 +0000 UTC m=+0.342211591 container start 99c040a8ea21a126a555ed58cbf812a7ac48aa74b0f48321838ee279307269b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-01-31T05:41:03.644 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:41:03 smithi045 podman[94474]: 2024-01-31 05:41:03.41122257 +0000 UTC m=+0.396727442 container attach 99c040a8ea21a126a555ed58cbf812a7ac48aa74b0f48321838ee279307269b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-01-31T05:41:03.644 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jan 31 05:41:03 smithi045 podman[94474]: 2024-01-31 05:41:03.643553602 +0000 UTC m=+0.629058468 container died 99c040a8ea21a126a555ed58cbf812a7ac48aa74b0f48321838ee279307269b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-31T05:41:04.311 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.0.service' 2024-01-31T05:41:04.354 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:41:04.354 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-01-31T05:41:04.354 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-01-31T05:41:04.354 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.1 2024-01-31T05:41:04.648 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:04 smithi156 systemd[1]: Stopping Ceph osd.1 for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:41:04.977 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:04 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1[40319]: 2024-01-31T05:41:04.650+0000 7fc80e581700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:41:04.978 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:04 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1[40319]: 2024-01-31T05:41:04.650+0000 7fc80e581700 -1 osd.1 122 *** Got signal Terminated *** 2024-01-31T05:41:04.978 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:04 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1[40319]: 2024-01-31T05:41:04.650+0000 7fc80e581700 -1 osd.1 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-31T05:41:10.477 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:10 smithi156 podman[75087]: 2024-01-31 05:41:10.155484857 +0000 UTC m=+5.611783163 container stop 8c803c56beeeafdbb6b363eb33659b82d50251b188f79d54c26b128682a4dbec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:41:10.477 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:10 smithi156 podman[75087]: 2024-01-31 05:41:10.185000759 +0000 UTC m=+5.641299140 container died 8c803c56beeeafdbb6b363eb33659b82d50251b188f79d54c26b128682a4dbec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, GIT_CLEAN=True) 2024-01-31T05:41:10.761 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:10 smithi156 podman[75087]: 2024-01-31 05:41:10.576207715 +0000 UTC m=+6.032506009 container cleanup 8c803c56beeeafdbb6b363eb33659b82d50251b188f79d54c26b128682a4dbec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-01-31T05:41:10.761 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:10 smithi156 bash[75087]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1 2024-01-31T05:41:11.182 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:10 smithi156 podman[75115]: 2024-01-31 05:41:10.760962114 +0000 UTC m=+0.599003818 container remove 8c803c56beeeafdbb6b363eb33659b82d50251b188f79d54c26b128682a4dbec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3) 2024-01-31T05:41:11.446 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:11 smithi156 podman[75209]: 2024-01-31 05:41:11.085226046 +0000 UTC m=+0.017307782 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:41:11.447 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:11 smithi156 podman[75209]: 2024-01-31 05:41:11.186816005 +0000 UTC m=+0.118897765 container create 744608c9e290c8c2f1cfa6446949103749579f756ae42b87b2323839fd17d019 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0) 2024-01-31T05:41:11.447 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:11 smithi156 podman[75209]: 2024-01-31 05:41:11.447645742 +0000 UTC m=+0.379727473 container init 744608c9e290c8c2f1cfa6446949103749579f756ae42b87b2323839fd17d019 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-01-31T05:41:11.727 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:11 smithi156 podman[75209]: 2024-01-31 05:41:11.450802759 +0000 UTC m=+0.382884477 container start 744608c9e290c8c2f1cfa6446949103749579f756ae42b87b2323839fd17d019 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1-deactivate, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-01-31T05:41:11.727 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:11 smithi156 podman[75209]: 2024-01-31 05:41:11.530679371 +0000 UTC m=+0.462761105 container attach 744608c9e290c8c2f1cfa6446949103749579f756ae42b87b2323839fd17d019 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-01-31T05:41:12.016 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jan 31 05:41:11 smithi156 podman[75209]: 2024-01-31 05:41:11.733004264 +0000 UTC m=+0.665085990 container died 744608c9e290c8c2f1cfa6446949103749579f756ae42b87b2323839fd17d019 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-31T05:41:12.290 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.1.service' 2024-01-31T05:41:12.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:41:12.333 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-01-31T05:41:12.334 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-01-31T05:41:12.334 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.2 2024-01-31T05:41:12.667 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:12 smithi186 systemd[1]: Stopping Ceph osd.2 for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:41:13.092 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:12 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2[40830]: 2024-01-31T05:41:12.666+0000 7f536dbd1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:41:13.092 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:12 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2[40830]: 2024-01-31T05:41:12.666+0000 7f536dbd1700 -1 osd.2 122 *** Got signal Terminated *** 2024-01-31T05:41:13.092 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:12 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2[40830]: 2024-01-31T05:41:12.666+0000 7f536dbd1700 -1 osd.2 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-31T05:41:18.526 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:18 smithi186 podman[73267]: 2024-01-31 05:41:18.2231823 +0000 UTC m=+5.690106576 container died cdec22b13c6d1ceb0d902bda6cadd3f40321b88809820b87ada9407f294cc3f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2) 2024-01-31T05:41:18.842 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:18 smithi186 podman[73267]: 2024-01-31 05:41:18.600771364 +0000 UTC m=+6.067695634 container cleanup cdec22b13c6d1ceb0d902bda6cadd3f40321b88809820b87ada9407f294cc3f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123) 2024-01-31T05:41:18.842 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:18 smithi186 bash[73267]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2 2024-01-31T05:41:18.842 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:18 smithi186 podman[73286]: 2024-01-31 05:41:18.776723578 +0000 UTC m=+0.543572083 container remove cdec22b13c6d1ceb0d902bda6cadd3f40321b88809820b87ada9407f294cc3f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS) 2024-01-31T05:41:19.587 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:19 smithi186 podman[73375]: 2024-01-31 05:41:19.107374679 +0000 UTC m=+0.018006875 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:41:19.588 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:19 smithi186 podman[73375]: 2024-01-31 05:41:19.219737096 +0000 UTC m=+0.130369316 container create 167bafb257b8dbcffb30f41764aa5da55640e3ee2ce52a16341c38140b47bef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2-deactivate, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-01-31T05:41:19.588 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:19 smithi186 podman[73375]: 2024-01-31 05:41:19.430278122 +0000 UTC m=+0.340910320 container init 167bafb257b8dbcffb30f41764aa5da55640e3ee2ce52a16341c38140b47bef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-01-31T05:41:19.588 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:19 smithi186 podman[73375]: 2024-01-31 05:41:19.433680261 +0000 UTC m=+0.344312441 container start 167bafb257b8dbcffb30f41764aa5da55640e3ee2ce52a16341c38140b47bef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123) 2024-01-31T05:41:19.589 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:19 smithi186 podman[73375]: 2024-01-31 05:41:19.588330051 +0000 UTC m=+0.498962234 container attach 167bafb257b8dbcffb30f41764aa5da55640e3ee2ce52a16341c38140b47bef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-01-31T05:41:20.373 INFO:journalctl@ceph.osd.2.smithi186.stdout:Jan 31 05:41:20 smithi186 podman[73375]: 2024-01-31 05:41:20.111204881 +0000 UTC m=+1.021837062 container died 167bafb257b8dbcffb30f41764aa5da55640e3ee2ce52a16341c38140b47bef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-2-deactivate, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123) 2024-01-31T05:41:20.686 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.2.service' 2024-01-31T05:41:20.727 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:41:20.727 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-01-31T05:41:20.728 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-01-31T05:41:20.728 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.3 2024-01-31T05:41:21.142 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:20 smithi045 systemd[1]: Stopping Ceph osd.3 for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:41:21.393 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:21 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3[53235]: 2024-01-31T05:41:21.141+0000 7f26dae04700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:41:21.393 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:21 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3[53235]: 2024-01-31T05:41:21.141+0000 7f26dae04700 -1 osd.3 122 *** Got signal Terminated *** 2024-01-31T05:41:21.393 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:21 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3[53235]: 2024-01-31T05:41:21.141+0000 7f26dae04700 -1 osd.3 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-31T05:41:22.976 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:22 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:22.694+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:23.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:23 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:23.689+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:24.630 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:24.340+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:24.893 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:24 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3[53235]: 2024-01-31T05:41:24.631+0000 7f26d7c00700 -1 osd.3 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.871923+0000 front 2024-01-31T05:41:00.872052+0000 (oldest deadline 2024-01-31T05:41:23.772031+0000) 2024-01-31T05:41:24.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:24 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:24.720+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:25.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:25 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:25.327+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:25.842 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:25 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:25.397+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:26.143 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:25 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3[53235]: 2024-01-31T05:41:25.650+0000 7f26d7c00700 -1 osd.3 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.871923+0000 front 2024-01-31T05:41:00.872052+0000 (oldest deadline 2024-01-31T05:41:23.772031+0000) 2024-01-31T05:41:26.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:25 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:25.746+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:26.592 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:26 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:26.413+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:26.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:26 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:26.229+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:26.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:26 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:26.345+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:26.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:26 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:26.717+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:27.074 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:26 smithi045 podman[94593]: 2024-01-31 05:41:26.753841762 +0000 UTC m=+5.829512717 container died b9dc97c7ce0ec606b97e0ba2a1764519b4629a3aef2efcb00d1698649c1dc7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0) 2024-01-31T05:41:27.074 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:26 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:26.958+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:27.339 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:27 smithi045 podman[94593]: 2024-01-31 05:41:27.074966357 +0000 UTC m=+6.150637312 container cleanup b9dc97c7ce0ec606b97e0ba2a1764519b4629a3aef2efcb00d1698649c1dc7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-31T05:41:27.339 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:27 smithi045 bash[94593]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3 2024-01-31T05:41:27.339 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:27 smithi045 podman[94612]: 2024-01-31 05:41:27.276065578 +0000 UTC m=+0.516928661 container remove b9dc97c7ce0ec606b97e0ba2a1764519b4629a3aef2efcb00d1698649c1dc7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:41:27.592 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:27 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:27.376+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:27.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:27 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:27.249+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:27.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:27 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:27.338+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:27.961 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:27 smithi045 podman[94703]: 2024-01-31 05:41:27.603573576 +0000 UTC m=+0.017463484 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:41:27.961 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:27 smithi045 podman[94703]: 2024-01-31 05:41:27.7352209 +0000 UTC m=+0.149110846 container create 5e3129933bc6de6b0724328fc38d2b9f8cdbfa1af9e844245f8efa47c39a1780 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.31.3) 2024-01-31T05:41:27.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:27 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:27.675+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:28.250 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:27 smithi045 podman[94703]: 2024-01-31 05:41:27.962420689 +0000 UTC m=+0.376310592 container init 5e3129933bc6de6b0724328fc38d2b9f8cdbfa1af9e844245f8efa47c39a1780 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=) 2024-01-31T05:41:28.251 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:27 smithi045 podman[94703]: 2024-01-31 05:41:27.965852379 +0000 UTC m=+0.379742286 container start 5e3129933bc6de6b0724328fc38d2b9f8cdbfa1af9e844245f8efa47c39a1780 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-01-31T05:41:28.251 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:28 smithi045 podman[94703]: 2024-01-31 05:41:28.045530861 +0000 UTC m=+0.459420770 container attach 5e3129933bc6de6b0724328fc38d2b9f8cdbfa1af9e844245f8efa47c39a1780 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, ceph=True, org.label-schema.vendor=CentOS) 2024-01-31T05:41:28.251 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:27 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:27.963+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:28.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:28 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:28.210+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:28.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:28 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:28.375+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:28.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:28 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:28.385+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:28.643 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jan 31 05:41:28 smithi045 podman[94703]: 2024-01-31 05:41:28.251458919 +0000 UTC m=+0.665348836 container died 5e3129933bc6de6b0724328fc38d2b9f8cdbfa1af9e844245f8efa47c39a1780 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:41:28.971 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.3.service' 2024-01-31T05:41:28.976 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:28 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:28.630+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:29.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:41:29.014 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-01-31T05:41:29.014 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-01-31T05:41:29.014 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.4 2024-01-31T05:41:29.303 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:29 smithi156 systemd[1]: Stopping Ceph osd.4 for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:41:29.347 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:28 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:28.982+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:29.583 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:29 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4[47345]: 2024-01-31T05:41:29.303+0000 7fa635734700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:41:29.584 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:29 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4[47345]: 2024-01-31T05:41:29.303+0000 7fa635734700 -1 osd.4 122 *** Got signal Terminated *** 2024-01-31T05:41:29.584 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:29 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4[47345]: 2024-01-31T05:41:29.303+0000 7fa635734700 -1 osd.4 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-31T05:41:29.592 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:29 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:29.386+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:29.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:29 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:29.176+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:29.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:29 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:29.348+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:29.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:29 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:29.584+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:30.330 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:29 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:29.952+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:30.398 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:30 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:30.130+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:30.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:30 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:30.330+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:30.826 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:30 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:30.398+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:30.849 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:30 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:30.546+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:31.134 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:30 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:30.849+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:31.342 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:31 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:31.087+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:31.373 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:30 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:30.998+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:31.476 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:31 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4[47345]: 2024-01-31T05:41:31.134+0000 7fa62e528700 -1 osd.4 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:06.080729+0000 front 2024-01-31T05:41:06.080751+0000 (oldest deadline 2024-01-31T05:41:30.780964+0000) 2024-01-31T05:41:31.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:31 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:31.373+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:31.842 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:31 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:31.369+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:31.842 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:31 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:31.370+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.006913+0000 front 2024-01-31T05:41:05.006949+0000 (oldest deadline 2024-01-31T05:41:30.907067+0000) 2024-01-31T05:41:31.888 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:31 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:31.524+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:32.227 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:32 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4[47345]: 2024-01-31T05:41:32.095+0000 7fa62e528700 -1 osd.4 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:06.080729+0000 front 2024-01-31T05:41:06.080751+0000 (oldest deadline 2024-01-31T05:41:30.780964+0000) 2024-01-31T05:41:32.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:31 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:31.888+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:32.344 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:31 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:31.988+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:32.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:32 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:32.136+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:32.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:32 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:32.326+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:32.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:32 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:32.326+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.006913+0000 front 2024-01-31T05:41:05.006949+0000 (oldest deadline 2024-01-31T05:41:30.907067+0000) 2024-01-31T05:41:32.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:32 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:32.344+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:32.876 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:32 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:32.565+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:33.227 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:33 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4[47345]: 2024-01-31T05:41:33.126+0000 7fa62e528700 -1 osd.4 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:06.080729+0000 front 2024-01-31T05:41:06.080751+0000 (oldest deadline 2024-01-31T05:41:30.780964+0000) 2024-01-31T05:41:33.228 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:32 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:32.876+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:33.337 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:32 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:32.985+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:33.494 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:33 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:33.144+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:33.494 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:33 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:33.167+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:33.495 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:33 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:33.316+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:33.495 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:33 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:33.316+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.006913+0000 front 2024-01-31T05:41:05.006949+0000 (oldest deadline 2024-01-31T05:41:30.907067+0000) 2024-01-31T05:41:33.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:33 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:33.338+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:33.922 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:33 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:33.557+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:33.922 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:33 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:33.557+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:34.226 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:33 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:33.923+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:34.227 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:34 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4[47345]: 2024-01-31T05:41:34.155+0000 7fa62e528700 -1 osd.4 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:06.080729+0000 front 2024-01-31T05:41:06.080751+0000 (oldest deadline 2024-01-31T05:41:30.780964+0000) 2024-01-31T05:41:34.319 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:33 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:33.951+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:34.319 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:33 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:33.951+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:34.595 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:34 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:34.215+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:34.595 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:34 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:34.315+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:34.595 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:34 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:34.315+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.006913+0000 front 2024-01-31T05:41:05.006949+0000 (oldest deadline 2024-01-31T05:41:30.907067+0000) 2024-01-31T05:41:34.595 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:34 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:34.105+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:34.644 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:34 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:34.320+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:34.644 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:34 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:34.320+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:34.788 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:34 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:34.542+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:34.788 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:34 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:34.542+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:35.052 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:34 smithi156 podman[75315]: 2024-01-31 05:41:34.788378522 +0000 UTC m=+5.586855177 container died 0b8d1860f711fa4858ded814778c84d55dabfa19a8dd60901044bbf3e7812bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2) 2024-01-31T05:41:35.053 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:34 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:34.875+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:35.274 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:34 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:34.921+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:35.274 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:34 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:34.921+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:35.319 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:35 smithi156 podman[75315]: 2024-01-31 05:41:35.135708192 +0000 UTC m=+5.934184843 container cleanup 0b8d1860f711fa4858ded814778c84d55dabfa19a8dd60901044bbf3e7812bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-01-31T05:41:35.319 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:35 smithi156 bash[75315]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4 2024-01-31T05:41:35.592 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:35 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:35.136+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:35.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:35 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:35.201+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:35.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:35 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:35.201+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:35.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:35 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:35.353+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:35.594 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:35 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:35.353+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.006913+0000 front 2024-01-31T05:41:05.006949+0000 (oldest deadline 2024-01-31T05:41:30.907067+0000) 2024-01-31T05:41:35.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:35 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:35.274+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:35.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:35 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:35.274+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:35.727 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:35 smithi156 podman[75341]: 2024-01-31 05:41:35.320074797 +0000 UTC m=+0.527248815 container remove 0b8d1860f711fa4858ded814778c84d55dabfa19a8dd60901044bbf3e7812bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, GIT_CLEAN=True) 2024-01-31T05:41:35.727 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:35 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:35.558+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:35.727 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:35 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:35.558+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:36.022 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:35 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:35.829+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:36.023 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:35 smithi156 podman[75434]: 2024-01-31 05:41:35.662150235 +0000 UTC m=+0.017708198 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:41:36.023 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:35 smithi156 podman[75434]: 2024-01-31 05:41:35.771038199 +0000 UTC m=+0.126596151 container create 1d26505f18b2460f25668b9c89ec09c21ef894669ff9f958032621f069bfce11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4-deactivate, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-01-31T05:41:36.023 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:35 smithi156 podman[75434]: 2024-01-31 05:41:35.956900212 +0000 UTC m=+0.312458177 container init 1d26505f18b2460f25668b9c89ec09c21ef894669ff9f958032621f069bfce11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4-deactivate, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-01-31T05:41:36.023 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:35 smithi156 podman[75434]: 2024-01-31 05:41:35.959879846 +0000 UTC m=+0.315437795 container start 1d26505f18b2460f25668b9c89ec09c21ef894669ff9f958032621f069bfce11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4-deactivate, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-01-31T05:41:36.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:35 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:35.872+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:36.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:35 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:35.872+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:36.477 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:36 smithi156 podman[75434]: 2024-01-31 05:41:36.023449664 +0000 UTC m=+0.379007632 container attach 1d26505f18b2460f25668b9c89ec09c21ef894669ff9f958032621f069bfce11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-01-31T05:41:36.478 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:36 smithi156 podman[75434]: 2024-01-31 05:41:36.241145541 +0000 UTC m=+0.596703505 container died 1d26505f18b2460f25668b9c89ec09c21ef894669ff9f958032621f069bfce11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-01-31T05:41:36.592 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:36 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:36.155+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:36.592 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:36 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:36.387+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:36.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:36 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:36.387+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.006913+0000 front 2024-01-31T05:41:05.006949+0000 (oldest deadline 2024-01-31T05:41:30.907067+0000) 2024-01-31T05:41:36.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:36 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:36.161+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:36.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:36 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:36.161+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:36.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:36 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:36.310+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:36.644 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:36 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:36.310+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:36.826 INFO:journalctl@ceph.osd.4.smithi156.stdout:Jan 31 05:41:36 smithi156 podman[75461]: 2024-01-31 05:41:36.809817226 +0000 UTC m=+0.564178999 container remove 1d26505f18b2460f25668b9c89ec09c21ef894669ff9f958032621f069bfce11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-4-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3) 2024-01-31T05:41:36.826 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:36 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:36.570+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:36.826 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:36 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:36.570+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:36.831 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.4.service' 2024-01-31T05:41:36.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:41:36.874 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-01-31T05:41:36.874 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-01-31T05:41:36.874 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.5 2024-01-31T05:41:37.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:36 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:36.857+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:37.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:36 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:36.857+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:37.203 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:36 smithi186 systemd[1]: Stopping Ceph osd.5 for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:41:37.204 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:37 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:37.124+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:37.226 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:36 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:36.863+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:37.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:37 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:37.204+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:37.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:37 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:37.204+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:37.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:37 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:37.210+0000 7f014813f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:41:37.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:37 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:37.210+0000 7f014813f700 -1 osd.5 122 *** Got signal Terminated *** 2024-01-31T05:41:37.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:37 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:37.210+0000 7f014813f700 -1 osd.5 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-31T05:41:37.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:37 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:37.395+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:37.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:37 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:37.395+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.006913+0000 front 2024-01-31T05:41:05.006949+0000 (oldest deadline 2024-01-31T05:41:30.907067+0000) 2024-01-31T05:41:37.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:37 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:37.346+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:37.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:37 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:37.346+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:37.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:37 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:37.816+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:37.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:37 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:37.597+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:37.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:37 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:37.597+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:38.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:37 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:37.859+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:38.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:37 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:37.859+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:38.373 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:38 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:38.108+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:38.373 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:38 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:38.216+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:38.373 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:38 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:38.216+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:38.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:38 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:38.329+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:38.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:38 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:38.329+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:38.842 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:38 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:38.373+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:38.842 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:38 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:38.373+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.006913+0000 front 2024-01-31T05:41:05.006949+0000 (oldest deadline 2024-01-31T05:41:30.907067+0000) 2024-01-31T05:41:38.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:38 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:38.640+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:38.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:38 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:38.640+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:38.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:38 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:38.839+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:38.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:38 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:38.839+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:39.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:38 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:38.837+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:39.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:38 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:38.837+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:39.592 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:39 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:39.335+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:39.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:39 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:39.335+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.006913+0000 front 2024-01-31T05:41:05.006949+0000 (oldest deadline 2024-01-31T05:41:30.907067+0000) 2024-01-31T05:41:39.593 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:39 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:39.102+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:39.593 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:39 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:39.102+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:39.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:39 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:39.248+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:39.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:39 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:39.248+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:39.594 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:39 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:39.248+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:39.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:39 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:39.354+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:39.644 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:39 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:39.354+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:39.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:39 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:39.836+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:39.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:39 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:39.836+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:39.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:39 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:39.671+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:39.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:39 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:39.671+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:39.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:39 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:39.671+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:40.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:39 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:39.874+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:40.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:39 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:39.874+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:40.342 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:40 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:40.289+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:40.342 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:40 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:40.289+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.006913+0000 front 2024-01-31T05:41:05.006949+0000 (oldest deadline 2024-01-31T05:41:30.907067+0000) 2024-01-31T05:41:40.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:40 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:40.067+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:40.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:40 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:40.067+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:40.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:40 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:40.273+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:40.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:40 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:40.273+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:40.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:40 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:40.273+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:40.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:40 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:40.367+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:40.644 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:40 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:40.367+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:40.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:40 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:40.853+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:40.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:40 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:40.853+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:40.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:40 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:40.629+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:40.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:40 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:40.629+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:40.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:40 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:40.629+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:41.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:40 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:40.894+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:41.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:40 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:40.894+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:41.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:40 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:40.894+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:41.592 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:41 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:41.114+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:41.592 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:41 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:41.114+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:41.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:41 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:41.319+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:41.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:41 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:41.319+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:41.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:41 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:41.319+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:41.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:41 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:41.268+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:58.606206+0000 front 2024-01-31T05:40:58.606050+0000 (oldest deadline 2024-01-31T05:41:25.006517+0000) 2024-01-31T05:41:41.593 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:41 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:41.268+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.006913+0000 front 2024-01-31T05:41:05.006949+0000 (oldest deadline 2024-01-31T05:41:30.907067+0000) 2024-01-31T05:41:41.594 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:41 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5[47664]: 2024-01-31T05:41:41.268+0000 7f0144f3b700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:16.609178+0000 front 2024-01-31T05:41:16.609207+0000 (oldest deadline 2024-01-31T05:41:40.709352+0000) 2024-01-31T05:41:41.856 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:41.413+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:41.856 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:41.413+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:41.856 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:41.413+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.933706+0000 front 2024-01-31T05:41:15.933651+0000 (oldest deadline 2024-01-31T05:41:41.233704+0000) 2024-01-31T05:41:41.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:41 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:41.887+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:41.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:41 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:41.887+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:41.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:41 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:41.662+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:41.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:41 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:41.662+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:41.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:41 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:41.662+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:42.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:41.855+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:42.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:41.855+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:42.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:41 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:41.855+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:42.361 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:42 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:42.104+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:42.361 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:42 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:42.104+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:42.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:42 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:42.369+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:42.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:42 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:42.369+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:42.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:42 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:42.369+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.933706+0000 front 2024-01-31T05:41:15.933651+0000 (oldest deadline 2024-01-31T05:41:41.233704+0000) 2024-01-31T05:41:42.731 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:42 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:42.361+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:42.731 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:42 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:42.361+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:42.731 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:42 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:42.361+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:42.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:42 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:42.918+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:42.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:42 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:42.918+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:42.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:42 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:42.690+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:42.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:42 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:42.690+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:42.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:42 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:42.690+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:43.003 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:42 smithi186 podman[73505]: 2024-01-31 05:41:42.731666958 +0000 UTC m=+5.653439388 container died f8fb18af320c849dcc4f958cd840e8a290026b78dbdead64513dd397e361d650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-31T05:41:43.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:42 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:42.846+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:43.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:42 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:42.847+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:43.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:42 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:42.847+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:43.314 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:43 smithi186 podman[73505]: 2024-01-31 05:41:43.093965162 +0000 UTC m=+6.015737562 container cleanup f8fb18af320c849dcc4f958cd840e8a290026b78dbdead64513dd397e361d650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.license=GPLv2) 2024-01-31T05:41:43.314 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:43 smithi186 bash[73505]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5 2024-01-31T05:41:43.315 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:43 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:43.072+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:43.315 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:43 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:43.072+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:43.592 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:43 smithi186 podman[73522]: 2024-01-31 05:41:43.33660257 +0000 UTC m=+0.598609442 container remove f8fb18af320c849dcc4f958cd840e8a290026b78dbdead64513dd397e361d650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:41:43.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:43 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:43.314+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:43.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:43 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:43.314+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:43.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:43 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:43.314+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:43.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:43 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:43.379+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:43.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:43 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:43.379+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:43.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:43 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:43.379+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.933706+0000 front 2024-01-31T05:41:15.933651+0000 (oldest deadline 2024-01-31T05:41:41.233704+0000) 2024-01-31T05:41:43.947 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:43 smithi186 podman[73614]: 2024-01-31 05:41:43.558450881 +0000 UTC m=+0.017857255 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:41:43.948 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:43 smithi186 podman[73614]: 2024-01-31 05:41:43.662430246 +0000 UTC m=+0.121836653 container create 2597ef426bcb5b7dcbde4e2aae0938d5a71be582881ef39c925acb32a4d836ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, ceph=True) 2024-01-31T05:41:43.948 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:43 smithi186 podman[73614]: 2024-01-31 05:41:43.87310551 +0000 UTC m=+0.332511875 container init 2597ef426bcb5b7dcbde4e2aae0938d5a71be582881ef39c925acb32a4d836ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.31.3) 2024-01-31T05:41:43.948 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:43 smithi186 podman[73614]: 2024-01-31 05:41:43.876154612 +0000 UTC m=+0.335560979 container start 2597ef426bcb5b7dcbde4e2aae0938d5a71be582881ef39c925acb32a4d836ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-01-31T05:41:43.948 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:43 smithi186 podman[73614]: 2024-01-31 05:41:43.948061464 +0000 UTC m=+0.407467847 container attach 2597ef426bcb5b7dcbde4e2aae0938d5a71be582881ef39c925acb32a4d836ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-01-31T05:41:43.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:43 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:43.878+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:43.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:43 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:43.878+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:43.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:43 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:43.689+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:43.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:43 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:43.689+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:43.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:43 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:43.689+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:44.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:43 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:43.870+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:44.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:43 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:43.870+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:44.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:43 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:43.870+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:44.278 INFO:journalctl@ceph.osd.5.smithi186.stdout:Jan 31 05:41:44 smithi186 podman[73614]: 2024-01-31 05:41:44.152922588 +0000 UTC m=+0.612328947 container died 2597ef426bcb5b7dcbde4e2aae0938d5a71be582881ef39c925acb32a4d836ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-01-31T05:41:44.279 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:44 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:44.117+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:44.279 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:44 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:44.117+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:44.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:44 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:44.278+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:44.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:44 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:44.278+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:44.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:44 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:44.278+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:44.842 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.5.service' 2024-01-31T05:41:44.853 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:44.398+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:44.853 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:44.398+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:44.853 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:44.398+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.933706+0000 front 2024-01-31T05:41:15.933651+0000 (oldest deadline 2024-01-31T05:41:41.233704+0000) 2024-01-31T05:41:44.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:41:44.883 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-01-31T05:41:44.883 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-01-31T05:41:44.883 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.6 2024-01-31T05:41:44.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:44 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:44.871+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:44.952 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:44 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:44.871+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:44.953 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:44 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:44.659+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:44.953 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:44 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:44.659+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:44.953 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:44 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:44.659+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:45.143 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:44 smithi045 systemd[1]: Stopping Ceph osd.6 for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:41:45.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:44.853+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:45.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:44.853+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:45.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:44 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:44.853+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:45.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:45.083+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:45.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:45.083+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:45.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:45.314+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:45.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:45.314+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:45.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:45 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:45.314+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:45.409 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:45.166+0000 7f4f61e2e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:41:45.409 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:45.166+0000 7f4f61e2e700 -1 osd.6 122 *** Got signal Terminated *** 2024-01-31T05:41:45.410 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:45.166+0000 7f4f61e2e700 -1 osd.6 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-31T05:41:45.830 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:45.421+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:45.830 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:45.421+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:45.831 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:45.421+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.933706+0000 front 2024-01-31T05:41:15.933651+0000 (oldest deadline 2024-01-31T05:41:41.233704+0000) 2024-01-31T05:41:45.869 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:45.660+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:45.869 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:45.660+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:45.869 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:45.660+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:46.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:45.829+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:46.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:45.829+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:46.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:45 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:45.829+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:46.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:45.869+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:46.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:45 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:45.869+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:46.592 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:46 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:46.133+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:46.593 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:46 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:46.133+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:46.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:46 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:46.307+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:46.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:46 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:46.307+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:46.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:46 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:46.307+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:46.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:46 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:46.470+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:46.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:46 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:46.470+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:46.643 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:46 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:46.470+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.933706+0000 front 2024-01-31T05:41:15.933651+0000 (oldest deadline 2024-01-31T05:41:41.233704+0000) 2024-01-31T05:41:46.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:46 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:46.629+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:46.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:46 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:46.629+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:46.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:46 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:46.629+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:46.978 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:46 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:46.877+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:46.978 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:46 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:46.877+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:47.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:46 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:46.865+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:47.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:46 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:46.865+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:47.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:46 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:46.865+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:47.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:47 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:47.090+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:47.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:47 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:47.090+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:47.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:47 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:47.284+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:47.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:47 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:47.284+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:47.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:47 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:47.284+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:47.893 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:47 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:47.491+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:47.893 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:47 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:47.491+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:47.894 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:47 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:47.491+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.933706+0000 front 2024-01-31T05:41:15.933651+0000 (oldest deadline 2024-01-31T05:41:41.233704+0000) 2024-01-31T05:41:47.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:47 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:47.657+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:47.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:47 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:47.657+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:47.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:47 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:47.657+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:47.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:47 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:47.829+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:47.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:47 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:47.829+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:47.978 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:47 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:47.829+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.502443+0000 front 2024-01-31T05:41:23.502544+0000 (oldest deadline 2024-01-31T05:41:47.602384+0000) 2024-01-31T05:41:48.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:48 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:48.072+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:48.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:48 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:48.072+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:48.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:48 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:48.274+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:48.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:48 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:48.274+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:48.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:48 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:48.274+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:48.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:47 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:47.911+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:48.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:47 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:47.911+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:48.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:47 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:47.911+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:48.893 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:48 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:48.483+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:48.894 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:48 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:48.483+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:48.894 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:48 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:48.483+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.933706+0000 front 2024-01-31T05:41:15.933651+0000 (oldest deadline 2024-01-31T05:41:41.233704+0000) 2024-01-31T05:41:48.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:48 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:48.611+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:48.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:48 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:48.611+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:48.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:48 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:48.611+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:48.978 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:48 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:48.818+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:48.978 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:48 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:48.818+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:48.978 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:48 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:48.818+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.502443+0000 front 2024-01-31T05:41:23.502544+0000 (oldest deadline 2024-01-31T05:41:47.602384+0000) 2024-01-31T05:41:49.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:49 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:49.085+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:49.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:49 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:49.085+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:49.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:49 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:49.288+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:49.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:49 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:49.288+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:49.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:49 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:49.288+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:49.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:48 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:48.908+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:49.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:48 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:48.908+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:49.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:48 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:48.908+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:49.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:48 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:48.908+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:41:49.863 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:49 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:49.584+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:49.864 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:49 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:49.584+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:49.864 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:49 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:49.584+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:49.864 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:49 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:49.584+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:41:49.893 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:49 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:49.452+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:01.331482+0000 front 2024-01-31T05:41:01.331588+0000 (oldest deadline 2024-01-31T05:41:23.631626+0000) 2024-01-31T05:41:49.893 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:49 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:49.452+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.532676+0000 front 2024-01-31T05:41:09.532816+0000 (oldest deadline 2024-01-31T05:41:33.632536+0000) 2024-01-31T05:41:49.893 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:49 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6[59807]: 2024-01-31T05:41:49.452+0000 7f4f5ac22700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.933706+0000 front 2024-01-31T05:41:15.933651+0000 (oldest deadline 2024-01-31T05:41:41.233704+0000) 2024-01-31T05:41:50.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:49 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:49.864+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:50.228 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:49 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:49.864+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:50.228 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:49 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:49.864+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.502443+0000 front 2024-01-31T05:41:23.502544+0000 (oldest deadline 2024-01-31T05:41:47.602384+0000) 2024-01-31T05:41:50.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:49 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:49.894+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:50.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:49 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:49.894+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:50.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:49 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:49.894+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:50.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:49 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:49.894+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:41:50.592 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:50 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:50.111+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:50.592 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:50 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:50.111+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:50.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:50 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:50.328+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:50.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:50 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:50.328+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:50.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:50 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:50.328+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:50.942 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:50 smithi045 podman[94810]: 2024-01-31 05:41:50.662874227 +0000 UTC m=+5.589155210 container died c5b8a08fde1f0208aeb55c9eb8a289901183f59d0302f11d9c18934e9da76842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123) 2024-01-31T05:41:50.942 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:50 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:50.905+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:50.942 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:50 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:50.905+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:50.943 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:50 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:50.905+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:50.943 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:50 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:50.905+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:41:50.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:50 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:50.608+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:50.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:50 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:50.608+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:50.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:50 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:50.608+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:50.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:50 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:50.608+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:41:50.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:50 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:50.848+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:50.978 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:50 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:50.848+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:50.978 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:50 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:50.848+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.502443+0000 front 2024-01-31T05:41:23.502544+0000 (oldest deadline 2024-01-31T05:41:47.602384+0000) 2024-01-31T05:41:51.285 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:51 smithi045 podman[94810]: 2024-01-31 05:41:51.04233431 +0000 UTC m=+5.968615295 container cleanup c5b8a08fde1f0208aeb55c9eb8a289901183f59d0302f11d9c18934e9da76842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6, ceph=True, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True) 2024-01-31T05:41:51.285 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:51 smithi045 bash[94810]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6 2024-01-31T05:41:51.592 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:51 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:51.145+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:51.593 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:51 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:51.145+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:51.593 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:51 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:51.145+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:41:51.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:51 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:51.317+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:51.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:51 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:51.317+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:51.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:51 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:51.317+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:51.605 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:51 smithi045 podman[94840]: 2024-01-31 05:41:51.285248025 +0000 UTC m=+0.614772306 container remove c5b8a08fde1f0208aeb55c9eb8a289901183f59d0302f11d9c18934e9da76842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, GIT_BRANCH=HEAD) 2024-01-31T05:41:51.851 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:51 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:51.568+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:51.851 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:51 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:51.568+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:51.851 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:51 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:51.568+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:51.851 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:51 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:51.568+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:41:51.893 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:51 smithi045 podman[94929]: 2024-01-31 05:41:51.509738344 +0000 UTC m=+0.018207676 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:41:51.894 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:51 smithi045 podman[94929]: 2024-01-31 05:41:51.611026865 +0000 UTC m=+0.119496194 container create dc5c28fab9e30c55669a5a64e9413a19ba36eade43184174fadebcd763c74a46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=) 2024-01-31T05:41:51.894 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:51 smithi045 podman[94929]: 2024-01-31 05:41:51.838282937 +0000 UTC m=+0.346752269 container init dc5c28fab9e30c55669a5a64e9413a19ba36eade43184174fadebcd763c74a46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-01-31T05:41:51.894 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:51 smithi045 podman[94929]: 2024-01-31 05:41:51.841381304 +0000 UTC m=+0.349850636 container start dc5c28fab9e30c55669a5a64e9413a19ba36eade43184174fadebcd763c74a46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6-deactivate, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:41:52.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:51 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:51.851+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:52.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:51 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:51.851+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:52.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:51 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:51.851+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.502443+0000 front 2024-01-31T05:41:23.502544+0000 (oldest deadline 2024-01-31T05:41:47.602384+0000) 2024-01-31T05:41:52.393 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:51 smithi045 podman[94929]: 2024-01-31 05:41:51.932593938 +0000 UTC m=+0.441063284 container attach dc5c28fab9e30c55669a5a64e9413a19ba36eade43184174fadebcd763c74a46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-01-31T05:41:52.394 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jan 31 05:41:52 smithi045 podman[94929]: 2024-01-31 05:41:52.127556855 +0000 UTC m=+0.636026193 container died dc5c28fab9e30c55669a5a64e9413a19ba36eade43184174fadebcd763c74a46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-01-31T05:41:52.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:51 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:51.895+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:52.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:51 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:51.895+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:52.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:51 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:51.895+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:52.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:51 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:51.895+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:41:52.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:52 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:52.341+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:52.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:52 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:52.342+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:52.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:52 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:52.342+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:52.593 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:52 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:52.111+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:52.593 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:52 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:52.111+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:52.593 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:52 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:52.111+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:41:52.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.6.service' 2024-01-31T05:41:52.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:41:52.769 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-01-31T05:41:52.769 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-01-31T05:41:52.770 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.7 2024-01-31T05:41:52.818 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:52 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:52.555+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:52.819 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:52 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:52.555+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:52.819 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:52 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:52.555+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:52.819 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:52 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:52.555+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:41:53.078 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:52 smithi156 systemd[1]: Stopping Ceph osd.7 for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:41:53.078 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:52 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:52.849+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:53.078 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:52 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:52.849+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:53.079 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:52 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:52.849+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.502443+0000 front 2024-01-31T05:41:23.502544+0000 (oldest deadline 2024-01-31T05:41:47.602384+0000) 2024-01-31T05:41:53.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:52 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:52.862+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:53.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:52 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:52.862+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:53.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:52 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:52.862+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:53.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:52 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:52.862+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:41:53.342 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:53 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:53.311+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:53.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:53 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:53.311+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:53.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:53 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:53.311+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:53.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:53 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:53.085+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:53.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:53 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:53.085+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:53.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:53 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:53.085+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:41:53.477 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:53 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:53.078+0000 7f9c11e91700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:41:53.477 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:53 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:53.078+0000 7f9c11e91700 -1 osd.7 122 *** Got signal Terminated *** 2024-01-31T05:41:53.477 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:53 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:53.078+0000 7f9c11e91700 -1 osd.7 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-31T05:41:53.899 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:53 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:53.595+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:53.899 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:53 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:53.595+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:53.899 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:53 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:53.595+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:53.899 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:53 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:53.595+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:41:54.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:53 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:53.888+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:54.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:53 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:53.888+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:54.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:53 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:53.888+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:54.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:53 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:53.888+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:41:54.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:53 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:53.898+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:54.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:53 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:53.898+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:54.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:53 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:53.898+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.502443+0000 front 2024-01-31T05:41:23.502544+0000 (oldest deadline 2024-01-31T05:41:47.602384+0000) 2024-01-31T05:41:54.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:54 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:54.267+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:54.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:54 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:54.267+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:54.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:54 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:54.267+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:54.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:54 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:54.061+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:54.344 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:54 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:54.061+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:54.344 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:54 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:54.061+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:41:54.894 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:54 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:54.615+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:54.894 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:54 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:54.615+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:54.894 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:54 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:54.615+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:54.894 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:54 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:54.615+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:41:55.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:54 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:54.894+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:55.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:54 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:54.894+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:55.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:54 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:54.894+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.502443+0000 front 2024-01-31T05:41:23.502544+0000 (oldest deadline 2024-01-31T05:41:47.602384+0000) 2024-01-31T05:41:55.342 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:55 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:55.315+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:55.342 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:55 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:55.315+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:55.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:55 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:55.315+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:55.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:55 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:55.079+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:55.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:55 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:55.079+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:55.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:55 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:55.079+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:41:55.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:54 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:54.913+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:55.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:54 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:54.913+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:55.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:54 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:54.913+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:55.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:54 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:54.913+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:41:55.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:55 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:55.899+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:55.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:55 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:55.899+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:55.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:55 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:55.899+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.502443+0000 front 2024-01-31T05:41:23.502544+0000 (oldest deadline 2024-01-31T05:41:47.602384+0000) 2024-01-31T05:41:55.977 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:55 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:55.899+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:29.303270+0000 front 2024-01-31T05:41:29.303365+0000 (oldest deadline 2024-01-31T05:41:55.203405+0000) 2024-01-31T05:41:55.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:55 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:55.655+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:55.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:55 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:55.655+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:55.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:55 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:55.655+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:55.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:55 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:55.655+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:41:56.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:55 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:55.962+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:56.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:55 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:55.962+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:56.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:55 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:55.962+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:56.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:55 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:55.962+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:41:56.306 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:56 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:56.041+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:56.306 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:56 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:56.041+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:56.306 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:56 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:56.041+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:41:56.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:56 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:56.305+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:56.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:56 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:56.305+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:56.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:56 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:56.305+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:56.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:56 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:56.693+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:56.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:56 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:56.693+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:56.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:56 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:56.693+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:56.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:56 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:56.693+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:41:56.978 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:56 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:56.862+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:56.978 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:56 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:56.862+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:56.978 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:56 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:56.862+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.502443+0000 front 2024-01-31T05:41:23.502544+0000 (oldest deadline 2024-01-31T05:41:47.602384+0000) 2024-01-31T05:41:56.978 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:56 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:56.862+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:29.303270+0000 front 2024-01-31T05:41:29.303365+0000 (oldest deadline 2024-01-31T05:41:55.203405+0000) 2024-01-31T05:41:57.322 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:56 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:56.993+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:57.323 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:56 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:56.993+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:57.323 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:56 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:56.993+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:41:57.323 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:56 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:56.993+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.581136+0000 front 2024-01-31T05:41:30.581042+0000 (oldest deadline 2024-01-31T05:41:56.981605+0000) 2024-01-31T05:41:57.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:57 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:57.000+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:57.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:57 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:57.000+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:57.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:57 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:57.000+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:57.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:57 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:57.000+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:41:57.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:57 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:57.322+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:57.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:57 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:57.322+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:57.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:57 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:57.322+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:57.979 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:57 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:57.877+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.399400+0000 front 2024-01-31T05:41:05.399383+0000 (oldest deadline 2024-01-31T05:41:30.699525+0000) 2024-01-31T05:41:57.980 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:57 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:57.877+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.800539+0000 front 2024-01-31T05:41:14.800570+0000 (oldest deadline 2024-01-31T05:41:38.300640+0000) 2024-01-31T05:41:57.980 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:57 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:57.877+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.502443+0000 front 2024-01-31T05:41:23.502544+0000 (oldest deadline 2024-01-31T05:41:47.602384+0000) 2024-01-31T05:41:57.980 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:57 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7[53964]: 2024-01-31T05:41:57.877+0000 7f9c0ec8d700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:29.303270+0000 front 2024-01-31T05:41:29.303365+0000 (oldest deadline 2024-01-31T05:41:55.203405+0000) 2024-01-31T05:41:57.980 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:57 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:57.662+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:57.980 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:57 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:57.662+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:57.981 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:57 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:57.662+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:57.981 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:57 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:57.662+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:41:58.312 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:58 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:58.028+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:58.312 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:58 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:58.028+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:58.312 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:58 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:58.028+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:41:58.312 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:58 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:58.028+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.581136+0000 front 2024-01-31T05:41:30.581042+0000 (oldest deadline 2024-01-31T05:41:56.981605+0000) 2024-01-31T05:41:58.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:57 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:57.955+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:58.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:57 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:57.955+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:58.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:57 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:57.955+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:58.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:57 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:57.955+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:41:58.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:58 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:58.312+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:58.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:58 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:58.312+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:58.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:58 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:58.312+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:58.811 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:58 smithi156 podman[75541]: 2024-01-31 05:41:58.535717503 +0000 UTC m=+5.574555119 container died 1d56671cd591e440fcf5c896ca1456d7700a100de2e14b3a88d8430e7a515d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:41:58.811 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:58 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:58.695+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:58.812 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:58 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:58.695+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:58.812 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:58 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:58.695+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:58.812 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:58 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:58.695+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:41:59.186 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:58 smithi156 podman[75541]: 2024-01-31 05:41:58.903056724 +0000 UTC m=+5.941894347 container cleanup 1d56671cd591e440fcf5c896ca1456d7700a100de2e14b3a88d8430e7a515d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-01-31T05:41:59.187 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:58 smithi156 bash[75541]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7 2024-01-31T05:41:59.342 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:59 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:59.262+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:41:59.342 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:59 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:59.262+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:41:59.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:41:59 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:41:59.262+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:41:59.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:59 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:59.029+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:41:59.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:59 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:59.029+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:41:59.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:59 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:59.029+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:41:59.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:41:59 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:41:59.029+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.581136+0000 front 2024-01-31T05:41:30.581042+0000 (oldest deadline 2024-01-31T05:41:56.981605+0000) 2024-01-31T05:41:59.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:59 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:59.005+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:41:59.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:59 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:59.005+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:41:59.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:59 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:59.005+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:41:59.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:41:59 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:41:59.005+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:41:59.477 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:59 smithi156 podman[75561]: 2024-01-31 05:41:59.187432961 +0000 UTC m=+0.650246421 container remove 1d56671cd591e440fcf5c896ca1456d7700a100de2e14b3a88d8430e7a515d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, ceph=True, org.label-schema.build-date=20240123, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD) 2024-01-31T05:41:59.806 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:59 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:59.732+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:41:59.806 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:59 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:59.732+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:41:59.806 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:59 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:59.732+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:41:59.806 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:41:59 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:41:59.732+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:41:59.807 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:59 smithi156 podman[75653]: 2024-01-31 05:41:59.413058192 +0000 UTC m=+0.017649435 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:41:59.807 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:59 smithi156 podman[75653]: 2024-01-31 05:41:59.521612383 +0000 UTC m=+0.126203655 container create 4eb2fb6cbdc381ce5143353c5bcc42dc31d8d1f6b5247557f57c07c619532182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7-deactivate, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-31T05:41:59.807 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:59 smithi156 podman[75653]: 2024-01-31 05:41:59.715722361 +0000 UTC m=+0.320313598 container init 4eb2fb6cbdc381ce5143353c5bcc42dc31d8d1f6b5247557f57c07c619532182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-31T05:41:59.807 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:59 smithi156 podman[75653]: 2024-01-31 05:41:59.719155462 +0000 UTC m=+0.323746693 container start 4eb2fb6cbdc381ce5143353c5bcc42dc31d8d1f6b5247557f57c07c619532182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-31T05:41:59.807 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:41:59 smithi156 podman[75653]: 2024-01-31 05:41:59.806975511 +0000 UTC m=+0.411566752 container attach 4eb2fb6cbdc381ce5143353c5bcc42dc31d8d1f6b5247557f57c07c619532182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-01-31T05:42:00.227 INFO:journalctl@ceph.osd.7.smithi156.stdout:Jan 31 05:42:00 smithi156 podman[75653]: 2024-01-31 05:42:00.005656462 +0000 UTC m=+0.610247684 container died 4eb2fb6cbdc381ce5143353c5bcc42dc31d8d1f6b5247557f57c07c619532182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-7-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, GIT_CLEAN=True, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:42:00.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:00.045+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:42:00.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:00.045+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:42:00.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:00.045+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:42:00.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:00.045+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.581136+0000 front 2024-01-31T05:41:30.581042+0000 (oldest deadline 2024-01-31T05:41:56.981605+0000) 2024-01-31T05:42:00.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:00.252+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:00.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:00.252+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:00.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:00.252+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:00.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:00.252+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:00.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:00.021+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:00.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:00.021+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:00.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:00.021+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:00.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:00.021+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:00.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:00 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:00.021+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:00.569 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.7.service' 2024-01-31T05:42:00.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:42:00.610 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-01-31T05:42:00.610 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-01-31T05:42:00.610 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.8 2024-01-31T05:42:00.911 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:00 smithi186 systemd[1]: Stopping Ceph osd.8 for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:42:00.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:00.684+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:00.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:00.684+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:00.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:00.684+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:00.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:00.684+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:00.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:00 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:00.684+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:01.255 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:00.911+0000 7fa3926a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:42:01.255 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:00.911+0000 7fa3926a5700 -1 osd.8 122 *** Got signal Terminated *** 2024-01-31T05:42:01.255 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:00 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:00.911+0000 7fa3926a5700 -1 osd.8 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-31T05:42:01.256 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:01 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:01.013+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:42:01.256 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:01 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:01.013+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:42:01.256 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:01 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:01.013+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:42:01.256 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:01 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:01.013+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.581136+0000 front 2024-01-31T05:41:30.581042+0000 (oldest deadline 2024-01-31T05:41:56.981605+0000) 2024-01-31T05:42:01.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:01.006+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:01.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:01.006+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:01.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:01.006+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:01.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:01.006+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:01.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:01 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:01.006+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:01.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:01 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:01.255+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:01.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:01 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:01.255+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:01.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:01 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:01.255+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:01.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:01 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:01.255+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:02.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:01.733+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:02.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:01.733+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:02.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:01.733+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:02.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:01.733+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:02.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:01 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:01.733+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:02.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:02 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:02.025+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:42:02.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:02 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:02.025+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:42:02.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:02 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:02.025+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:42:02.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:02 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:02.025+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.581136+0000 front 2024-01-31T05:41:30.581042+0000 (oldest deadline 2024-01-31T05:41:56.981605+0000) 2024-01-31T05:42:02.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:02 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:02.237+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:02.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:02 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:02.237+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:02.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:02 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:02.237+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:02.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:02 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:02.237+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:02.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:02 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:02.027+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:02.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:02 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:02.027+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:02.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:02 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:02.027+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:02.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:02 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:02.027+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:02.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:02 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:02.027+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:02.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:02 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:02.721+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:02.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:02 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:02.721+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:02.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:02 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:02.721+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:02.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:02 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:02.721+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:02.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:02 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:02.721+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:03.281 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:03 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:03.002+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:42:03.281 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:03 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:03.002+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:42:03.281 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:03 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:03.002+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:42:03.281 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:03 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:03.002+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.581136+0000 front 2024-01-31T05:41:30.581042+0000 (oldest deadline 2024-01-31T05:41:56.981605+0000) 2024-01-31T05:42:03.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:03 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:03.022+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:03.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:03 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:03.022+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:03.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:03 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:03.022+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:03.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:03 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:03.022+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:03.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:03 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:03.022+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:03.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:03 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:03.281+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:03.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:03 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:03.281+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:03.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:03 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:03.281+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:03.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:03 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:03.281+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:03.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:03 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:03.679+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:03.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:03 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:03.679+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:03.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:03 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:03.679+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:03.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:03 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:03.679+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:03.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:03 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:03.679+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:04.296 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:04.014+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:42:04.297 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:04.014+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:42:04.297 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:04.014+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:42:04.297 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:04.014+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.581136+0000 front 2024-01-31T05:41:30.581042+0000 (oldest deadline 2024-01-31T05:41:56.981605+0000) 2024-01-31T05:42:04.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:04 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:04.036+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:04.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:04 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:04.036+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:04.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:04 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:04.036+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:04.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:04 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:04.036+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:04.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:04 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:04.036+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:04.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:04.296+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:04.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:04.296+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:04.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:04.296+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:04.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:04.296+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:04.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:04.296+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:04.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:04 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:04.683+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:04.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:04 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:04.683+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:04.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:04 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:04.683+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:04.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:04 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:04.683+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:04.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:04 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:04.683+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:05.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:04.998+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:07.876314+0000 front 2024-01-31T05:41:07.876170+0000 (oldest deadline 2024-01-31T05:41:32.576244+0000) 2024-01-31T05:42:05.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:04.998+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.677430+0000 front 2024-01-31T05:41:13.677452+0000 (oldest deadline 2024-01-31T05:41:38.377654+0000) 2024-01-31T05:42:05.342 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:04.998+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:25.880423+0000 front 2024-01-31T05:41:25.880371+0000 (oldest deadline 2024-01-31T05:41:50.580511+0000) 2024-01-31T05:42:05.343 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8[54538]: 2024-01-31T05:42:04.998+0000 7fa38b499700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.581136+0000 front 2024-01-31T05:41:30.581042+0000 (oldest deadline 2024-01-31T05:41:56.981605+0000) 2024-01-31T05:42:05.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:05 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:05.247+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:05.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:05 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:05.247+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:05.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:05 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:05.247+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:05.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:05 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:05.247+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:05.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:05 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:05.247+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:05.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:05 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:05.006+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:05.456 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:05 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:05.006+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:05.456 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:05 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:05.006+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:05.457 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:05 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:05.006+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:05.457 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:05 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:05.006+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:05.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:05.688+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:05.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:05.688+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:05.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:05.688+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:05.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:05.688+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:05.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:05 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:05.688+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:06.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:05 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:05.977+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:06.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:05 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:05.977+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:06.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:05 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:05.977+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:06.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:05 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:05.977+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:06.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:05 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:05.977+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:06.592 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:06 smithi186 podman[73722]: 2024-01-31 05:42:06.390801662 +0000 UTC m=+5.588436394 container died ec75bc9a672000510000440f16e490d8ed39d60505f05d70a4ca25b0a5156b3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-01-31T05:42:06.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:06 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:06.294+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:06.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:06 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:06.294+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:06.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:06 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:06.294+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:06.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:06 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:06.294+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:06.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:06 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:06.294+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:06.914 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:06 smithi186 podman[73722]: 2024-01-31 05:42:06.730295786 +0000 UTC m=+5.927930504 container cleanup ec75bc9a672000510000440f16e490d8ed39d60505f05d70a4ca25b0a5156b3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2) 2024-01-31T05:42:06.915 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:06 smithi186 bash[73722]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8 2024-01-31T05:42:06.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:06 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:06.701+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:06.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:06 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:06.701+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:06.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:06 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:06.701+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:06.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:06 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:06.701+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:06.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:06 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:06.701+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:07.276 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:06 smithi186 podman[73752]: 2024-01-31 05:42:06.915358567 +0000 UTC m=+0.518434037 container remove ec75bc9a672000510000440f16e490d8ed39d60505f05d70a4ca25b0a5156b3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-01-31T05:42:07.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:06 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:06.931+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:07.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:06 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:06.931+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:07.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:06 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:06.931+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:07.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:06 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:06.931+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:07.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:06 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:06.931+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:07.559 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:07 smithi186 podman[73838]: 2024-01-31 05:42:07.24015801 +0000 UTC m=+0.018012883 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:42:07.560 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:07 smithi186 podman[73838]: 2024-01-31 05:42:07.359014518 +0000 UTC m=+0.136869378 container create ef7f3a321479ae038c2b5db58e26cb37dbac67d79f69ae9b36f53b27ad9264ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8-deactivate, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.31.3) 2024-01-31T05:42:07.560 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:07 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:07.276+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:07.560 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:07 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:07.276+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:07.560 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:07 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:07.276+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:07.560 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:07 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:07.276+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:07.560 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:07 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:07.276+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:07.842 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:07 smithi186 podman[73838]: 2024-01-31 05:42:07.560429423 +0000 UTC m=+0.338284297 container init ef7f3a321479ae038c2b5db58e26cb37dbac67d79f69ae9b36f53b27ad9264ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS) 2024-01-31T05:42:07.842 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:07 smithi186 podman[73838]: 2024-01-31 05:42:07.56370336 +0000 UTC m=+0.341558217 container start ef7f3a321479ae038c2b5db58e26cb37dbac67d79f69ae9b36f53b27ad9264ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8-deactivate, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-01-31T05:42:07.843 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:07 smithi186 podman[73838]: 2024-01-31 05:42:07.643512246 +0000 UTC m=+0.421367125 container attach ef7f3a321479ae038c2b5db58e26cb37dbac67d79f69ae9b36f53b27ad9264ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:42:08.137 INFO:journalctl@ceph.osd.8.smithi186.stdout:Jan 31 05:42:07 smithi186 podman[73838]: 2024-01-31 05:42:07.844227347 +0000 UTC m=+0.622082214 container died ef7f3a321479ae038c2b5db58e26cb37dbac67d79f69ae9b36f53b27ad9264ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, ceph=True, org.label-schema.vendor=CentOS) 2024-01-31T05:42:08.226 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:07 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:07.748+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:08.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:07 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:07.748+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:08.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:07 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:07.748+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:08.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:07 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:07.748+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:08.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:07 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:07.748+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:08.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:07 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:07.748+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:08.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:07 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:07.915+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:08.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:07 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:07.915+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:08.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:07 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:07.915+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:08.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:07 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:07.915+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:08.394 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:07 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:07.915+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:08.513 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:08 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:08.307+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:08.513 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:08 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:08.307+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:08.513 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:08 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:08.307+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:08.513 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:08 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:08.307+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:08.513 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:08 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:08.307+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:08.536 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.8.service' 2024-01-31T05:42:08.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:42:08.581 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-01-31T05:42:08.581 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-01-31T05:42:08.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.9 2024-01-31T05:42:08.867 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:08 smithi045 systemd[1]: Stopping Ceph osd.9 for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:42:09.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:08 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:08.867+0000 7f9f69a13700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:42:09.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:08 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:08.867+0000 7f9f69a13700 -1 osd.9 122 *** Got signal Terminated *** 2024-01-31T05:42:09.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:08 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:08.867+0000 7f9f69a13700 -1 osd.9 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-31T05:42:09.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:08 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:08.882+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:09.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:08 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:08.882+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:09.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:08 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:08.882+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:09.145 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:08 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:08.882+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:09.145 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:08 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:08.882+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:09.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:08 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:08.757+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:09.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:08 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:08.757+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:09.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:08 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:08.757+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:09.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:08 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:08.757+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:09.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:08 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:08.757+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:09.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:08 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:08.757+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:09.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:09 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:09.335+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:09.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:09 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:09.335+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:09.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:09 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:09.335+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:09.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:09 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:09.335+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:09.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:09 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:09.335+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:10.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:09 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:09.834+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:10.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:09 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:09.834+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:10.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:09 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:09.834+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:10.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:09 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:09.834+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:10.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:09 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:09.834+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:10.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:09 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:09.730+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:10.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:09 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:09.730+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:10.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:09 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:09.730+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:10.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:09 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:09.730+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:10.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:09 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:09.730+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:10.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:09 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:09.730+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:10.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:10 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:10.302+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:10.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:10 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:10.302+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:10.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:10 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:10.302+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:10.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:10 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:10.302+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:10.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:10 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:10.302+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:10.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:10 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:10.721+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:10.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:10 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:10.721+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:10.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:10 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:10.721+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:10.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:10 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:10.721+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:10.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:10 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:10.721+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:10.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:10 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:10.721+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:11.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:10 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:10.798+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:11.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:10 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:10.798+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:11.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:10 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:10.798+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:11.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:10 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:10.798+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:11.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:10 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:10.798+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:11.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:11 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:11.311+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:11.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:11 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:11.311+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:11.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:11 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:11.311+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:11.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:11 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:11.311+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:11.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:11 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:11.311+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:11.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:11 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:11.671+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:11.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:11 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:11.671+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:11.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:11 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:11.671+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:11.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:11 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:11.671+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:11.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:11 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:11.671+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:11.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:11 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:11.671+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:12.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:11 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:11.779+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:12.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:11 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:11.779+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:12.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:11 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:11.779+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:12.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:11 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:11.779+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:12.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:11 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:11.779+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:12.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:12 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:12.337+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:12.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:12 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:12.337+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:12.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:12 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:12.337+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:12.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:12 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:12.337+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:12.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:12 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:12.337+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:12.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:12 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:12.695+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:12.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:12 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:12.695+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:12.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:12 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:12.695+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:12.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:12 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:12.695+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:12.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:12 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:12.695+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:12.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:12 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:12.695+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:13.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:12 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:12.804+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:13.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:12 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:12.804+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:13.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:12 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:12.804+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:13.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:12 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:12.804+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:13.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:12 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:12.804+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:13.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:12 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:12.804+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:49.537465+0000 front 2024-01-31T05:41:49.537517+0000 (oldest deadline 2024-01-31T05:42:11.837610+0000) 2024-01-31T05:42:13.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:13 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:13.373+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:13.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:13 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:13.373+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:13.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:13 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:13.373+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:13.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:13 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:13.373+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:13.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:13 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:13.373+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:13.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:13 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:13.659+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:13.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:13 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:13.659+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:13.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:13 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:13.659+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:13.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:13 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:13.659+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:13.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:13 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:13.659+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:13.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:13 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:13.659+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:14.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:13 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:13.821+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.128095+0000 front 2024-01-31T05:41:00.128119+0000 (oldest deadline 2024-01-31T05:41:26.028257+0000) 2024-01-31T05:42:14.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:13 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:13.821+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:09.529423+0000 front 2024-01-31T05:41:09.529454+0000 (oldest deadline 2024-01-31T05:41:33.629332+0000) 2024-01-31T05:42:14.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:13 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:13.821+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:15.930393+0000 front 2024-01-31T05:41:15.930421+0000 (oldest deadline 2024-01-31T05:41:40.030383+0000) 2024-01-31T05:42:14.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:13 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:13.821+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:24.131466+0000 front 2024-01-31T05:41:24.131519+0000 (oldest deadline 2024-01-31T05:41:48.831640+0000) 2024-01-31T05:42:14.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:13 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:13.821+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:33.234636+0000 front 2024-01-31T05:41:33.234614+0000 (oldest deadline 2024-01-31T05:41:59.134702+0000) 2024-01-31T05:42:14.144 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:13 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9[66879]: 2024-01-31T05:42:13.821+0000 7f9f62807700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:49.537465+0000 front 2024-01-31T05:41:49.537517+0000 (oldest deadline 2024-01-31T05:42:11.837610+0000) 2024-01-31T05:42:14.643 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:14 smithi045 podman[95036]: 2024-01-31 05:42:14.361700643 +0000 UTC m=+5.589077555 container died 78e372d5937c0d6642b92b0ad69fd9ecebe6ee77193b3079645724d99ee081da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, ceph=True, maintainer=Guillaume Abrioux ) 2024-01-31T05:42:14.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:14 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:14.359+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:14.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:14 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:14.359+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:14.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:14 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:14.359+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:14.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:14 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:14.359+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:14.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:14 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:14.359+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:14.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:14 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:14.656+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:14.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:14 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:14.656+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:14.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:14 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:14.656+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:14.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:14 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:14.656+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:14.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:14 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:14.656+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:14.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:14 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:14.656+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:14.985 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:14 smithi045 podman[95036]: 2024-01-31 05:42:14.801336653 +0000 UTC m=+6.028713567 container cleanup 78e372d5937c0d6642b92b0ad69fd9ecebe6ee77193b3079645724d99ee081da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.vendor=CentOS) 2024-01-31T05:42:14.985 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:14 smithi045 bash[95036]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9 2024-01-31T05:42:15.393 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:14 smithi045 podman[95054]: 2024-01-31 05:42:14.985716148 +0000 UTC m=+0.619319292 container remove 78e372d5937c0d6642b92b0ad69fd9ecebe6ee77193b3079645724d99ee081da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-01-31T05:42:15.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:15 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:15.334+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:15.595 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:15 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:15.334+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:15.595 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:15 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:15.334+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:15.595 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:15 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:15.334+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:15.595 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:15 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:15.334+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:15.721 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:15 smithi045 podman[95143]: 2024-01-31 05:42:15.305225279 +0000 UTC m=+0.017393152 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:42:15.721 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:15 smithi045 podman[95143]: 2024-01-31 05:42:15.419788709 +0000 UTC m=+0.131956617 container create 559b7d61dcf6cf85fbb3249711bd49b9e6414b3605aa83c2c4fd5fcd52fa9d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-01-31T05:42:15.721 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:15 smithi045 podman[95143]: 2024-01-31 05:42:15.638530092 +0000 UTC m=+0.350697960 container init 559b7d61dcf6cf85fbb3249711bd49b9e6414b3605aa83c2c4fd5fcd52fa9d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-01-31T05:42:15.722 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:15 smithi045 podman[95143]: 2024-01-31 05:42:15.641706706 +0000 UTC m=+0.353874572 container start 559b7d61dcf6cf85fbb3249711bd49b9e6414b3605aa83c2c4fd5fcd52fa9d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9-deactivate, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:42:15.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:15 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:15.680+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:15.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:15 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:15.680+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:15.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:15 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:15.680+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:15.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:15 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:15.680+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:15.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:15 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:15.680+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:15.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:15 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:15.680+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:15.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:15 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:15.680+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:49.113714+0000 front 2024-01-31T05:41:49.113586+0000 (oldest deadline 2024-01-31T05:42:15.013567+0000) 2024-01-31T05:42:16.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:15 smithi045 podman[95143]: 2024-01-31 05:42:15.72171976 +0000 UTC m=+0.433887648 container attach 559b7d61dcf6cf85fbb3249711bd49b9e6414b3605aa83c2c4fd5fcd52fa9d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:42:16.143 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jan 31 05:42:15 smithi045 podman[95143]: 2024-01-31 05:42:15.934163501 +0000 UTC m=+0.646331379 container died 559b7d61dcf6cf85fbb3249711bd49b9e6414b3605aa83c2c4fd5fcd52fa9d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-9-deactivate, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:42:16.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:16 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:16.289+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:16.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:16 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:16.289+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:16.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:16 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:16.289+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:16.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:16 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:16.289+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:16.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:16 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:16.289+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:16.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:16 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:16.289+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:16.619 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.9.service' 2024-01-31T05:42:16.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:42:16.662 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-01-31T05:42:16.662 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-01-31T05:42:16.662 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.10 2024-01-31T05:42:16.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:16 smithi156 systemd[1]: Stopping Ceph osd.10 for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:42:16.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:16 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:16.720+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:16.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:16 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:16.720+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:16.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:16 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:16.720+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:16.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:16 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:16.720+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:16.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:16 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:16.720+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:16.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:16 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:16.720+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:16.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:16 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:16.720+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:49.113714+0000 front 2024-01-31T05:41:49.113586+0000 (oldest deadline 2024-01-31T05:42:15.013567+0000) 2024-01-31T05:42:17.477 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:16 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:16.982+0000 7fc56f52f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:42:17.477 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:16 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:16.983+0000 7fc56f52f700 -1 osd.10 122 *** Got signal Terminated *** 2024-01-31T05:42:17.477 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:16 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:16.983+0000 7fc56f52f700 -1 osd.10 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-31T05:42:17.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:17 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:17.271+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:17.592 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:17 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:17.271+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:17.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:17 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:17.271+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:17.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:17 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:17.271+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:17.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:17 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:17.271+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:17.593 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:17 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:17.271+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:17.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:17 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:17.709+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:17.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:17 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:17.709+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:17.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:17 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:17.709+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:17.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:17 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:17.709+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:17.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:17 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:17.709+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:17.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:17 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:17.709+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:17.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:17 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:17.709+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:49.113714+0000 front 2024-01-31T05:41:49.113586+0000 (oldest deadline 2024-01-31T05:42:15.013567+0000) 2024-01-31T05:42:18.585 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:18 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:18.319+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:18.586 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:18 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:18.319+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:18.586 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:18 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:18.319+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:18.586 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:18 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:18.319+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:18.586 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:18 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:18.319+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:18.586 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:18 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:18.319+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:19.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:18 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:18.738+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:19.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:18 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:18.738+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:19.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:18 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:18.738+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:19.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:18 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:18.738+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:19.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:18 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:18.738+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:19.228 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:18 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:18.738+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:19.228 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:18 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:18.738+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:49.113714+0000 front 2024-01-31T05:41:49.113586+0000 (oldest deadline 2024-01-31T05:42:15.013567+0000) 2024-01-31T05:42:19.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:19 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:19.361+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:19.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:19 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:19.361+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:19.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:19 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:19.361+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:19.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:19 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:19.361+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:19.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:19 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:19.361+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:19.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:19 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:19.361+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:19.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:19 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:19.361+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-01-31T05:41:55.701753+0000 front 2024-01-31T05:41:55.701737+0000 (oldest deadline 2024-01-31T05:42:19.201524+0000) 2024-01-31T05:42:20.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:19 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:19.750+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:20.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:19 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:19.750+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:20.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:19 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:19.750+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:20.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:19 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:19.750+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:20.228 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:19 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:19.750+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:20.228 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:19 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:19.750+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:20.228 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:19 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:19.750+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:49.113714+0000 front 2024-01-31T05:41:49.113586+0000 (oldest deadline 2024-01-31T05:42:15.013567+0000) 2024-01-31T05:42:20.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:20 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:20.352+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:20.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:20 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:20.352+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:20.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:20 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:20.352+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:20.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:20 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:20.352+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:20.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:20 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:20.352+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:20.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:20 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:20.352+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:20.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:20 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:20.352+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-01-31T05:41:55.701753+0000 front 2024-01-31T05:41:55.701737+0000 (oldest deadline 2024-01-31T05:42:19.201524+0000) 2024-01-31T05:42:20.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:20 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:20.714+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:20.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:20 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:20.714+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:20.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:20 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:20.714+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:20.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:20 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:20.714+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:20.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:20 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:20.714+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:20.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:20 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:20.714+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:20.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:20 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:20.714+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:49.113714+0000 front 2024-01-31T05:41:49.113586+0000 (oldest deadline 2024-01-31T05:42:15.013567+0000) 2024-01-31T05:42:21.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:21 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:21.386+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:21.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:21 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:21.386+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:21.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:21 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:21.386+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:21.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:21 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:21.386+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:21.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:21 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:21.386+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:21.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:21 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:21.386+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:21.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:21 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:21.386+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-01-31T05:41:55.701753+0000 front 2024-01-31T05:41:55.701737+0000 (oldest deadline 2024-01-31T05:42:19.201524+0000) 2024-01-31T05:42:21.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:21 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:21.675+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:40:57.105254+0000 front 2024-01-31T05:40:57.105183+0000 (oldest deadline 2024-01-31T05:41:22.405087+0000) 2024-01-31T05:42:21.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:21 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:21.675+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:08.806729+0000 front 2024-01-31T05:41:08.806692+0000 (oldest deadline 2024-01-31T05:41:33.506922+0000) 2024-01-31T05:42:21.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:21 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:21.675+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:13.507730+0000 front 2024-01-31T05:41:13.507715+0000 (oldest deadline 2024-01-31T05:41:39.407689+0000) 2024-01-31T05:42:21.977 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:21 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:21.675+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-01-31T05:41:23.508821+0000 front 2024-01-31T05:41:23.508780+0000 (oldest deadline 2024-01-31T05:41:48.808848+0000) 2024-01-31T05:42:21.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:21 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:21.675+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:30.509772+0000 front 2024-01-31T05:41:30.509840+0000 (oldest deadline 2024-01-31T05:41:59.910505+0000) 2024-01-31T05:42:21.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:21 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:21.675+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:41.611657+0000 front 2024-01-31T05:41:41.611741+0000 (oldest deadline 2024-01-31T05:42:06.911685+0000) 2024-01-31T05:42:21.978 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:21 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10[60902]: 2024-01-31T05:42:21.675+0000 7fc568323700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:49.113714+0000 front 2024-01-31T05:41:49.113586+0000 (oldest deadline 2024-01-31T05:42:15.013567+0000) 2024-01-31T05:42:22.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:22 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:22.417+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:22.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:22 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:22.417+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:22.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:22 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:22.417+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:22.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:22 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:22.417+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:22.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:22 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:22.417+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:22.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:22 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:22.417+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:22.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:22 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:22.417+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-01-31T05:41:55.701753+0000 front 2024-01-31T05:41:55.701737+0000 (oldest deadline 2024-01-31T05:42:19.201524+0000) 2024-01-31T05:42:22.853 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:22 smithi156 podman[75770]: 2024-01-31 05:42:22.55665759 +0000 UTC m=+5.694157335 container died 9ec11346a9b6cc0566207dff870ad9c0d81d60058756e25fe885f4db7f283947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-01-31T05:42:23.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:22 smithi156 podman[75770]: 2024-01-31 05:42:22.95350238 +0000 UTC m=+6.091002127 container cleanup 9ec11346a9b6cc0566207dff870ad9c0d81d60058756e25fe885f4db7f283947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.31.3) 2024-01-31T05:42:23.204 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:22 smithi156 bash[75770]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10 2024-01-31T05:42:23.477 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:23 smithi156 podman[75788]: 2024-01-31 05:42:23.204552663 +0000 UTC m=+0.639910533 container remove 9ec11346a9b6cc0566207dff870ad9c0d81d60058756e25fe885f4db7f283947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-01-31T05:42:23.831 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:23 smithi156 podman[75877]: 2024-01-31 05:42:23.428861807 +0000 UTC m=+0.017260009 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:42:23.832 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:23 smithi156 podman[75877]: 2024-01-31 05:42:23.538560166 +0000 UTC m=+0.126958354 container create c4601151f11ff09909d35d083ac073c3bac5ffbbf81f8174460306eb6d2fa301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3) 2024-01-31T05:42:23.832 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:23 smithi156 podman[75877]: 2024-01-31 05:42:23.749218692 +0000 UTC m=+0.337616880 container init c4601151f11ff09909d35d083ac073c3bac5ffbbf81f8174460306eb6d2fa301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:42:23.832 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:23 smithi156 podman[75877]: 2024-01-31 05:42:23.752487499 +0000 UTC m=+0.340885680 container start c4601151f11ff09909d35d083ac073c3bac5ffbbf81f8174460306eb6d2fa301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-01-31T05:42:23.832 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:23 smithi156 podman[75877]: 2024-01-31 05:42:23.832538878 +0000 UTC m=+0.420937084 container attach c4601151f11ff09909d35d083ac073c3bac5ffbbf81f8174460306eb6d2fa301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-01-31T05:42:23.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:23 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:23.454+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:23.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:23 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:23.454+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:23.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:23 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:23.454+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:23.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:23 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:23.454+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:23.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:23 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:23.454+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:23.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:23 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:23.454+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:23.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:23 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:23.454+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-01-31T05:41:55.701753+0000 front 2024-01-31T05:41:55.701737+0000 (oldest deadline 2024-01-31T05:42:19.201524+0000) 2024-01-31T05:42:24.227 INFO:journalctl@ceph.osd.10.smithi156.stdout:Jan 31 05:42:24 smithi156 podman[75877]: 2024-01-31 05:42:24.031961101 +0000 UTC m=+0.620359302 container died c4601151f11ff09909d35d083ac073c3bac5ffbbf81f8174460306eb6d2fa301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-10-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:42:24.607 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.10.service' 2024-01-31T05:42:24.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:42:24.649 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-01-31T05:42:24.649 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-01-31T05:42:24.650 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.11 2024-01-31T05:42:24.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:24 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:24.478+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:24.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:24 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:24.478+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:24.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:24 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:24.478+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:24.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:24 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:24.478+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:24.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:24 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:24.478+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:24.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:24 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:24.478+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:24.844 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:24 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:24.478+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-01-31T05:41:55.701753+0000 front 2024-01-31T05:41:55.701737+0000 (oldest deadline 2024-01-31T05:42:19.201524+0000) 2024-01-31T05:42:24.844 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:24 smithi186 systemd[1]: Stopping Ceph osd.11 for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:42:25.342 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:24 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:24.948+0000 7f1ec9f9c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:42:25.342 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:24 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:24.948+0000 7f1ec9f9c700 -1 osd.11 122 *** Got signal Terminated *** 2024-01-31T05:42:25.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:24 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:24.948+0000 7f1ec9f9c700 -1 osd.11 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-31T05:42:25.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:25 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:25.487+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:25.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:25 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:25.487+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:25.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:25 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:25.487+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:25.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:25 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:25.487+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:25.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:25 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:25.487+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:25.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:25 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:25.487+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:25.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:25 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:25.487+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-01-31T05:41:55.701753+0000 front 2024-01-31T05:41:55.701737+0000 (oldest deadline 2024-01-31T05:42:19.201524+0000) 2024-01-31T05:42:26.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:26 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:26.489+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:26.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:26 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:26.489+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:26.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:26 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:26.489+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:26.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:26 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:26.489+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:26.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:26 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:26.489+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:26.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:26 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:26.489+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:26.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:26 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:26.489+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-01-31T05:41:55.701753+0000 front 2024-01-31T05:41:55.701737+0000 (oldest deadline 2024-01-31T05:42:19.201524+0000) 2024-01-31T05:42:27.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:27 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:27.442+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:27.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:27 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:27.442+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:27.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:27 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:27.442+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:27.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:27 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:27.442+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:27.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:27 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:27.442+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:27.844 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:27 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:27.442+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:27.844 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:27 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:27.442+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-01-31T05:41:55.701753+0000 front 2024-01-31T05:41:55.701737+0000 (oldest deadline 2024-01-31T05:42:19.201524+0000) 2024-01-31T05:42:27.844 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:27 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:27.442+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2024-01-31T05:42:05.102333+0000 front 2024-01-31T05:42:05.102311+0000 (oldest deadline 2024-01-31T05:42:27.402206+0000) 2024-01-31T05:42:28.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:28 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:28.441+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:28.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:28 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:28.441+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:28.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:28 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:28.441+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:28.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:28 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:28.441+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:28.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:28 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:28.441+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:28.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:28 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:28.441+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:28.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:28 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:28.441+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-01-31T05:41:55.701753+0000 front 2024-01-31T05:41:55.701737+0000 (oldest deadline 2024-01-31T05:42:19.201524+0000) 2024-01-31T05:42:28.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:28 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:28.441+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2024-01-31T05:42:05.102333+0000 front 2024-01-31T05:42:05.102311+0000 (oldest deadline 2024-01-31T05:42:27.402206+0000) 2024-01-31T05:42:29.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:29 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:29.451+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-01-31T05:41:00.094212+0000 front 2024-01-31T05:41:00.094053+0000 (oldest deadline 2024-01-31T05:41:25.394100+0000) 2024-01-31T05:42:29.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:29 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:29.451+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-01-31T05:41:05.394534+0000 front 2024-01-31T05:41:05.394558+0000 (oldest deadline 2024-01-31T05:41:34.795175+0000) 2024-01-31T05:42:29.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:29 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:29.451+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-01-31T05:41:14.795644+0000 front 2024-01-31T05:41:14.795612+0000 (oldest deadline 2024-01-31T05:41:38.895723+0000) 2024-01-31T05:42:29.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:29 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:29.451+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-01-31T05:41:34.098515+0000 front 2024-01-31T05:41:34.098452+0000 (oldest deadline 2024-01-31T05:41:59.998420+0000) 2024-01-31T05:42:29.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:29 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:29.451+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-01-31T05:41:39.998966+0000 front 2024-01-31T05:41:39.999008+0000 (oldest deadline 2024-01-31T05:42:03.999828+0000) 2024-01-31T05:42:29.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:29 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:29.451+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-01-31T05:41:46.901087+0000 front 2024-01-31T05:41:46.901028+0000 (oldest deadline 2024-01-31T05:42:15.701168+0000) 2024-01-31T05:42:29.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:29 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:29.451+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-01-31T05:41:55.701753+0000 front 2024-01-31T05:41:55.701737+0000 (oldest deadline 2024-01-31T05:42:19.201524+0000) 2024-01-31T05:42:29.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:29 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11[61183]: 2024-01-31T05:42:29.451+0000 7f1ec6d98700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2024-01-31T05:42:05.102333+0000 front 2024-01-31T05:42:05.102311+0000 (oldest deadline 2024-01-31T05:42:27.402206+0000) 2024-01-31T05:42:30.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:30 smithi186 podman[73945]: 2024-01-31 05:42:30.304426665 +0000 UTC m=+5.464055635 container died 2e5486c92ec8683165e3fd99bbdf639bc35355f5a5d5783504a3652920a18500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-01-31T05:42:30.589 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:30 smithi186 podman[73945]: 2024-01-31 05:42:30.589867492 +0000 UTC m=+5.749496459 container cleanup 2e5486c92ec8683165e3fd99bbdf639bc35355f5a5d5783504a3652920a18500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11, org.label-schema.build-date=20240123, ceph=True, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-01-31T05:42:30.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:30 smithi186 bash[73945]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11 2024-01-31T05:42:30.842 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:30 smithi186 podman[73961]: 2024-01-31 05:42:30.824295524 +0000 UTC m=+0.517686701 container remove 2e5486c92ec8683165e3fd99bbdf639bc35355f5a5d5783504a3652920a18500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-01-31T05:42:31.468 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:31 smithi186 podman[74052]: 2024-01-31 05:42:31.048345937 +0000 UTC m=+0.017804132 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-31T05:42:31.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:31 smithi186 podman[74052]: 2024-01-31 05:42:31.150078104 +0000 UTC m=+0.119536325 container create c337919ec5e841cc7b089202eb1725be570b22d3a871a7a3eed0ed2c1bd1b8da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, io.buildah.version=1.31.3) 2024-01-31T05:42:31.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:31 smithi186 podman[74052]: 2024-01-31 05:42:31.377513311 +0000 UTC m=+0.346971492 container init c337919ec5e841cc7b089202eb1725be570b22d3a871a7a3eed0ed2c1bd1b8da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:42:31.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:31 smithi186 podman[74052]: 2024-01-31 05:42:31.38088813 +0000 UTC m=+0.350346310 container start c337919ec5e841cc7b089202eb1725be570b22d3a871a7a3eed0ed2c1bd1b8da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11-deactivate, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-01-31T05:42:31.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:31 smithi186 podman[74052]: 2024-01-31 05:42:31.469082173 +0000 UTC m=+0.438540364 container attach c337919ec5e841cc7b089202eb1725be570b22d3a871a7a3eed0ed2c1bd1b8da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:42:31.843 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:31 smithi186 podman[74052]: 2024-01-31 05:42:31.658160625 +0000 UTC m=+0.627618832 container died c337919ec5e841cc7b089202eb1725be570b22d3a871a7a3eed0ed2c1bd1b8da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2) 2024-01-31T05:42:32.171 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.11.service' 2024-01-31T05:42:32.196 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:32 smithi186 podman[74079]: 2024-01-31 05:42:32.147157774 +0000 UTC m=+0.483676951 container remove c337919ec5e841cc7b089202eb1725be570b22d3a871a7a3eed0ed2c1bd1b8da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=) 2024-01-31T05:42:32.197 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:32 smithi186 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.11.service: Deactivated successfully. 2024-01-31T05:42:32.197 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:32 smithi186 systemd[1]: Stopped Ceph osd.11 for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:42:32.197 INFO:journalctl@ceph.osd.11.smithi186.stdout:Jan 31 05:42:32 smithi186 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@osd.11.service: Consumed 1min 52.973s CPU time. 2024-01-31T05:42:32.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:42:32.212 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-01-31T05:42:32.213 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-01-31T05:42:32.213 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.a 2024-01-31T05:42:32.535 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:42:32 smithi045 systemd[1]: Stopping Ceph mds.a for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:42:32.893 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:42:32 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-a[73452]: 2024-01-31T05:42:32.535+0000 7f10b5463700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:42:32.893 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:42:32 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-a[73452]: 2024-01-31T05:42:32.535+0000 7f10b5463700 -1 mds.a *** got signal Terminated *** 2024-01-31T05:42:36.643 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:42:36 smithi045 podman[95261]: 2024-01-31 05:42:36.362322379 +0000 UTC m=+3.953239659 container died 63269242125a7a34628ea2701d23dbfb2bb73600cf4f743ba63b164af3324c6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-01-31T05:42:37.109 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:42:36 smithi045 podman[95261]: 2024-01-31 05:42:36.859278613 +0000 UTC m=+4.450195892 container cleanup 63269242125a7a34628ea2701d23dbfb2bb73600cf4f743ba63b164af3324c6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-31T05:42:37.109 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jan 31 05:42:36 smithi045 bash[95261]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-a 2024-01-31T05:42:37.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.a.service' 2024-01-31T05:42:37.275 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:42:37.275 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-01-31T05:42:37.275 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-01-31T05:42:37.275 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.d 2024-01-31T05:42:37.393 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jan 31 05:42:37 smithi045 systemd[1]: Stopping Ceph mds.d for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:42:38.143 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jan 31 05:42:37 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-d[72504]: 2024-01-31T05:42:37.721+0000 7f4b7b4ef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:42:38.144 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jan 31 05:42:37 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-d[72504]: 2024-01-31T05:42:37.721+0000 7f4b7b4ef700 -1 mds.d *** got signal Terminated *** 2024-01-31T05:42:41.643 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jan 31 05:42:41 smithi045 podman[95383]: 2024-01-31 05:42:41.332013456 +0000 UTC m=+3.838579878 container died 0500c71b12ff184f78c6313980d5abfde785f374c9c060091fabafe902911d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-d, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:42:42.046 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jan 31 05:42:41 smithi045 podman[95383]: 2024-01-31 05:42:41.79619744 +0000 UTC m=+4.302763859 container cleanup 0500c71b12ff184f78c6313980d5abfde785f374c9c060091fabafe902911d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-01-31T05:42:42.046 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jan 31 05:42:41 smithi045 bash[95383]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-d 2024-01-31T05:42:42.177 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.d.service' 2024-01-31T05:42:42.211 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:42:42.211 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-01-31T05:42:42.212 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-01-31T05:42:42.212 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.g 2024-01-31T05:42:42.393 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:42:42 smithi045 systemd[1]: Stopping Ceph mds.g for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:42:42.893 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:42:42 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-g[73099]: 2024-01-31T05:42:42.532+0000 7f8e97d5a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:42:42.893 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:42:42 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-g[73099]: 2024-01-31T05:42:42.532+0000 7f8e97d5a700 -1 mds.g *** got signal Terminated *** 2024-01-31T05:42:45.940 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:42:45 smithi045 podman[95504]: 2024-01-31 05:42:45.682194125 +0000 UTC m=+3.251834275 container died 4dc91940072917083e37d55e6a3b7dae9c582bc2850d4144915eb12d3605c2af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-g, ceph=True, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-01-31T05:42:46.299 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:42:46 smithi045 podman[95504]: 2024-01-31 05:42:46.040493827 +0000 UTC m=+3.610133979 container cleanup 4dc91940072917083e37d55e6a3b7dae9c582bc2850d4144915eb12d3605c2af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-31T05:42:46.299 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jan 31 05:42:46 smithi045 bash[95504]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-g 2024-01-31T05:42:46.468 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.g.service' 2024-01-31T05:42:46.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:42:46.502 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-01-31T05:42:46.502 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-01-31T05:42:46.503 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.j 2024-01-31T05:42:46.818 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:42:46 smithi045 systemd[1]: Stopping Ceph mds.j for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:42:47.143 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:42:46 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-j[72113]: 2024-01-31T05:42:46.818+0000 7f7634f3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:42:47.143 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:42:46 smithi045 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-j[72113]: 2024-01-31T05:42:46.818+0000 7f7634f3a700 -1 mds.j *** got signal Terminated *** 2024-01-31T05:42:50.752 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:42:50 smithi045 podman[95624]: 2024-01-31 05:42:50.396371871 +0000 UTC m=+3.674777153 container died aa11c2272224b2e3f2e5008710ffd818b484f3da8d95fa2874a1b9d47eaec252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-j, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123) 2024-01-31T05:42:51.023 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:42:50 smithi045 podman[95624]: 2024-01-31 05:42:50.843986046 +0000 UTC m=+4.122391329 container cleanup aa11c2272224b2e3f2e5008710ffd818b484f3da8d95fa2874a1b9d47eaec252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-j, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-01-31T05:42:51.023 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jan 31 05:42:50 smithi045 bash[95624]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-j 2024-01-31T05:42:51.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.j.service' 2024-01-31T05:42:51.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:42:51.263 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-01-31T05:42:51.263 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-01-31T05:42:51.264 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.b 2024-01-31T05:42:51.635 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:42:51 smithi156 systemd[1]: Stopping Ceph mds.b for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:42:51.976 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:42:51 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b[66600]: 2024-01-31T05:42:51.634+0000 7f9a957c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:42:51.977 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:42:51 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b[66600]: 2024-01-31T05:42:51.634+0000 7f9a957c9700 -1 mds.b *** got signal Terminated *** 2024-01-31T05:42:53.951 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:42:53 smithi156 podman[75983]: 2024-01-31 05:42:53.677006869 +0000 UTC m=+2.219283274 container died d817dbe780f46cafc31cd0710c2390c59de6d0a1560ed518253e9fe27fbb43f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-01-31T05:42:54.226 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:42:54 smithi156 podman[75983]: 2024-01-31 05:42:54.051604234 +0000 UTC m=+2.593880633 container cleanup d817dbe780f46cafc31cd0710c2390c59de6d0a1560ed518253e9fe27fbb43f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.31.3) 2024-01-31T05:42:54.227 INFO:journalctl@ceph.mds.b.smithi156.stdout:Jan 31 05:42:54 smithi156 bash[75983]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-b 2024-01-31T05:42:54.476 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.b.service' 2024-01-31T05:42:54.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:42:54.514 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-01-31T05:42:54.514 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-01-31T05:42:54.514 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.e 2024-01-31T05:42:54.977 INFO:journalctl@ceph.mds.e.smithi156.stdout:Jan 31 05:42:54 smithi156 systemd[1]: Stopping Ceph mds.e for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:42:54.977 INFO:journalctl@ceph.mds.e.smithi156.stdout:Jan 31 05:42:54 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-e[65977]: 2024-01-31T05:42:54.813+0000 7f5e3372c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:42:54.977 INFO:journalctl@ceph.mds.e.smithi156.stdout:Jan 31 05:42:54 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-e[65977]: 2024-01-31T05:42:54.813+0000 7f5e3372c700 -1 mds.e *** got signal Terminated *** 2024-01-31T05:42:57.946 INFO:journalctl@ceph.mds.e.smithi156.stdout:Jan 31 05:42:57 smithi156 podman[76116]: 2024-01-31 05:42:57.68602948 +0000 UTC m=+2.958553763 container died d7a87260df21645966a577a33deb371cccf5397eabae27816377b5bf85f96b7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240123, GIT_CLEAN=True) 2024-01-31T05:42:58.227 INFO:journalctl@ceph.mds.e.smithi156.stdout:Jan 31 05:42:58 smithi156 podman[76116]: 2024-01-31 05:42:58.05514135 +0000 UTC m=+3.327665610 container cleanup d7a87260df21645966a577a33deb371cccf5397eabae27816377b5bf85f96b7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, io.buildah.version=1.31.3) 2024-01-31T05:42:58.227 INFO:journalctl@ceph.mds.e.smithi156.stdout:Jan 31 05:42:58 smithi156 bash[76116]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-e 2024-01-31T05:42:58.479 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.e.service' 2024-01-31T05:42:58.514 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:42:58.514 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-01-31T05:42:58.515 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-01-31T05:42:58.515 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h 2024-01-31T05:42:58.585 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.h.service' 2024-01-31T05:42:58.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:42:58.659 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-01-31T05:42:58.660 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-01-31T05:42:58.660 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.k 2024-01-31T05:42:58.977 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:42:58 smithi156 systemd[1]: Stopping Ceph mds.k for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:42:59.477 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:42:59 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-k[65651]: 2024-01-31T05:42:59.083+0000 7fbd10e61700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:42:59.477 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:42:59 smithi156 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-k[65651]: 2024-01-31T05:42:59.083+0000 7fbd10e61700 -1 mds.k *** got signal Terminated *** 2024-01-31T05:43:03.358 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:43:03 smithi156 podman[76272]: 2024-01-31 05:43:03.090138165 +0000 UTC m=+4.219641922 container died 6ee9d87f4ebf0e4dd5e470a266d9d360745b12d0e80b25687103d58f94086268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-k, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-01-31T05:43:03.727 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:43:03 smithi156 podman[76272]: 2024-01-31 05:43:03.450061426 +0000 UTC m=+4.579565169 container cleanup 6ee9d87f4ebf0e4dd5e470a266d9d360745b12d0e80b25687103d58f94086268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-01-31T05:43:03.727 INFO:journalctl@ceph.mds.k.smithi156.stdout:Jan 31 05:43:03 smithi156 bash[76272]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-k 2024-01-31T05:43:03.861 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.k.service' 2024-01-31T05:43:03.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:43:03.898 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-01-31T05:43:03.898 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-01-31T05:43:03.899 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.c 2024-01-31T05:43:04.205 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:43:03 smithi186 systemd[1]: Stopping Ceph mds.c for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:43:04.592 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:43:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-c[66673]: 2024-01-31T05:43:04.204+0000 7f524f3db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:43:04.592 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:43:04 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-c[66673]: 2024-01-31T05:43:04.204+0000 7f524f3db700 -1 mds.c *** got signal Terminated *** 2024-01-31T05:43:06.114 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:43:05 smithi186 podman[74158]: 2024-01-31 05:43:05.856227385 +0000 UTC m=+1.767983937 container died 3505dd9563489f0a7f8f57be0f49288b401c8d2de16f1c142924ac179ecb1f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-c, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-01-31T05:43:06.505 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:43:06 smithi186 podman[74158]: 2024-01-31 05:43:06.213212159 +0000 UTC m=+2.124968711 container cleanup 3505dd9563489f0a7f8f57be0f49288b401c8d2de16f1c142924ac179ecb1f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-c, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-01-31T05:43:06.505 INFO:journalctl@ceph.mds.c.smithi186.stdout:Jan 31 05:43:06 smithi186 bash[74158]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-c 2024-01-31T05:43:06.637 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.c.service' 2024-01-31T05:43:06.676 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:43:06.676 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-01-31T05:43:06.676 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-01-31T05:43:06.676 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.f 2024-01-31T05:43:06.842 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:43:06 smithi186 systemd[1]: Stopping Ceph mds.f for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:43:07.342 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:43:06 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f[66358]: 2024-01-31T05:43:06.992+0000 7fd516f9a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:43:07.342 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:43:06 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f[66358]: 2024-01-31T05:43:06.992+0000 7fd516f9a700 -1 mds.f *** got signal Terminated *** 2024-01-31T05:43:10.134 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:43:09 smithi186 podman[74290]: 2024-01-31 05:43:09.852666757 +0000 UTC m=+2.961202113 container died c7d29b680cd7134ccf239ed653a97535ef164642e1096d6765fe089ef3623c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240123) 2024-01-31T05:43:10.467 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:43:10 smithi186 podman[74290]: 2024-01-31 05:43:10.216864565 +0000 UTC m=+3.325399914 container cleanup c7d29b680cd7134ccf239ed653a97535ef164642e1096d6765fe089ef3623c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-01-31T05:43:10.467 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:43:10 smithi186 bash[74290]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f 2024-01-31T05:43:10.583 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:43:10 smithi186 podman[74304]: 2024-01-31 05:43:10.467975665 +0000 UTC m=+0.610613902 container remove c7d29b680cd7134ccf239ed653a97535ef164642e1096d6765fe089ef3623c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-f, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-31T05:43:10.583 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:43:10 smithi186 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.f.service: Deactivated successfully. 2024-01-31T05:43:10.583 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:43:10 smithi186 systemd[1]: Stopped Ceph mds.f for 3a1f4122-bff7-11ee-95b3-87774f69a715. 2024-01-31T05:43:10.583 INFO:journalctl@ceph.mds.f.smithi186.stdout:Jan 31 05:43:10 smithi186 systemd[1]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.f.service: Consumed 17.043s CPU time. 2024-01-31T05:43:10.584 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.f.service' 2024-01-31T05:43:10.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:43:10.619 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-01-31T05:43:10.620 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-01-31T05:43:10.620 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.i 2024-01-31T05:43:10.842 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:43:10 smithi186 systemd[1]: Stopping Ceph mds.i for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:43:11.342 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:43:10 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-i[66979]: 2024-01-31T05:43:10.928+0000 7f47f0818700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:43:11.342 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:43:10 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-i[66979]: 2024-01-31T05:43:10.928+0000 7f47f0818700 -1 mds.i *** got signal Terminated *** 2024-01-31T05:43:14.245 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:43:13 smithi186 podman[74409]: 2024-01-31 05:43:13.860098138 +0000 UTC m=+3.028712455 container died d1b57c01fe3a4fcf76412ccba4e0b30ae19741e237dde3b8452697288d188d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-i, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-31T05:43:14.587 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:43:14 smithi186 podman[74409]: 2024-01-31 05:43:14.328339888 +0000 UTC m=+3.496954181 container cleanup d1b57c01fe3a4fcf76412ccba4e0b30ae19741e237dde3b8452697288d188d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-i, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-01-31T05:43:14.587 INFO:journalctl@ceph.mds.i.smithi186.stdout:Jan 31 05:43:14 smithi186 bash[74409]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-i 2024-01-31T05:43:14.747 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.i.service' 2024-01-31T05:43:14.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:43:14.781 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-01-31T05:43:14.781 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-01-31T05:43:14.782 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.l 2024-01-31T05:43:15.098 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:43:14 smithi186 systemd[1]: Stopping Ceph mds.l for 3a1f4122-bff7-11ee-95b3-87774f69a715... 2024-01-31T05:43:15.592 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:43:15 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l[66049]: 2024-01-31T05:43:15.098+0000 7f1a420bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-31T05:43:15.592 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:43:15 smithi186 ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l[66049]: 2024-01-31T05:43:15.098+0000 7f1a420bb700 -1 mds.l *** got signal Terminated *** 2024-01-31T05:43:18.148 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:43:17 smithi186 podman[74531]: 2024-01-31 05:43:17.854700375 +0000 UTC m=+2.863317939 container died ab8db6740b0a6ede1119f591d4bb07229546071db57634f6c40003321c5bd783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l, org.label-schema.build-date=20240123, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-31T05:43:18.148 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:43:18 smithi186 podman[74531]: 2024-01-31 05:43:18.148603956 +0000 UTC m=+3.157221509 container cleanup ab8db6740b0a6ede1119f591d4bb07229546071db57634f6c40003321c5bd783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-01-31T05:43:18.450 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:43:18 smithi186 bash[74531]: ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l 2024-01-31T05:43:18.450 INFO:journalctl@ceph.mds.l.smithi186.stdout:Jan 31 05:43:18 smithi186 podman[74546]: 2024-01-31 05:43:18.341196868 +0000 UTC m=+0.487724506 container remove ab8db6740b0a6ede1119f591d4bb07229546071db57634f6c40003321c5bd783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-3a1f4122-bff7-11ee-95b3-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-31T05:43:18.462 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a1f4122-bff7-11ee-95b3-87774f69a715@mds.l.service' 2024-01-31T05:43:18.496 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-31T05:43:18.497 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-01-31T05:43:18.497 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 --force --keep-logs 2024-01-31T05:43:18.695 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:43:21.201 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 --force --keep-logs 2024-01-31T05:43:21.393 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:43:23.922 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 --force --keep-logs 2024-01-31T05:43:24.112 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:43:26.474 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-31T05:43:26.508 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-31T05:43:26.542 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-31T05:43:26.571 INFO:tasks.cephadm:Archiving crash dumps... 2024-01-31T05:43:26.572 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495/remote/smithi045/crash 2024-01-31T05:43:26.573 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/crash -- . 2024-01-31T05:43:26.614 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/crash: Cannot open: No such file or directory 2024-01-31T05:43:26.614 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2024-01-31T05:43:26.616 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495/remote/smithi156/crash 2024-01-31T05:43:26.618 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/crash -- . 2024-01-31T05:43:26.678 INFO:teuthology.orchestra.run.smithi156.stderr:tar: /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/crash: Cannot open: No such file or directory 2024-01-31T05:43:26.679 INFO:teuthology.orchestra.run.smithi156.stderr:tar: Error is not recoverable: exiting now 2024-01-31T05:43:26.680 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495/remote/smithi186/crash 2024-01-31T05:43:26.681 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/crash -- . 2024-01-31T05:43:26.713 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/crash: Cannot open: No such file or directory 2024-01-31T05:43:26.713 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2024-01-31T05:43:26.715 INFO:tasks.cephadm:Checking cluster log for badness... 2024-01-31T05:43:26.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-01-31T05:43:26.748 INFO:teuthology.orchestra.run.smithi045.stderr:grep: /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph.log: No such file or directory 2024-01-31T05:43:26.750 INFO:tasks.cephadm:Compressing logs... 2024-01-31T05:43:26.750 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-31T05:43:26.792 DEBUG:teuthology.orchestra.run.smithi156:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-31T05:43:26.795 DEBUG:teuthology.orchestra.run.smithi186:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-31T05:43:26.818 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-01-31T05:43:26.819 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-01-31T05:43:26.819 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.6.log 2024-01-31T05:43:26.822 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mon.a.log 2024-01-31T05:43:26.822 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.g.log 2024-01-31T05:43:26.822 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.j.log 2024-01-31T05:43:26.822 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.3.log 2024-01-31T05:43:26.823 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mgr.x.log 2024-01-31T05:43:26.823 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.0.log 2024-01-31T05:43:26.823 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-volume.log 2024-01-31T05:43:26.824 INFO:teuthology.orchestra.run.smithi156.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-01-31T05:43:26.824 INFO:teuthology.orchestra.run.smithi156.stderr:: No such file or directory 2024-01-31T05:43:26.824 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.10.log 2024-01-31T05:43:26.825 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.b.log 2024-01-31T05:43:26.825 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.4.log 2024-01-31T05:43:26.825 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.1.log 2024-01-31T05:43:26.827 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.b.log: /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.k.log 2024-01-31T05:43:26.827 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.1.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-01-31T05:43:26.827 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.h.log 2024-01-31T05:43:26.827 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.7.log 2024-01-31T05:43:26.827 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.e.log 2024-01-31T05:43:26.827 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mgr.y.log 2024-01-31T05:43:26.833 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.0.log: 94.6% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.g.log.gz 2024-01-31T05:43:26.833 INFO:teuthology.orchestra.run.smithi045.stderr: 94.6% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.j.log.gz 2024-01-31T05:43:26.833 INFO:teuthology.orchestra.run.smithi045.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-01-31T05:43:26.834 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.k.log.gz 2024-01-31T05:43:26.834 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-volume.log 2024-01-31T05:43:26.838 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.9.log 2024-01-31T05:43:26.838 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.d.log 2024-01-31T05:43:26.839 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.a.log 2024-01-31T05:43:26.840 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-01-31T05:43:26.841 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-01-31T05:43:26.841 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.l.log 2024-01-31T05:43:26.841 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.i.log 2024-01-31T05:43:26.841 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.11.log 2024-01-31T05:43:26.841 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mgr.z.log 2024-01-31T05:43:26.841 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.8.log 2024-01-31T05:43:26.841 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.5.log 2024-01-31T05:43:26.841 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.f.log 2024-01-31T05:43:26.842 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.2.log 2024-01-31T05:43:26.842 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.c.log 2024-01-31T05:43:26.842 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-volume.log 2024-01-31T05:43:26.842 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mon.c.log 2024-01-31T05:43:26.843 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.d.log.gz 2024-01-31T05:43:26.843 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mon.b.log 2024-01-31T05:43:26.844 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-volume.log: 92.2% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mgr.y.log.gz 2024-01-31T05:43:26.848 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-volume.log: /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mon.c.log: 92.3% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mgr.z.log.gz 2024-01-31T05:43:26.849 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.a.log: 94.5% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.a.log.gz 2024-01-31T05:43:26.850 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-volume.log.gz 2024-01-31T05:43:26.854 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-volume.log.gz 2024-01-31T05:43:26.864 INFO:teuthology.orchestra.run.smithi186.stderr: 92.2% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.l.log.gz 2024-01-31T05:43:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-01-31T05:43:27.027 INFO:teuthology.orchestra.run.smithi186.stderr: 91.0% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-volume.log.gz 2024-01-31T05:43:27.246 INFO:teuthology.orchestra.run.smithi045.stderr: 89.8% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mgr.x.log.gz 2024-01-31T05:43:27.620 INFO:teuthology.orchestra.run.smithi156.stderr: 94.3% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.h.log.gz 2024-01-31T05:43:28.038 INFO:teuthology.orchestra.run.smithi045.stderr: 90.4% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mon.a.log.gz 2024-01-31T05:43:28.697 INFO:teuthology.orchestra.run.smithi156.stderr: 93.5% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mon.b.log.gz 2024-01-31T05:43:34.348 INFO:teuthology.orchestra.run.smithi186.stderr: 91.9% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mon.c.log.gz 2024-01-31T05:43:34.971 INFO:teuthology.orchestra.run.smithi156.stderr: 94.4% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.4.log.gz 2024-01-31T05:43:36.276 INFO:teuthology.orchestra.run.smithi045.stderr: 94.3% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.9.log.gz 2024-01-31T05:43:36.882 INFO:teuthology.orchestra.run.smithi045.stderr: 94.0% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.0.log.gz 2024-01-31T05:43:37.867 INFO:teuthology.orchestra.run.smithi045.stderr: 94.2% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.6.log.gz 2024-01-31T05:43:38.044 INFO:teuthology.orchestra.run.smithi045.stderr: 94.2% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.3.log.gz 2024-01-31T05:43:38.046 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-01-31T05:43:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m11.240s 2024-01-31T05:43:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m41.999s 2024-01-31T05:43:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m1.421s 2024-01-31T05:43:40.693 INFO:teuthology.orchestra.run.smithi156.stderr: 94.0% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.1.log.gz 2024-01-31T05:43:40.944 INFO:teuthology.orchestra.run.smithi156.stderr: 94.2% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.7.log.gz 2024-01-31T05:43:41.564 INFO:teuthology.orchestra.run.smithi156.stderr: 94.3% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.10.log.gz 2024-01-31T05:43:46.116 INFO:teuthology.orchestra.run.smithi156.stderr: 95.1% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.e.log.gz 2024-01-31T05:43:46.960 INFO:teuthology.orchestra.run.smithi156.stderr: 94.9% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.b.log.gz 2024-01-31T05:43:46.962 INFO:teuthology.orchestra.run.smithi156.stderr: 2024-01-31T05:43:46.963 INFO:teuthology.orchestra.run.smithi156.stderr:real 0m20.154s 2024-01-31T05:43:46.963 INFO:teuthology.orchestra.run.smithi156.stderr:user 1m29.768s 2024-01-31T05:43:46.963 INFO:teuthology.orchestra.run.smithi156.stderr:sys 0m3.054s 2024-01-31T05:44:33.705 INFO:teuthology.orchestra.run.smithi186.stderr: 95.2% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.f.log.gz 2024-01-31T05:44:46.024 INFO:teuthology.orchestra.run.smithi186.stderr: 94.0% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.11.log.gz 2024-01-31T05:44:48.429 INFO:teuthology.orchestra.run.smithi186.stderr: 94.4% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.2.log.gz 2024-01-31T05:44:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: 94.4% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.8.log.gz 2024-01-31T05:45:19.791 INFO:teuthology.orchestra.run.smithi186.stderr: 94.3% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-osd.5.log.gz 2024-01-31T05:49:15.960 INFO:teuthology.orchestra.run.smithi186.stderr: 93.5% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.i.log.gz 2024-01-31T05:55:07.975 INFO:teuthology.orchestra.run.smithi186.stderr: 93.3% -- replaced with /var/log/ceph/3a1f4122-bff7-11ee-95b3-87774f69a715/ceph-mds.c.log.gz 2024-01-31T05:55:08.029 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-01-31T05:55:08.029 INFO:teuthology.orchestra.run.smithi186.stderr:real 11m41.220s 2024-01-31T05:55:08.029 INFO:teuthology.orchestra.run.smithi186.stderr:user 8m27.458s 2024-01-31T05:55:08.029 INFO:teuthology.orchestra.run.smithi186.stderr:sys 0m21.307s 2024-01-31T05:55:08.030 INFO:tasks.cephadm:Archiving logs... 2024-01-31T05:55:08.030 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495/remote/smithi045/log 2024-01-31T05:55:08.032 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-31T05:55:13.392 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495/remote/smithi156/log 2024-01-31T05:55:13.394 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-31T05:55:23.165 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495/remote/smithi186/log 2024-01-31T05:55:23.166 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-31T05:56:42.222 INFO:tasks.cephadm:Removing cluster... 2024-01-31T05:56:42.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 --force 2024-01-31T05:56:42.421 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:56:42.726 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 --force 2024-01-31T05:56:42.919 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:56:43.266 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a1f4122-bff7-11ee-95b3-87774f69a715 --force 2024-01-31T05:56:43.713 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 3a1f4122-bff7-11ee-95b3-87774f69a715 2024-01-31T05:56:44.376 INFO:tasks.cephadm:Removing cephadm ... 2024-01-31T05:56:44.376 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2024-01-31T05:56:44.395 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf /home/ubuntu/cephtest/cephadm 2024-01-31T05:56:44.410 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/cephadm 2024-01-31T05:56:44.425 INFO:tasks.cephadm:Teardown complete 2024-01-31T05:56:44.426 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-31T05:56:44.437 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-31T05:56:44.437 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-31T05:56:44.440 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-31T05:56:44.453 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-31T05:56:44.574 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-31T05:56:44.584 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-31T05:56:44.584 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-31T05:56:44.587 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-31T05:56:44.589 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-31T05:56:44.600 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2024-01-31T05:56:44.600 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-01-31T05:56:44.603 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-01-31T05:56:44.603 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-31T05:56:44.604 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2024-01-31T05:56:44.604 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 +641us[ +642us] +/- 80ms 2024-01-31T05:56:44.604 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 -685us[ -684us] +/- 74ms 2024-01-31T05:56:44.604 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +132us[ +133us] +/- 75ms 2024-01-31T05:56:44.604 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:56:44.605 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-31T05:56:44.606 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-01-31T05:56:44.606 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 41 +647us[ +651us] +/- 80ms 2024-01-31T05:56:44.606 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -694us[ -689us] +/- 74ms 2024-01-31T05:56:44.606 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +127us[ +127us] +/- 75ms 2024-01-31T05:56:44.606 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:56:44.705 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-31T05:56:44.705 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-01-31T05:56:44.705 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 30 +619us[ +619us] +/- 80ms 2024-01-31T05:56:44.705 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv02.front.sepia.ceph.com 3 6 377 31 -706us[ -731us] +/- 74ms 2024-01-31T05:56:44.705 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +118us[ +93us] +/- 75ms 2024-01-31T05:56:44.705 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:56:44.707 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-31T05:56:44.717 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-31T05:56:44.718 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-31T05:56:44.752 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-31T05:56:44.837 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-31T05:56:44.908 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-31T05:56:45.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-31T05:56:45.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:56:45.047 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-31T05:56:45.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:56:45.077 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-31T05:56:45.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:56:45.107 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-31T05:56:45.117 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-31T05:56:45.151 INFO:teuthology.task.internal:Duration was 3114.101925 seconds 2024-01-31T05:56:45.152 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-31T05:56:45.202 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-31T05:56:45.202 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-31T05:56:45.204 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-31T05:56:45.207 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-31T05:56:45.247 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-31T05:56:45.250 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-31T05:56:45.285 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-31T05:56:45.754 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-31T05:56:45.754 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-01-31T05:56:45.754 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-31T05:56:45.776 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2024-01-31T05:56:45.776 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-31T05:56:45.819 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2024-01-31T05:56:45.820 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-31T05:56:45.839 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-31T05:56:45.839 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-31T05:56:45.842 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-31T05:56:45.862 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-31T05:56:45.916 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-31T05:56:45.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-31T05:56:45.918 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-31T05:56:45.930 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-31T05:56:46.329 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-31T05:56:46.340 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-31T05:56:46.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-31T05:56:46.367 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-31T05:56:46.394 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-31T05:56:46.423 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-31T05:56:46.434 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-31T05:56:46.436 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-31T05:56:46.439 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-31T05:56:46.459 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-01-31T05:56:46.461 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2024-01-31T05:56:46.502 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2024-01-31T05:56:46.528 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-31T05:56:46.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:56:46.542 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-31T05:56:46.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:56:46.556 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-31T05:56:46.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:56:46.583 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-31T05:56:46.594 INFO:teuthology.task.internal:Transferring archived files... 2024-01-31T05:56:46.594 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495/remote/smithi045 2024-01-31T05:56:46.595 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-31T05:56:46.638 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495/remote/smithi156 2024-01-31T05:56:46.638 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-31T05:56:46.675 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495/remote/smithi186 2024-01-31T05:56:46.675 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-31T05:56:46.712 INFO:teuthology.task.internal:Removing archive directory... 2024-01-31T05:56:46.713 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-31T05:56:46.715 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-31T05:56:46.717 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-31T05:56:46.761 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-31T05:56:46.774 INFO:teuthology.task.internal:Not uploading archives. 2024-01-31T05:56:46.774 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-31T05:56:46.813 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-31T05:56:46.813 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-31T05:56:46.816 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-31T05:56:46.818 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-31T05:56:46.828 INFO:teuthology.orchestra.run.smithi045.stdout: 265695 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 31 05:56 /home/ubuntu/cephtest 2024-01-31T05:56:46.830 INFO:teuthology.orchestra.run.smithi156.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 31 05:56 /home/ubuntu/cephtest 2024-01-31T05:56:46.832 INFO:teuthology.orchestra.run.smithi186.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 31 05:56 /home/ubuntu/cephtest 2024-01-31T05:56:46.833 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-31T05:56:46.860 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-31T05:56:46.963 INFO:teuthology.nuke:Checking targets against current locks 2024-01-31T05:56:46.987 DEBUG:teuthology.nuke:shortname: smithi045 2024-01-31T05:56:46.987 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-31T05:56:47.009 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-31 04:55:51.352273', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNSxe+NPNLn2hNrNVy6jyPJzBOyr5pUwBsl+J1EMRy/k41qx1/zLWtQoT+TEgppHcHPgn2ctwSeUCDXAQJjpL0Q='} 2024-01-31T05:56:47.015 DEBUG:teuthology.nuke:shortname: smithi156 2024-01-31T05:56:47.015 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-31T05:56:47.033 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-31 04:55:51.355689', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMHl72PSsbKgQJaUREZyo9KxvIc+cVgWoxkXfYzE0Zxg5U7AvqHorr/G8Yk+4nL75wm6Pvb3oCKBkTNjb/O6drg='} 2024-01-31T05:56:47.037 DEBUG:teuthology.nuke:shortname: smithi186 2024-01-31T05:56:47.037 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-31T05:56:47.051 INFO:teuthology.orchestra.console.smithi045:Power off 2024-01-31T05:56:47.051 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-31T05:56:47.062 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539495', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-31 04:55:51.354172', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC9j+A9OPXsHtLkVf29d1PbaVSgqkap1n/t4GAgqgP/OQcNQ8qF93XQFRqQO/n7pX/WRFarKi1NFu1L3JuBotS4='} 2024-01-31T05:56:47.075 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off 2024-01-31T05:56:47.075 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:56:47.097 INFO:teuthology.orchestra.console.smithi156:Power off 2024-01-31T05:56:47.097 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-31T05:56:47.108 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-01-31T05:56:47.123 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2024-01-31T05:56:47.124 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:56:47.138 INFO:teuthology.orchestra.console.smithi186:Power off 2024-01-31T05:56:47.138 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-31T05:56:47.146 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-01-31T05:56:47.160 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2024-01-31T05:56:47.160 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:56:47.180 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-01-31T05:56:51.110 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:56:51.147 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:56:51.181 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:56:51.222 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-01-31T05:56:51.256 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-01-31T05:56:51.291 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-01-31T05:56:55.224 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:56:55.257 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:56:55.292 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:56:55.334 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2024-01-31T05:56:55.367 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-01-31T05:56:55.401 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2024-01-31T05:56:55.436 INFO:teuthology.orchestra.console.smithi045:Power off completed 2024-01-31T05:56:55.502 INFO:teuthology.orchestra.console.smithi186:Power off completed 2024-01-31T05:56:59.368 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:56:59.478 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2024-01-31T05:56:59.579 INFO:teuthology.orchestra.console.smithi156:Power off completed 2024-01-31T05:56:59.686 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 3114.1019246578217 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi045 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=661946e7044723130c03627323f6a879a9848cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=2d6a28534a334789bad205b1b525d50a status: fail success: false 2024-01-31T05:56:59.687 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-31T05:56:59.770 INFO:teuthology.run:FAIL